Debugging requests with cURL

For more than one time I had to debug HTTP request or response headers and other details. To do that, I use two techniques, both based on cURL library. I explain them ahead.

Technique #1: From the command line

This is the easiest way to debug. It doesn’t require writing any actual code. Just call curl program from the command line, as usual, adding a new param: -vvv. This will enable the highest verbosity level.

$ curl -vvv http://google.com
* Rebuilt URL to: http://google.com/
* Trying 2800:3f0:4001:802::200e...
* Connected to google.com (2800:3f0:4001:802::200e) port 80 (#0)
> GET / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html; charset=UTF-8
< Location: http://www.google.com.br/?gfe_rd=cr&ei=bUG8V53JGcvK8gfp3L-YBg
< Content-Length: 262
< Date: Tue, 23 Aug 2016 12:28:29 GMT
< 
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.com.br/?gfe_rd=cr&amp;ei=bUG8V53JGcvK8gfp3L-YBg">here</A>.
</BODY></HTML>
* Connection #0 to host google.com left intact

As you can see in the example above, it outputs all request and response info.

It’s possible to output everything to a file, by adding > output_file.txt to the end of the command. Using our previous call:

$ curl -vvv http://google.com > output.txt

Well, one may now ask: if this is so easy, why do you have a second way to debug request? Following we’ll see why that.

Technique #2: From a PHP script

I’ve written on debugging cURL and PHP at Kettle.io Blog. Let’s say you have to send a dynamic header with the request, like a JWT authorization token. It’s not impossible to that from the command line, but it’s easier using programming. For those cases, I use the cURL PHP extension. Check out the script below.

$url = 'http://google.com';
$headers = [
    'Accept' => 'application/json',
];

/*
 * We're going to use the output buffer to store the debug info.
 */
ob_start();
$out = fopen('php://output', 'w');

$handler = curl_init($url);

/*
 * Here we set the library verbosity and redirect the error output to the 
 * output buffer.
 */
curl_setopt($handler, CURLOPT_VERBOSE, true);
curl_setopt($handler, CURLOPT_STDERR, $out);

$requestHeaders = [];
foreach ($headers as $k => $v) {
    $requestHeaders[] = $k . ': ' . $v;
}
curl_setopt($handler, CURLOPT_HTTPHEADER, $requestHeaders);
curl_setopt($handler, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($handler);
fclose($out);

/*
 * Joining debug info and response body.
 */
$data = ob_get_clean();
$data .= PHP_EOL . $response . PHP_EOL;
echo $data;

Now, you can customize this code to add some dynamic data to a header or any other request part. After doing that, run it using the PHP program from the command line:

$ php curldebug.php

P.S.: I’m assuming that you saved the script as curldebug.php.

As we did with the curl program, it’s possible to output everything to a file. Just append the > output_file.txt to the call.

Conclusion

Debugging requests can be a lifesaver when dealing with third-party APIs and other services. Headers may contain helpful info to find what is going wrong with that weird response body.

Advertisements

Declarando arrays em shell script, no bash

Apenas uma referência básica de declaração de arrays em shell script, no bash. Em relação ao uso, existem várias formas e possibilidades.

Exemplo com strings:

nomes=( "Esse" "Aquele" "Outro" )

Exemplo com inteiros:

ids=( 1 2 3 4 5 6 7 )

Recursive chmod

Sometimes you need to change the access control to files matching some requirement. E.g. today I needed grant execute access to the owner of all PHP scripts running under Apache HTTP Server, in a specific directory.

Continue reading “Recursive chmod”

Writing an init script for a Java application

UPDATE, Oct, 1st 2014

This is an old post (from 5 years ago when I writing this update) and the solution isn’t the best. So take a look at the comments – specially that one by Tinoco.


The need make us study topics we think we’ll never need. Today I had to write a simple bash program to start (and stop) a Java application, a simple init script.

After much searching, I’ve found some simple explanations for some specific points of the bash script syntax.

I don’t know who is on the right side of this story, but seems that professionals who write manuals and tutorials for this language doesn’t think like common web programmers, such as Java or PHP programmers. I sought to much on Google to find things like how I could associate the returning value of a function to a variable.

But finally, I’ll leave aside the claims and move to example:

Continue reading “Writing an init script for a Java application”