Debugging Nginx Errors

Dealing with errors in nginx can be a frustrating experience if nginx isn’t configured correctly.

Sadly, the default value for error log is less than optimal, and some of the tricks to getting information from nginx are not obvious.

This post is intended to be a reference for the tools nginx provide and how to configure them; as well as a general guide on what’s important when facing issues in nginx.

Knowing the basics is the most important part to understanding the more difficult issues.

Don’t skip basic nginx syntax or how an nginx request flows. Once you understand these concepts the non-error issues become far more tangible to work with.

The Error Log

For issues where there’s an error involved, having nginx configured correctly is absolutely essential. The error_log directive should be configured with an error log level of warn, either at server or http level depending on whether you want per-vhost logs or server wide logs.

error_log /var/log/nginx/file.log warn;

Anything other than this will just waste your time, either by giving too much information or not enough. With data in our error log lets analyse one of the potential errors that might show up. For this example I will use the often encountered “Unable to open primary script” error.

2013/06/03 12:56:12 [error] 2641#0: *20499586 FastCGI sent in stderr: "Unable to open primary script: /home/user/public_html/wp-login.php (No such file or directory)" while reading response header from upstream, client: 10.0.0.1, server: www.example.com, request: "GET //wp-login.php HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.example.com"

If the error looks daunting then not to worry, most of it can be mentally discarded after just a brief glance. Essentially, all the important information here can be distilled down to this.

2013/06/03 FastCGI sent "Unable to open primary script: /home/user/public_html/wp-login.php" request: "GET //wp-login.php"

What’s important here is first the date to know if it’s a recent error. If you’re monitoring your error log and get notice on new entries this can even be discarded. 2nd is the source of the error, our FastCGI backend in this case. The 3rd bit is the actual error and the 4th final bit is the request that caused it. These are all the essentials we need to.

1. Know where the error came from.
2. Know where we can most likely reproduce it.
3. Which words to Google for.

Handling multiple error logs

It’s common pratice in nginx to have many different error logs. For example the error log is often split per vhost and over servers. If you run a live site do yourself a favour and setup log aggregation to have one easy place to search for issues. If you want to save some time dealing with this there are services like Loggly that handle it for you.

Getting Errors from the Backend

If your backend outputs error messages to stderr then nginx will put these in the nginx error log as well. In the case of PHP this can be done by configuring the php.ini with log_errors set to stderr or on and having no error_log defined. Finally, you also have to set catch_workers_output = yes in the php-fpm.conf file.

For other backends the procedure might be different but the concept the same, make sure errors are output to stderr and nginx will log them.

Getting Debug Output

Despite experience, sometimes understanding what nginx is doing can be quite difficult. Turning on debug mode is usually quite an adventure and often not needed at all with a few handy tricks. Often, all you need is to know the value of a variable to nudge you in the right direction, for those situations we have the return directive.

What this directive allows us to do is output arbitrary text in any server or location block — and it supports variables. This is useful if you want to know if your request is flowing into a location like you expect it to or if you need to know the value of some variable. For instance, to understand what URI you’re asking PHP to execute you could do:

return 200 $request_filename;

It is also possible to output multiple variables in the same return directive.

return 200 $document_root-$fastcgi_script_name;

Finally, you can specify a custom log_format for the access_log, which can be useful if you can’t output debug info to the browser. I find this more cumbersome and usually prefer to have a proper testing environment, but it can do in a pinch.

My Book

If you liked the style of this blog post then check out my nginx book – Instant Nginx Starter. It introduces you to the syntax and flow of nginx and minimizes your time spent debugging errors.