502 Bad Gateway troubleshooting in Nginx - Commands & Configuration

Introduction

A 502 Bad Gateway error happens when a server acting as a gateway or proxy gets an invalid response from the upstream server. This error stops users from accessing the website or service. Troubleshooting helps find and fix the cause quickly.

When your website shows a 502 error instead of loading normally

When your Nginx server is set up as a reverse proxy but the backend app is not responding

When you recently changed backend server settings and users report connection problems

When you want to check if your backend service is running and reachable

When you want to verify Nginx configuration for proxying requests

Commands

Check if the Nginx service is running properly on your server.

Terminal

systemctl status nginx

Expected Output

● nginx.service - A high performance web server and a reverse proxy server

   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)

   Active: active (running) since Fri 2024-06-14 10:00:00 UTC; 5min ago

 Main PID: 1234 (nginx)

    Tasks: 3 (limit: 1152)

   Memory: 5.0M

   CGroup: /system.slice/nginx.service

           ├─1234 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;

           ├─1235 nginx: worker process

           └─1236 nginx: worker process

Test if the backend server (running on port 8080) is responding correctly.

Terminal

curl -I http://localhost:8080

Expected Output

HTTP/1.1 200 OK

Server: ExampleBackend

Content-Type: text/html

Content-Length: 612

Connection: keep-alive

Check Nginx configuration syntax to ensure there are no errors causing the 502 error.

Terminal

nginx -t

Expected Output

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

View the last 20 lines of the Nginx error log to find clues about the 502 error.

Terminal

tail -n 20 /var/log/nginx/error.log

Expected Output

2024/06/14 10:05:00 [error] 1235#1235: *45 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.10, server: example.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "example.com"

→

-n 20 - Show last 20 lines of the log file

Key Concept

If you remember nothing else from this pattern, remember: a 502 error means Nginx cannot reach or get a valid response from the backend server.

Common Mistakes

Not checking if the backend server is running before troubleshooting Nginx.

If the backend is down, Nginx will always return 502 errors regardless of its own configuration.

Always verify the backend service is up and responding before changing Nginx settings.

Ignoring Nginx error logs and only checking access logs.

Error logs contain detailed reasons for 502 errors, which access logs do not show.

Check /var/log/nginx/error.log to find connection or configuration errors causing 502.

Not testing Nginx configuration syntax after changes.

Syntax errors can cause Nginx to fail or misroute requests, leading to 502 errors.

Run 'nginx -t' after any config change to ensure syntax is correct.

Summary

Check if Nginx service is running using 'systemctl status nginx'.

Verify backend server responds correctly with 'curl -I http://localhost:8080'.

Test Nginx configuration syntax using 'nginx -t'.

Review Nginx error logs with 'tail -n 20 /var/log/nginx/error.log' to find connection issues.

Practice

1. What does a 502 Bad Gateway error usually indicate in nginx?

easy

A. nginx cannot get a valid response from the backend server

B. nginx configuration file has a syntax error

C. nginx server is overloaded with too many requests

D. nginx is running out of disk space

Solution

1. Step 1: Understand the meaning of 502 Bad Gateway

   A 502 error means the gateway (nginx) got an invalid response from the backend server it tries to connect to.

2. Step 2: Identify the cause related to backend communication

   This error is not about nginx config syntax or server load, but about backend response failure.

3. Final Answer:

   nginx cannot get a valid response from the backend server -> Option A

4. Quick Check:

   502 Bad Gateway = backend response failure [OK]

Hint: 502 means backend server response problem [OK]

Common Mistakes:

• Confusing 502 with nginx config syntax errors
• Thinking 502 means nginx is overloaded
• Assuming 502 is due to disk space issues

2. Which of the following nginx configuration directives is essential to correctly proxy requests to a backend server to avoid 502 errors?

easy

A. listen 8080;

B. root /var/www/html;

C. server_name localhost;

D. proxy_pass http://backend_server;

Solution

1. Step 1: Identify the directive that defines backend server address

   The proxy_pass directive tells nginx where to send requests, crucial for proxying.

2. Step 2: Understand other directives' roles

   listen sets port, server_name sets hostname, root sets static file location; none define backend proxy.

3. Final Answer:

   proxy_pass http://backend_server; -> Option D

4. Quick Check:

   proxy_pass sets backend URL [OK]

Hint: proxy_pass sets backend URL to avoid 502 [OK]

Common Mistakes:

• Using listen instead of proxy_pass for backend
• Confusing server_name with backend address
• Setting root instead of proxy_pass for proxying

3. Given this nginx error log snippet:
connect() failed (111: Connection refused) while connecting to upstream
What is the most likely cause of the 502 error?

medium

A. nginx has insufficient permissions to read files

B. nginx configuration syntax error

C. Backend server is not running or not accepting connections

D. DNS resolution failure for backend server

Solution

1. Step 1: Interpret the error message

   "Connection refused" means nginx tried to connect but backend server refused, likely because it is down or not listening.

2. Step 2: Rule out other causes

   Syntax errors cause config reload failures, permissions affect file access, DNS failure shows different errors.

3. Final Answer:

   Backend server is not running or not accepting connections -> Option C

4. Quick Check:

   Connection refused = backend down [OK]

Hint: Connection refused means backend server is down [OK]

Common Mistakes:

• Assuming syntax error from connection refused
• Blaming nginx file permissions
• Thinking DNS failure causes connection refused

4. You see a 502 error in nginx logs caused by upstream timed out. Which fix is most appropriate?

medium

A. Clear browser cache and cookies

B. Increase proxy_read_timeout in nginx config

C. Change server_name directive

D. Restart nginx service immediately

Solution

1. Step 1: Understand the timeout error

   "upstream timed out" means nginx waited too long for backend response.

2. Step 2: Adjust nginx timeout settings

   Increasing proxy_read_timeout lets nginx wait longer before giving 502 error.

3. Final Answer:

   Increase proxy_read_timeout in nginx config -> Option B

4. Quick Check:

   Timeout error fixed by increasing proxy_read_timeout [OK]

Hint: Increase proxy_read_timeout to fix upstream timeout [OK]

Common Mistakes:

• Restarting nginx without config change
• Changing server_name unrelated to timeout
• Clearing browser cache won't fix backend timeout

5. You fixed a backend server crash causing 502 errors. What is the correct next step to apply nginx changes and confirm the fix?

hard

A. Reload or restart nginx and check error logs for 502 errors

B. Clear browser cache and refresh the page

C. Edit nginx config to change server_name

D. Delete nginx access logs to free space

Solution

1. Step 1: Apply nginx configuration and backend fixes

   After fixing backend, reload or restart nginx to apply any config changes and reset connections.

2. Step 2: Verify fix by checking logs

   Check nginx error logs to confirm 502 errors no longer appear, ensuring the problem is solved.

3. Final Answer:

   Reload or restart nginx and check error logs for 502 errors -> Option A

4. Quick Check:

   Reload nginx and check logs to confirm fix [OK]

Hint: Reload nginx and check logs after backend fix [OK]

Common Mistakes:

• Only clearing browser cache without server reload
• Changing unrelated config like server_name
• Deleting logs instead of reading them