Conditional logging in Nginx - Commands & Configuration

Introduction

Sometimes you want to log only certain requests in your web server to save space or focus on important events. Conditional logging in nginx lets you decide which requests get logged based on rules you set.

When you want to log only error responses to find problems faster.

When you want to skip logging requests for static files like images to reduce log size.

When you want to log requests from specific IP addresses for security monitoring.

When you want to disable logging for health check requests to keep logs clean.

When you want to log only POST requests to analyze form submissions.

Config File - nginx.conf

nginx.conf

http {
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent"';

    map $status $loggable {
        ~^[23] 1;
        default 0;
    }

    access_log /var/log/nginx/access.log main if=$loggable;

    server {
        listen 80;
        server_name example.com;

        location / {
            root /usr/share/nginx/html;
            index index.html;
        }
    }
}

The log_format defines how each log entry looks.

The map block creates a variable $loggable that is 1 for status codes starting with 2 or 3 (success and redirects), and 0 otherwise.

The access_log directive uses the if=$loggable condition to log only requests where $loggable is 1, meaning successful and redirect statuses.

The server block defines a simple web server listening on port 80.

Commands

Check the nginx configuration file syntax to make sure there are no errors before reloading.

Terminal

nginx -t

Expected Output

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

Reload nginx to apply the new configuration without stopping the server.

Terminal

systemctl reload nginx

Expected Output

No output (command runs silently)

View the last 5 lines of the access log to verify that only requests matching the condition are logged.

Terminal

tail -n 5 /var/log/nginx/access.log

Expected Output

192.168.1.10 - - [27/Apr/2024:12:00:01 +0000] "GET /notfound HTTP/1.1" 404 150 "-" "Mozilla/5.0"

192.168.1.11 - - [27/Apr/2024:12:00:05 +0000] "POST /submit HTTP/1.1" 500 200 "-" "Mozilla/5.0"

Key Concept

If you remember nothing else from this pattern, remember: nginx can log requests selectively using variables and the 'if' condition in the access_log directive.

Common Mistakes

Using 'if' inside the server or location block to control logging instead of the 'if' parameter in access_log.

The 'if' directive inside server or location blocks does not control logging and can cause unexpected behavior.

Use the 'if' parameter directly in the access_log directive with a variable that controls logging.

Not testing nginx configuration with 'nginx -t' before reloading.

Reloading with a bad config will fail and may cause downtime.

Always run 'nginx -t' to verify syntax before reloading.

Setting the condition variable incorrectly so that all requests are logged or none are logged.

This defeats the purpose of conditional logging and can fill logs unnecessarily or miss important entries.

Use the 'map' directive carefully to set the variable based on exact conditions like status codes.

Summary

Define a log format to specify how logs look.

Use the 'map' directive to create a variable that decides which requests to log.

Apply conditional logging with 'access_log' using the 'if' parameter and the variable.

Test the configuration with 'nginx -t' before reloading nginx.

Check the logs to confirm only desired requests are logged.

Practice

1. What is the main purpose of conditional logging in nginx?

easy

A. To log only specific requests based on conditions

B. To disable all logging permanently

C. To log every request without any filter

D. To log errors only, ignoring access logs

Solution

1. Step 1: Understand logging basics in nginx

   Logging records requests to help monitor and debug web traffic.

2. Step 2: Identify the role of conditional logging

   Conditional logging allows filtering which requests get logged based on rules.

3. Final Answer:

   To log only specific requests based on conditions -> Option A

4. Quick Check:

   Conditional logging = selective logging [OK]

Hint: Conditional logging means logging only some requests [OK]

Common Mistakes:

• Thinking conditional logging disables all logs
• Confusing conditional logging with error-only logging
• Assuming it logs every request without filtering

2. Which of the following is the correct syntax to enable conditional logging in nginx using the access_log directive?

easy

A. access_log /var/log/nginx/access.log on_condition=$if;

B. access_log /var/log/nginx/access.log if=$condition;

C. access_log /var/log/nginx/access.log condition=$if;

D. access_log if=$condition /var/log/nginx/access.log;

Solution

1. Step 1: Recall the correct order of parameters in access_log

   The syntax is: access_log <path> [format] [if=condition];

2. Step 2: Identify the correct use of the if= option

   The condition must be specified as if=$variable after the log path.

3. Final Answer:

   access_log /var/log/nginx/access.log if=$condition; -> Option B

4. Quick Check:

   access_log path if=condition [OK]

Hint: Remember: if= comes after log file path in access_log [OK]

Common Mistakes:

• Placing if= before the log file path
• Using wrong parameter names like condition= or on_condition=
• Omitting the $ sign before the variable

3. Given the following nginx configuration snippet, what will be the effect on logging?

map $request_uri $loggable {
    default 1;
    "/health" 0;
}

access_log /var/log/nginx/access.log combined if=$loggable;

medium

A. Only requests to /health will be logged

B. No requests will be logged

C. All requests except to /health will be logged

D. All requests will be logged regardless of URI

Solution

1. Step 1: Understand the map directive

   The map sets $loggable to 0 for "/health" and 1 for all others.

2. Step 2: Analyze the access_log condition

   Logging happens only if $loggable is true (1), so requests to "/health" (0) are skipped.

3. Final Answer:

   All requests except to /health will be logged -> Option C

4. Quick Check:

   map 0 disables logging for /health [OK]

Hint: map 0 disables logging; 1 enables it [OK]

Common Mistakes:

• Assuming /health requests are logged
• Thinking map disables all logging
• Confusing default and specific URI values

4. Identify the error in this nginx configuration for conditional logging:

map $status $loggable {
    200 1;
    default 0;
}

access_log /var/log/nginx/access.log combined if=loggable;

medium

A. access_log path is invalid

B. map directive syntax is incorrect

C. Cannot use $status variable in map

D. Missing $ before loggable in access_log condition

Solution

1. Step 1: Check variable usage in access_log

   Variables must be prefixed with $ in conditions, so if=loggable is wrong.

2. Step 2: Confirm correct syntax

   Correct syntax is if=$loggable to reference the variable properly.

3. Final Answer:

   Missing $ before loggable in access_log condition -> Option D

4. Quick Check:

   Variables need $ prefix in if= [OK]

Hint: Always prefix variables with $ in if= conditions [OK]

Common Mistakes:

• Omitting $ before variable in if= condition
• Miswriting map syntax
• Assuming $status cannot be used in map

5. You want to log all requests except those with user agent containing "Googlebot". Which configuration correctly implements this conditional logging?

hard

A. map $http_user_agent $loggable {
default 1;
~Googlebot 0;
}
access_log /var/log/nginx/access.log combined if=$loggable;

B. map $http_user_agent $loggable {
default 0;
~Googlebot 1;
}
access_log /var/log/nginx/access.log combined if=$loggable;

C. map $http_user_agent $loggable {
default 1;
Googlebot 0;
}
access_log /var/log/nginx/access.log combined if=$loggable;

D. map $http_user_agent $loggable {
default 1;
~Googlebot 1;
}
access_log /var/log/nginx/access.log combined if=$loggable;

Solution

1. Step 1: Use map with regex to detect "Googlebot" in user agent

   The ~ prefix allows regex matching; setting 0 disables logging for matching agents.

2. Step 2: Set default to 1 to log all other requests

   Default 1 means log unless user agent matches Googlebot.

3. Step 3: Use if=$loggable in access_log to apply condition

   This ensures only requests with $loggable=1 are logged.

4. Final Answer:

   map $http_user_agent $loggable { default 1; ~Googlebot 0; } access_log /var/log/nginx/access.log combined if=$loggable; -> Option A

5. Quick Check:

   Regex ~Googlebot disables logging for bots [OK]

Hint: Use ~ for regex in map to match user agents [OK]

Common Mistakes:

• Using exact string without ~ for regex
• Reversing default values causing wrong logging
• Not prefixing variable with $ in if= condition