Why Audit logging in Elasticsearch? - Purpose & Use Cases

The Big Idea

What if you could instantly know who changed what in your system without digging through messy notes?

The Scenario

Imagine you run a busy online store. Every time someone changes a price or updates a product, you try to remember what was changed and by whom. You write notes on paper or in random files, hoping to track these changes later.

The Problem

This manual way is slow and confusing. You might forget to write down important details or mix up who did what. When a problem happens, you waste hours trying to find the cause, and sometimes you never do.

The Solution

Audit logging automatically records every important action in your system. It keeps a clear, organized history of who did what and when. This helps you quickly find problems and understand changes without guessing.

Before vs After

✗ Before

echo 'User X changed price from $10 to $12' >> changes.txt

✓ After

PUT /audit-logs/_doc
{
  "user": "X",
  "action": "price_change",
  "old_value": 10,
  "new_value": 12,
  "timestamp": "2024-06-01T12:00:00Z"
}

What It Enables

With audit logging, you can trust your system's history and fix issues faster, keeping your business safe and reliable.

Real Life Example

A bank uses audit logging to track every login and transaction change. If suspicious activity happens, they quickly see who did what and stop fraud before it grows.

Key Takeaways

Manual tracking is slow and error-prone.

Audit logging records actions automatically and clearly.

This helps find problems fast and improves trust in your system.

Practice

1. What is the main purpose of audit logging in Elasticsearch?

easy

A. To record user actions and security events

B. To improve search speed

C. To backup data automatically

D. To monitor cluster health

Solution

1. Step 1: Understand audit logging function

   Audit logging tracks what users do and records security-related events.

2. Step 2: Compare with other options

   Improving search speed, backing up data, or monitoring cluster health are different Elasticsearch features.

3. Final Answer:

   To record user actions and security events -> Option A

4. Quick Check:

   Audit logging = record user actions [OK]

Hint: Audit logging tracks user and security events only [OK]

Common Mistakes:

• Confusing audit logging with backup or monitoring
• Thinking it speeds up search queries
• Assuming it manages cluster health

2. Which setting enables audit logging in Elasticsearch's configuration file?

easy

A. security.audit.log: on

B. xpack.security.audit.enabled: true

C. audit.logging.enabled: yes

D. xpack.audit.security: enabled

Solution

1. Step 1: Identify correct setting syntax

   The official setting to enable audit logging is xpack.security.audit.enabled set to true.

2. Step 2: Check other options for correctness

   Other options use incorrect keys or values not recognized by Elasticsearch.

3. Final Answer:

   xpack.security.audit.enabled: true -> Option B

4. Quick Check:

   Enable audit logging = xpack.security.audit.enabled true [OK]

Hint: Look for 'xpack.security.audit.enabled' set to true [OK]

Common Mistakes:

• Using incorrect key names
• Using 'yes' or 'on' instead of true
• Mixing audit and security keys

3. Given this audit logging config snippet:

 xpack.security.audit.enabled: true
 xpack.security.audit.outputs: ["logfile", "index"] 

What does this configuration do?

medium

A. Enables audit logging and sends data to log files and Elasticsearch index

B. Disables audit logging but logs to file

C. Enables audit logging but only logs to console

D. Enables audit logging but stores data only in memory

Solution

1. Step 1: Analyze 'enabled' setting

   Setting xpack.security.audit.enabled: true turns audit logging on.

2. Step 2: Analyze 'outputs' setting

   Outputs set to ["logfile", "index"] means audit data is saved to log files and Elasticsearch indexes.

3. Final Answer:

   Enables audit logging and sends data to log files and Elasticsearch index -> Option A

4. Quick Check:

   Enabled + logfile and index outputs = Enables audit logging and sends data to log files and Elasticsearch index [OK]

Hint: Enabled true + outputs list means multiple destinations [OK]

Common Mistakes:

• Assuming logging is disabled
• Thinking output is only console
• Ignoring multiple output destinations

4. You enabled audit logging with xpack.security.audit.enabled: true but see no audit logs. What is a likely cause?

medium

A. User permissions prevent audit logging

B. Elasticsearch cluster is offline

C. Audit logging requires a restart of Kibana

D. Audit outputs are not configured, so logs have no destination

Solution

1. Step 1: Check audit logging enablement

   Audit logging is enabled, so it should produce logs if outputs are set.

2. Step 2: Verify output configuration

   If xpack.security.audit.outputs is missing or empty, logs have nowhere to go, so no logs appear.

3. Final Answer:

   Audit outputs are not configured, so logs have no destination -> Option D

4. Quick Check:

   Enabled but no outputs = no logs [OK]

Hint: Check audit outputs setting if no logs appear [OK]

Common Mistakes:

• Assuming cluster offline without checking
• Restarting Kibana instead of Elasticsearch
• Blaming user permissions without audit config check

5. You want to audit only authentication events and store them in a dedicated Elasticsearch index. Which configuration snippet achieves this?

hard

A. xpack.security.audit.enabled: true
xpack.security.audit.outputs: ["index", "logfile"]
xpack.security.audit.categories: ["access_denied"]

B. xpack.security.audit.enabled: true
xpack.security.audit.outputs: ["logfile"]
xpack.security.audit.categories: ["access_granted"]

C. xpack.security.audit.enabled: true
xpack.security.audit.outputs: ["index"]
xpack.security.audit.categories: ["authentication"]

D. xpack.security.audit.enabled: false
xpack.security.audit.outputs: ["index"]
xpack.security.audit.categories: ["authentication"]

Solution

1. Step 1: Enable audit logging

   Set xpack.security.audit.enabled: true to turn on audit logging.

2. Step 2: Set output to Elasticsearch index only

   Use xpack.security.audit.outputs: ["index"] to store logs in an index.

3. Step 3: Filter to authentication events

   Set xpack.security.audit.categories: ["authentication"] to audit only authentication events.

4. Final Answer:

   Enable audit, output to index, filter authentication events -> Option C

5. Quick Check:

   Enable + index output + authentication category = xpack.security.audit.enabled: true xpack.security.audit.outputs: ["index"] xpack.security.audit.categories: ["authentication"] [OK]

Hint: Enable true + output index + category authentication [OK]

Common Mistakes:

• Disabling audit logging by mistake
• Choosing wrong categories like access_granted
• Using logfile output instead of index only