Audit logging in Elasticsearch - Step-by-Step Execution

Concept Flow - Audit logging

Enable audit logging in elasticsearch.yml

↓

Elasticsearch starts with audit logging enabled

↓

User actions occur: login, data access, config changes

↓

Audit logs capture events with details

↓

Logs stored in specified audit log file or index

↓

Admin reviews logs for security and compliance

Audit logging in Elasticsearch tracks user and system actions by enabling logging in the config, capturing events, and storing them for review.

Execution Sample

Elasticsearch

xpack.security.audit.enabled: true
xpack.security.audit.outputs: [ "index", "logger" ]
xpack.security.audit.logger.events.include: [ "authentication_success", "authentication_failed" ]

This config enables audit logging, sets outputs to index and logger, and includes authentication events.

Execution Table

Step	Action	Config Setting	Event Captured	Log Output
1	Enable audit logging	xpack.security.audit.enabled: true	None yet	No logs yet
2	User login attempt	Include authentication_success	authentication_success	Logged to index and logger
3	User login failure	Include authentication_failed	authentication_failed	Logged to index and logger
4	User accesses data	Default events	access_granted	Logged if configured
5	Admin reviews logs	N/A	N/A	Logs show authentication and access events

💡 Audit logging runs continuously capturing configured events until disabled

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
audit_enabled	false	true	true	true	true
logged_events	[]	[authentication_success]	[authentication_success, authentication_failed]	[authentication_success, authentication_failed, access_granted]	[authentication_success, authentication_failed, access_granted]

Key Moments - 2 Insights

Why don't I see any audit logs after enabling audit logging?

Audit logging starts capturing events only after it is enabled (see Step 1). Also, only configured event types are logged (see Steps 2 and 3). If no matching events occur, no logs appear.

Are all user actions logged by default?

No, only events specified in the config (like authentication_success) are logged. Other events require explicit inclusion (see Step 4).

Visual Quiz - 3 Questions

Look at the execution table, at which step is the first audit log entry created?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Check the 'Log Output' column to see when logs first appear

According to the variable tracker, what is the value of 'audit_enabled' after Step 3?

A true

B false

C undefined

D null

💡 Hint

Look at the 'audit_enabled' row under 'After Step 3' column

If 'authentication_failed' events were not included in the config, what would happen at Step 3?

A Elasticsearch would error

B Event would be logged anyway

C Event would not be logged

D All events would be logged

💡 Hint

Refer to the 'Config Setting' and 'Event Captured' columns in the execution table

Concept Snapshot

Audit logging in Elasticsearch:
- Enable with xpack.security.audit.enabled: true
- Configure events to log (e.g., authentication_success)
- Logs output to index and/or logger
- Captures user and system actions
- Used for security and compliance review

Full Transcript

Audit logging in Elasticsearch is turned on by setting 'xpack.security.audit.enabled' to true in the configuration file. Once enabled, Elasticsearch captures specific events like successful or failed user logins, depending on which events are included in the config. These events are logged to configured outputs such as an index or logger. The logs help administrators monitor security-related actions. The execution table shows the step-by-step process from enabling audit logging, capturing events, to reviewing logs. Variables like 'audit_enabled' track whether logging is active, and 'logged_events' track which events have been recorded. Common confusions include why no logs appear immediately after enabling and which events are logged by default. The visual quiz tests understanding of when logs start, variable states, and the effect of config settings on event logging.

Practice

1. What is the main purpose of audit logging in Elasticsearch?

easy

A. To record user actions and security events

B. To improve search speed

C. To backup data automatically

D. To monitor cluster health

Solution

1. Step 1: Understand audit logging function

   Audit logging tracks what users do and records security-related events.

2. Step 2: Compare with other options

   Improving search speed, backing up data, or monitoring cluster health are different Elasticsearch features.

3. Final Answer:

   To record user actions and security events -> Option A

4. Quick Check:

   Audit logging = record user actions [OK]

Hint: Audit logging tracks user and security events only [OK]

Common Mistakes:

• Confusing audit logging with backup or monitoring
• Thinking it speeds up search queries
• Assuming it manages cluster health

2. Which setting enables audit logging in Elasticsearch's configuration file?

easy

A. security.audit.log: on

B. xpack.security.audit.enabled: true

C. audit.logging.enabled: yes

D. xpack.audit.security: enabled

Solution

1. Step 1: Identify correct setting syntax

   The official setting to enable audit logging is xpack.security.audit.enabled set to true.

2. Step 2: Check other options for correctness

   Other options use incorrect keys or values not recognized by Elasticsearch.

3. Final Answer:

   xpack.security.audit.enabled: true -> Option B

4. Quick Check:

   Enable audit logging = xpack.security.audit.enabled true [OK]

Hint: Look for 'xpack.security.audit.enabled' set to true [OK]

Common Mistakes:

• Using incorrect key names
• Using 'yes' or 'on' instead of true
• Mixing audit and security keys

3. Given this audit logging config snippet:

 xpack.security.audit.enabled: true
 xpack.security.audit.outputs: ["logfile", "index"] 

What does this configuration do?

medium

A. Enables audit logging and sends data to log files and Elasticsearch index

B. Disables audit logging but logs to file

C. Enables audit logging but only logs to console

D. Enables audit logging but stores data only in memory

Solution

1. Step 1: Analyze 'enabled' setting

   Setting xpack.security.audit.enabled: true turns audit logging on.

2. Step 2: Analyze 'outputs' setting

   Outputs set to ["logfile", "index"] means audit data is saved to log files and Elasticsearch indexes.

3. Final Answer:

   Enables audit logging and sends data to log files and Elasticsearch index -> Option A

4. Quick Check:

   Enabled + logfile and index outputs = Enables audit logging and sends data to log files and Elasticsearch index [OK]

Hint: Enabled true + outputs list means multiple destinations [OK]

Common Mistakes:

• Assuming logging is disabled
• Thinking output is only console
• Ignoring multiple output destinations

4. You enabled audit logging with xpack.security.audit.enabled: true but see no audit logs. What is a likely cause?

medium

A. User permissions prevent audit logging

B. Elasticsearch cluster is offline

C. Audit logging requires a restart of Kibana

D. Audit outputs are not configured, so logs have no destination

Solution

1. Step 1: Check audit logging enablement

   Audit logging is enabled, so it should produce logs if outputs are set.

2. Step 2: Verify output configuration

   If xpack.security.audit.outputs is missing or empty, logs have nowhere to go, so no logs appear.

3. Final Answer:

   Audit outputs are not configured, so logs have no destination -> Option D

4. Quick Check:

   Enabled but no outputs = no logs [OK]

Hint: Check audit outputs setting if no logs appear [OK]

Common Mistakes:

• Assuming cluster offline without checking
• Restarting Kibana instead of Elasticsearch
• Blaming user permissions without audit config check

5. You want to audit only authentication events and store them in a dedicated Elasticsearch index. Which configuration snippet achieves this?

hard

A. xpack.security.audit.enabled: true
xpack.security.audit.outputs: ["index", "logfile"]
xpack.security.audit.categories: ["access_denied"]

B. xpack.security.audit.enabled: true
xpack.security.audit.outputs: ["logfile"]
xpack.security.audit.categories: ["access_granted"]

C. xpack.security.audit.enabled: true
xpack.security.audit.outputs: ["index"]
xpack.security.audit.categories: ["authentication"]

D. xpack.security.audit.enabled: false
xpack.security.audit.outputs: ["index"]
xpack.security.audit.categories: ["authentication"]

Solution

1. Step 1: Enable audit logging

   Set xpack.security.audit.enabled: true to turn on audit logging.

2. Step 2: Set output to Elasticsearch index only

   Use xpack.security.audit.outputs: ["index"] to store logs in an index.

3. Step 3: Filter to authentication events

   Set xpack.security.audit.categories: ["authentication"] to audit only authentication events.

4. Final Answer:

   Enable audit, output to index, filter authentication events -> Option C

5. Quick Check:

   Enable + index output + authentication category = xpack.security.audit.enabled: true xpack.security.audit.outputs: ["index"] xpack.security.audit.categories: ["authentication"] [OK]

Hint: Enable true + output index + category authentication [OK]

Common Mistakes:

• Disabling audit logging by mistake
• Choosing wrong categories like access_granted
• Using logfile output instead of index only