Logstash overview in Elasticsearch - Step-by-Step Execution

Concept Flow - Logstash overview

Start Logstash

↓

Input: Receive Data

↓

Filter: Process & Transform

↓

Output: Send Data

↓

End

Logstash takes data in, processes it, and sends it out in a flow from input to filter to output.

Execution Sample

Elasticsearch

input {
  stdin {}
}
filter {
  mutate { lowercase => ["message"] }
}
output {
  stdout { codec => rubydebug }
}

This Logstash config reads input from the keyboard, makes the message lowercase, and prints it out.

Execution Table

Step	Action	Input Data	Filter Applied	Output Data
1	Receive input	HELLO WORLD	None yet	HELLO WORLD
2	Apply filter	HELLO WORLD	Lowercase message	hello world
3	Send output	hello world	Filter done	hello world
4	Wait for next input	None	None	None

💡 Logstash waits for new input after processing current data.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
message	None	HELLO WORLD	hello world	hello world	hello world

Key Moments - 2 Insights

Why does Logstash have separate input, filter, and output sections?

Because each section has a clear job: input gets data, filter changes data, output sends data. This flow is shown in the execution_table steps 1 to 3.

What happens if no filter is defined?

Data passes from input directly to output unchanged, like step 1 to 3 but skipping filtering in the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, what is the message content after Step 2?

A "hello world"

B "Hello World"

C "HELLO WORLD"

D "HELLO world"

💡 Hint

Check the 'Output Data' column at Step 2 in the execution_table.

At which step does Logstash apply the filter to the data?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Look at the 'Filter Applied' column in the execution_table.

If the filter was removed, what would be the output at Step 3?

A "Hello World"

B "hello world"

C "HELLO WORLD"

D No output

💡 Hint

Without filter, input data passes unchanged to output, see Step 1 and 3 comparison.

Concept Snapshot

Logstash processes data in three parts:
Input: where data comes in
Filter: where data is changed or enriched
Output: where data is sent out
Data flows step-by-step through these stages
Filters modify data before output

Full Transcript

Logstash is a tool that takes data from many sources, changes it if needed, and sends it somewhere else. It works in three steps: input, filter, and output. First, it gets data from input like a keyboard or file. Then it changes the data using filters, for example making text lowercase. Finally, it sends the changed data to output like the screen or a database. This flow helps organize data processing clearly and simply.