KStream and KTable concepts in Kafka - Step-by-Step Execution

Process Flow - KStream and KTable concepts

Start: Input Data

↓

Process each event

↓

Output transformed

↓

Downstream apps

Data flows into Kafka as events. KStream processes each event individually as a stream. KTable keeps the latest state per key, updating as new events arrive.

Execution Sample

Kafka

KStream<String, String> stream = builder.stream("input-topic");
KTable<String, String> table = builder.table("input-topic");

stream.foreach((k,v) -> System.out.println("Stream event: " + k + ":" + v));
table.toStream().foreach((k,v) -> System.out.println("Table state: " + k + ":" + v));

This code reads the same Kafka topic as a stream and as a table, printing each event and the latest state per key.

Process Table

Step	Input Event	KStream Action	KTable Action	Output
1	key1: value1	Process event key1:value1	Update key1 state to value1	Stream event: key1:value1 Table state: key1:value1
2	key2: value2	Process event key2:value2	Update key2 state to value2	Stream event: key2:value2 Table state: key2:value2
3	key1: value3	Process event key1:value3	Update key1 state to value3 (overwrite)	Stream event: key1:value3 Table state: key1:value3
4	key3: value4	Process event key3:value4	Update key3 state to value4	Stream event: key3:value4 Table state: key3:value4
5	No more events	Stop processing	Stop updating	End of stream and table output

💡 No more input events; processing stops.

Status Tracker

Variable	Start	After 1	After 2	After 3	After 4	Final
KStream current event	null	key1:value1	key2:value2	key1:value3	key3:value4	null
KTable state key1	null	value1	value1	value3	value3	value3
KTable state key2	null	null	value2	value2	value2	value2
KTable state key3	null	null	null	null	value4	value4

Key Moments - 2 Insights

Why does KTable overwrite the value for key1 at step 3?

Because KTable represents the latest state per key, it updates the stored value for key1 from value1 to value3 when a new event with key1 arrives, as shown in execution_table row 3.

Does KStream keep previous events after processing?

No, KStream processes each event as it arrives and does not store state. Each event is handled independently, as seen in the 'KStream Action' column of the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table at step 2. What is the KTable state for key2?

A null

B value2

C value1

D value3

💡 Hint

Check the 'KTable Action' and 'Output' columns at step 2 in the execution_table.

At which step does the KTable update the state for key1 to value3?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Look at the 'KTable Action' column for key1 updates in the execution_table.

If a new event with key2:value5 arrives after step 4, what will happen to the KTable state for key2?

A It updates to value5

B It stays as value2

C It deletes key2

D It duplicates value2

💡 Hint

KTable always keeps the latest state per key, updating on new events as shown in variable_tracker.

Concept Snapshot

KStream: Processes each event as a stream, no state kept.
KTable: Represents latest state per key, updates on new events.
Both read from Kafka topics but differ in state handling.
Use KStream for event processing, KTable for stateful views.
KTable updates overwrite previous values for the same key.

Full Transcript

This visual execution shows how Kafka Streams processes data as KStream and KTable. KStream handles each event individually, printing each as it arrives. KTable keeps the latest value per key, updating state when new events with the same key come in. The execution table traces each input event, showing KStream processing and KTable state updates. Variable tracking shows how KTable state changes per key over time. Key moments clarify why KTable overwrites values and how KStream differs by not storing state. The quiz tests understanding of state updates and event processing steps. This helps beginners see the difference between streaming events and maintaining state in Kafka Streams.

Practice

1. Which statement best describes the difference between KStream and KTable in Kafka Streams?

easy

A. KTable is used for event processing, and KStream is used for batch processing.

B. KStream processes continuous event streams, while KTable maintains the latest state of data.

C. KStream stores data permanently, while KTable only processes events temporarily.

D. KStream and KTable are identical and can be used interchangeably.

Solution

1. Step 1: Understand KStream purpose

   KStream is designed to handle continuous streams of events as they happen.

2. Step 2: Understand KTable purpose

   KTable keeps track of the latest state of data, updating as new information arrives.

3. Final Answer:

   KStream processes continuous event streams, while KTable maintains the latest state of data. -> Option B

4. Quick Check:

   KStream = events, KTable = state [OK]

Hint: Remember: Stream = events, Table = current state [OK]

Common Mistakes:

• Thinking KStream stores data permanently
• Confusing KTable as batch processor
• Assuming KStream and KTable are the same

2. Which of the following is the correct way to create a KTable from a Kafka topic named users in Java?

easy

A. builder.createKTable("users");

B. builder.stream("users").toTable();

C. builder.table("users");

D. builder.getTable("users");

Solution

1. Step 1: Recall KTable creation syntax

   In Kafka Streams Java API, KTable is created using builder.table(topicName).

2. Step 2: Check options for correct method

   Only builder.table("users") matches the correct syntax.

3. Final Answer:

   builder.table("users") -> Option C

4. Quick Check:

   KTable created with builder.table(topic) [OK]

Hint: Use builder.table() to create KTable from topic [OK]

Common Mistakes:

• Using builder.stream() then toTable() which is invalid
• Trying non-existent methods like createKTable()
• Confusing KStream and KTable creation methods

3. Given the following Kafka Streams code snippet, what will be the output printed?

var stream = builder.stream("orders");
var table = builder.table("orders");
stream.foreach((k, v) -> System.out.println("Stream: " + k + ":" + v));
table.toStream().foreach((k, v) -> System.out.println("Table: " + k + ":" + v));

Assuming the topic orders has these records in order: (1, "A"), (1, "B"), (2, "C").

medium

A. Stream prints all three records; Table prints only latest for each key: (1, "B") and (2, "C")

B. Stream prints only latest record per key; Table prints all records

C. Both Stream and Table print all three records

D. Stream prints no records; Table prints all three records

Solution

1. Step 1: Understand KStream output

   KStream processes every event as it arrives, so it prints all three records: Stream: 1:A, Stream: 1:B, Stream: 2:C.

2. Step 2: Understand KTable.toStream() output

   KTable.toStream() emits the latest update per key. It will print updates for key 1 twice (first "A", then "B") and key 2 once ("C"). However, in practice, KTable.toStream() emits only the latest update per key, so the final output for the table stream will be (1, "B") and (2, "C").

3. Final Answer:

   Stream prints all three records; Table prints only latest for each key: (1, "B") and (2, "C") -> Option A

4. Quick Check:

   KStream emits all events; KTable.toStream() emits latest per key [OK]

Hint: KStream prints all events; KTable.toStream() prints latest per key [OK]

Common Mistakes:

• Thinking KTable.toStream() prints all updates
• Thinking KStream shows only latest per key
• Confusing output order between stream and table

4. You wrote this code to convert a KStream to a KTable:

var stream = builder.stream("events");
var table = stream.toTable();

But it throws a compilation error. What is the likely cause?

medium

A. toTable() method does not exist on KStream interface.

B. You cannot convert a KStream to a KTable at all.

C. You must specify a key serde when converting to KTable.

D. You need to call toTable() with a materialized state store.

Solution

1. Step 1: Check KStream toTable() method requirements

   In Kafka Streams, toTable() requires a Materialized parameter to define state store details.

2. Step 2: Identify missing parameter causing error

   Not providing Materialized causes compilation error because Kafka Streams needs to know how to store the table.

3. Final Answer:

   You need to call toTable() with a materialized state store. -> Option D

4. Quick Check:

   toTable() needs Materialized param [OK]

Hint: toTable() requires Materialized argument [OK]

Common Mistakes:

• Assuming toTable() exists without parameters
• Thinking conversion is impossible
• Ignoring state store configuration

5. You want to build a Kafka Streams application that tracks the latest price of products and also processes every price update event for analytics. Which combination should you use?

hard

A. Use a KTable to keep latest prices and a KStream to process all price update events.

B. Use only a KStream for both latest prices and event processing.

C. Use only a KTable because it handles both events and state.

D. Use a KStream for latest prices and a KTable for event processing.

Solution

1. Step 1: Identify need for latest state tracking

   Tracking latest prices requires a KTable because it maintains the current state per product.

2. Step 2: Identify need for processing all events

   Processing every price update event for analytics requires a KStream to handle all incoming events.

3. Step 3: Combine both correctly

   Use KTable for latest prices and KStream for event processing simultaneously.

4. Final Answer:

   Use a KTable to keep latest prices and a KStream to process all price update events. -> Option A

5. Quick Check:

   KTable = state, KStream = all events [OK]

Hint: Use KTable for state, KStream for event analytics [OK]

Common Mistakes:

• Using only KStream loses latest state
• Using only KTable misses event details
• Swapping roles of KStream and KTable