Join operations (KStream-KStream, KStream-KTable) in Kafka - Step-by-Step Execution

Process Flow - Join operations (KStream-KStream, KStream-KTable)

Start with two streams or stream+table

↓

Match keys from both sides

↓

Apply join condition

↓

Combine records into one output

↓

Emit joined stream

↓

End

Join operations combine records from two streams or a stream and a table by matching keys and merging data into a new stream.

Execution Sample

Kafka

KStream<String, String> stream1 = builder.stream("topic1");
KStream<String, String> stream2 = builder.stream("topic2");

KStream<String, String> joined = stream1.join(stream2,
  (v1, v2) -> v1 + ":" + v2,
  JoinWindows.ofTimeDifferenceWithNoGrace(Duration.ofSeconds(5)));

joined.to("joined-topic");

This code joins two streams on matching keys within a 5-second window and outputs combined records.

Process Table

Step	Input Stream1 Record	Input Stream2 Record	Join Condition	Output Record
1	key1: A	key1: X	Keys match, within 5s window	key1: A:X
2	key2: B	key3: Y	Keys do not match	No output
3	key1: C	key1: Z	Keys match, within 5s window	key1: C:Z
4	key4: D	key1: W	Keys do not match	No output
5	End of input	End of input	No more records	Stream ends

💡 No more matching keys or records to join, stream processing ends.

Status Tracker

Variable	Start	After 1	After 2	After 3	After 4	Final
stream1 record	none	key1:A	key2:B	key1:C	key4:D	end
stream2 record	none	key1:X	key3:Y	key1:Z	key1:W	end
joined output	none	key1:A:X	none	key1:C:Z	none	end

Key Moments - 2 Insights

Why does no output appear when keys don't match?

Join only emits output when keys from both streams match (see rows 2 and 4 in execution_table). Non-matching keys produce no joined record.

How does the time window affect joining?

Records must arrive within the join window (5 seconds here) to be joined. If outside, no output is produced (implied in join condition column).

Visual Quiz - 3 Questions

Look at the execution_table, what is the output record at step 3?

A key1:A:X

B key1:C:Z

C key2:B:Y

D No output

💡 Hint

Check the Output Record column at step 3 in execution_table.

At which step do the keys not match and no output is produced?

A Step 2

B Step 1

C Step 3

D Step 5

💡 Hint

Look for 'No output' in the Output Record column in execution_table.

If the join window was removed, how would the output change?

A Fewer joined records

B No change in output

C More joined records regardless of time

D Stream would stop immediately

💡 Hint

Join window controls timing; removing it allows all matching keys to join anytime.

Concept Snapshot

Join operations combine two streams or a stream and a table by matching keys.
KStream-KStream join uses a time window to match records.
KStream-KTable join uses latest table state for each key.
Output is a new stream with combined records.
Only matching keys produce output.
Time window controls which records join in streams.

Full Transcript

Join operations in Kafka Streams combine data from two sources by matching keys. For KStream-KStream joins, records must arrive within a time window to be joined. For KStream-KTable joins, the stream record joins with the latest table value for the key. The process starts by reading records from both sources, matching keys, applying join conditions, and producing a combined output record. If keys don't match or records fall outside the time window, no output is produced. The joined results are emitted as a new stream for further processing or output. This visual trace shows step-by-step how records from two streams are joined based on keys and time window, tracking variables and outputs at each step.