Windowed operations in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a windowed operation in Kafka Streams?

A windowed operation groups records by key into time-based windows, allowing aggregation or processing of data within those time frames.

intermediate

Name the three main types of windows in Kafka Streams.

The three main window types are: Tumbling windows (fixed size, non-overlapping), Hopping windows (fixed size, overlapping), and Sliding windows (based on event time, overlapping).

intermediate

How does a tumbling window differ from a hopping window?

A tumbling window has fixed, non-overlapping intervals, so each event belongs to exactly one window. A hopping window has fixed size but advances by a smaller step, causing windows to overlap and events to belong to multiple windows.

advanced

What is the purpose of grace period in Kafka Streams windowed operations?

Grace period defines how long Kafka Streams waits for late-arriving events after the window end time before closing the window and emitting results.

beginner

Explain how windowed aggregation works in Kafka Streams.

Windowed aggregation groups records by key within defined time windows and applies aggregation functions (like count, sum) to produce summarized results per window.

Which window type in Kafka Streams has fixed size and non-overlapping intervals?

A Tumbling window

B Hopping window

C Sliding window

D Session window

Tumbling windows have fixed size and do not overlap, so each event belongs to exactly one window.

What does the grace period in windowed operations control?

A The window size

B How long to wait for late events after window end

C The advance interval of hopping windows

D The key grouping strategy

Grace period defines how long Kafka Streams waits for late-arriving events before closing the window.

In a hopping window, what causes windows to overlap?

A Window size is smaller than advance interval

B Windows have no fixed size

C Windows are based on event time

D Advance interval is smaller than window size

Hopping windows overlap because the advance interval (step) is smaller than the window size.

Which Kafka Streams window type is based on periods of activity separated by inactivity?

A Session window

B Hopping window

C Tumbling window

D Sliding window

Session windows group events separated by inactivity gaps, unlike fixed-size windows.

What is the main benefit of using windowed operations in stream processing?

A To process all data at once

B To avoid key-based grouping

C To group and aggregate data over time intervals

D To ignore event timestamps

Windowed operations allow grouping and aggregation of data within time intervals, enabling time-based analysis.

Describe the differences between tumbling, hopping, and sliding windows in Kafka Streams.

Think about window size, overlap, and how events are grouped.

Explain how Kafka Streams handles late-arriving events in windowed operations and the role of grace period.

Consider why late events might arrive and how to handle them.

Practice

1. What is the main purpose of windowed operations in Kafka Streams?

easy

A. To compress data for storage

B. To group data into time intervals for easier analysis

C. To encrypt data streams

D. To sort data alphabetically

Solution

1. Step 1: Understand windowed operations

   Windowed operations group data based on time intervals, making it easier to analyze events occurring within those intervals.

2. Step 2: Compare options

   Sorting, encrypting, and compressing are unrelated to windowing, which focuses on time-based grouping.

3. Final Answer:

   To group data into time intervals for easier analysis -> Option B

4. Quick Check:

   Windowed operations = time grouping [OK]

Hint: Windowed means grouping by time intervals [OK]

Common Mistakes:

• Confusing windowing with sorting
• Thinking windowing encrypts data
• Assuming windowing compresses data

2. Which of the following is the correct syntax to create a tumbling window of 5 minutes in Kafka Streams DSL?

easy

A. SlidingWindows.ofTimeDifferenceWithNoGrace(Duration.ofMinutes(5))

B. SessionWindows.of(Duration.ofMinutes(5))

C. TimeWindows.of(Duration.ofMinutes(5))

D. Windowed.of(Duration.ofMinutes(5))

Solution

1. Step 1: Identify tumbling window syntax

   Tumbling windows in Kafka Streams are created using TimeWindows.of() with a duration.

2. Step 2: Check options

   SessionWindows is for session windows, SlidingWindows is for sliding windows, and Windowed is not a window type class.

3. Final Answer:

   TimeWindows.of(Duration.ofMinutes(5)) -> Option C

4. Quick Check:

   Tumbling window = TimeWindows.of() [OK]

Hint: Tumbling windows use TimeWindows.of() syntax [OK]

Common Mistakes:

• Using SessionWindows for tumbling windows
• Confusing SlidingWindows with tumbling windows
• Using non-existent Windowed class

3. Given the following Kafka Streams code snippet, what will be the output count for the window ending at timestamp 60000?

stream.groupByKey()
      .windowedBy(TimeWindows.of(Duration.ofSeconds(30)))
      .count()
      .toStream()
      .foreach((windowedKey, count) -> System.out.println(windowedKey.window().end() + ": " + count));

Assume events with keys arrive at timestamps 10000, 20000, 35000, 40000, and 55000.

medium

A. 3

B. 4

C. 2

D. 5

Solution

1. Step 1: Identify window intervals

   Windows are 30 seconds long: [0-30000), [30000-60000), etc. Events at 10000 and 20000 fall in first window; 35000, 40000, 55000 fall in second window ending at 60000.

2. Step 2: Count events in window ending at 60000

   Events at 35000, 40000, and 55000 are inside the window ending at 60000, so count is 3.

3. Final Answer:

   3 -> Option A

4. Quick Check:

   Events in 30s window = 3 [OK]

Hint: Count events in the correct time window interval [OK]

Common Mistakes:

• Counting events from previous window
• Misunderstanding window boundaries
• Including events outside the window

4. Identify the error in this Kafka Streams windowed count code snippet:

stream.groupByKey()
      .windowedBy(SessionWindows.of(Duration.ofMinutes(1)))
      .count()
      .toStream()
      .foreach((windowedKey, count) -> System.out.println(windowedKey.window().start() + ": " + count));

medium

A. windowedBy() cannot be used with groupByKey()

B. toStream() is not available after count()

C. count() must be called before windowedBy()

D. SessionWindows.of() requires a gap duration, not a window size

Solution

1. Step 1: Check SessionWindows.of() usage

   SessionWindows require a gap duration (inactivity gap), not a fixed window size in of().

2. Step 2: Validate other method calls

   groupByKey() works with windowedBy(), count() is correctly placed, and toStream() is valid after count().

3. Final Answer:

   SessionWindows.of() requires a gap duration, not a window size -> Option D

4. Quick Check:

   SessionWindows need gap duration [OK]

Hint: SessionWindows need gap duration, not fixed size [OK]

Common Mistakes:

• Using fixed duration in SessionWindows.of()
• Misplacing count() before windowedBy()
• Assuming toStream() is invalid after count()

5. You want to count user clicks in overlapping 1-minute windows that advance every 30 seconds using Kafka Streams. Which window type and configuration should you use?

hard

A. Use TimeWindows.of(Duration.ofMinutes(1)).advanceBy(Duration.ofSeconds(30))

B. Use SlidingWindows.ofTimeDifferenceAndGrace(Duration.ofMinutes(1), Duration.ofSeconds(0)).advanceBy(Duration.ofSeconds(30))

C. Use SessionWindows.withGap(Duration.ofSeconds(30))

D. Use TimeWindows.of(Duration.ofSeconds(30)).advanceBy(Duration.ofMinutes(1))

Solution

1. Step 1: Identify overlapping window type

   Overlapping windows that slide are created with TimeWindows.of() and advanceBy() to set the slide interval.

2. Step 2: Match configuration to requirement

   Window size is 1 minute, sliding every 30 seconds, so TimeWindows.of(1 minute).advanceBy(30 seconds) fits perfectly.

3. Final Answer:

   Use TimeWindows.of(Duration.ofMinutes(1)).advanceBy(Duration.ofSeconds(30)) -> Option A

4. Quick Check:

   Sliding windows = TimeWindows with advanceBy [OK]

Hint: Sliding windows use TimeWindows with advanceBy() [OK]

Common Mistakes:

• Using SlidingWindows class which is deprecated
• Confusing session windows with sliding windows
• Swapping window size and advanceBy durations