Interactive queries in Kafka - Mini Project: Build & Apply

Interactive Queries with Kafka Streams

📖 Scenario: You are building a simple Kafka Streams application that counts the number of purchases per product in real time. You want to be able to query the current count of any product interactively.

🎯 Goal: Create a Kafka Streams application that maintains a count of purchases per product and supports interactive queries to get the current count for a given product.

📋 What You'll Learn

Create a Kafka Streams topology with a KTable counting purchases per product

Configure the application with an application ID

Implement interactive queries to fetch the current count for a product

Print the count result for a specific product

💡 Why This Matters

🌍 Real World

Interactive queries let you ask your Kafka Streams app for the latest computed data without waiting for output topics. This is useful for dashboards and real-time monitoring.

💼 Career

Kafka Streams interactive queries are important for building responsive, stateful stream processing applications in data engineering and backend development roles.

1

Create the initial purchase data stream

Create a Kafka Streams KStream named purchases from the topic purchase-events with key type String and value type String.

Kafka

# Create a KStream named purchases from topic 'purchase-events'
# Your code here

Hint

Use builder.stream("purchase-events") to create the stream.

2

Configure the Kafka Streams application ID

Create a Properties object named props and set the StreamsConfig.APPLICATION_ID_CONFIG property to "purchase-count-app".

Kafka

StreamsBuilder builder = new StreamsBuilder();
KStream purchases = builder.stream("purchase-events");

# Create Properties object props and set application ID to 'purchase-count-app'
# Your code here

Hint

Use props.put(StreamsConfig.APPLICATION_ID_CONFIG, "purchase-count-app").

3

Create a KTable counting purchases per product

Use the purchases stream to create a KTable<String, Long> named purchaseCounts that counts the number of purchases per product key using groupByKey() and count().

Kafka

StreamsBuilder builder = new StreamsBuilder();
KStream purchases = builder.stream("purchase-events");

Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "purchase-count-app");

# Create a KTable purchaseCounts that counts purchases per product key
# Your code here

Hint

Use purchases.groupByKey().count() to create the count table.

4

Print the current count for a product using interactive queries

Start the Kafka Streams application with KafkaStreams streams = new KafkaStreams(builder.build(), props); and streams.start(). Then, use streams.store() with QueryableStoreTypes.keyValueStore() to get the ReadOnlyKeyValueStore<String, Long> named store. Finally, print the count for the product key "product-123" using store.get("product-123").

Kafka

StreamsBuilder builder = new StreamsBuilder();
KStream purchases = builder.stream("purchase-events");

Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "purchase-count-app");

KTable purchaseCounts = purchases.groupByKey().count();

# Start KafkaStreams, get the store, and print count for 'product-123'
# Your code here

Hint

Use streams.store() with QueryableStoreTypes.keyValueStore() and then store.get("product-123").

Practice

1.

What is the main purpose of interactive queries in Kafka Streams?

easy

A. To create new Kafka topics dynamically

B. To write data back to Kafka topics

C. To read live state data directly from the Kafka Streams application

D. To monitor Kafka broker health status

Solution

1. Step 1: Understand interactive queries concept

   Interactive queries allow reading live state from Kafka Streams state stores.

2. Step 2: Identify the main use case

   The main use is to get real-time data from the app without extra databases.

3. Final Answer:

   To read live state data directly from the Kafka Streams application -> Option C

4. Quick Check:

   Interactive queries = read live state [OK]

Hint: Interactive queries read live state, not write or monitor [OK]

Common Mistakes:

• Confusing interactive queries with writing data
• Thinking it creates topics
• Assuming it monitors brokers

2.

Which of the following is the correct way to get a state store named myStore of type ReadOnlyKeyValueStore in Kafka Streams?

easy

A. store = streams.store("myStore", ReadOnlyKeyValueStore.class);

B. store = streams.getStore("myStore", ReadOnlyKeyValueStore);

C. store = streams.getStore("myStore", ReadOnlyKeyValueStore.class);

D. store = streams.store("myStore", ReadOnlyKeyValueStore);

Solution

1. Step 1: Recall Kafka Streams store retrieval syntax

   The correct method is streams.store(storeName, storeType.class) in Java.

2. Step 2: Match the correct syntax

   store = streams.store("myStore", ReadOnlyKeyValueStore.class); uses streams.store with class type, which is correct method name and syntax.

3. Final Answer:

   store = streams.store("myStore", ReadOnlyKeyValueStore.class); -> Option A

4. Quick Check:

   Use streams.store with class type [OK]

Hint: Use streams.store("name", StoreClass.class) syntax [OK]

Common Mistakes:

• Using streams.getStore instead of streams.store
• Omitting .class for type
• Passing type without .class

3.

Given the following code snippet in a Kafka Streams app, what will be the output if the key "user123" exists with value "active" in the state store?

var store = streams.store("userStatusStore", ReadOnlyKeyValueStore.class);
var status = store.get("user123");
System.out.println(status);

medium

A. null

B. "active"

C. Throws NullPointerException

D. Compilation error

Solution

1. Step 1: Understand store.get(key) behavior

   The get method returns the value for the key if it exists, else null.

2. Step 2: Check given key and value

   Since "user123" exists with value "active", store.get("user123") returns "active".

3. Final Answer:

   "active" -> Option B

4. Quick Check:

   store.get(existingKey) = value [OK]

Hint: store.get(key) returns value or null if missing [OK]

Common Mistakes:

• Assuming null is returned even if key exists
• Expecting exceptions on missing keys
• Confusing syntax causing compile errors

4.

Identify the error in this Kafka Streams interactive query code snippet:

var store = streams.store("ordersStore", ReadOnlyKeyValueStore.class);
var order = store.get(12345);
System.out.println(order);

medium

A. Passing integer key instead of string key

B. Missing try-catch block for exceptions

C. Using streams.store instead of streams.getStore

D. Incorrect store type ReadOnlyKeyValueStore

Solution

1. Step 1: Check method to get state store

   The method streams.store is correct.

2. Step 2: Validate key type and store type

   Key type mismatch: passing integer 12345 instead of string key (common for ordersStore).

3. Final Answer:

   Passing integer key instead of string key -> Option A

4. Quick Check:

   Key type must match store definition [OK]

Hint: Use correct key type in store.get(key) [OK]

Common Mistakes:

• Passing int literal without quotes for string keys
• Assuming key type must be string always
• Ignoring exception handling (not mandatory here)

5.

You want to build a real-time dashboard that shows the current count of active users using Kafka Streams interactive queries. Which approach best fits this requirement?

hard

A. Write user statuses to a separate database and query it periodically

B. Store counts in a local variable updated by Kafka Streams processors

C. Use Kafka consumer to read all user status events and aggregate externally

D. Use a ReadOnlyKeyValueStore to store user statuses and query counts live

Solution

1. Step 1: Identify real-time state storage method

   Kafka Streams state stores hold live data accessible via interactive queries.

2. Step 2: Choose best approach for live dashboard

   Using ReadOnlyKeyValueStore allows querying current counts directly without extra DB.

3. Final Answer:

   Use a ReadOnlyKeyValueStore to store user statuses and query counts live -> Option D

4. Quick Check:

   Interactive queries + state store = real-time dashboard [OK]

Hint: Use state store with interactive queries for live data [OK]

Common Mistakes:

• Relying on external DB instead of state store
• Polling Kafka consumer instead of querying store
• Using local variables which are not shared state