Interactive queries in Kafka - Commands & Configuration

Introduction

Interactive queries let you ask a running Kafka Streams application for the current state of its data. This helps you get real-time answers without waiting for data to be stored elsewhere.

When you want to check the current count of events processed by your streaming app.

When you need to get the latest value of a key from your Kafka Streams state store.

When you want to build a dashboard that shows live data from your streaming application.

When you want to avoid querying an external database and instead ask the streaming app directly.

When you want to combine streaming processing with fast, on-demand data lookups.

Commands

Create a Kafka topic named 'pageviews' with 3 partitions to hold streaming data.

Terminal

kafka-topics --create --topic pageviews --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1

Expected Output

Created topic pageviews.

→

--partitions - Number of partitions for parallelism

→

--replication-factor - Number of copies for fault tolerance

Start a producer to send sample pageview events to the 'pageviews' topic.

Terminal

kafka-console-producer --topic pageviews --bootstrap-server localhost:9092

Expected Output

No output (command runs silently)

→

--topic - Topic to send messages to

→

--bootstrap-server - Kafka server address

Query the Kafka Streams app's REST endpoint to get the current state for key '123' from the 'pageviews' store.

Terminal

curl http://localhost:8080/state/pageviews/123

Expected Output

{"key":"123","count":42}

Query the state store for a key '999' that may not exist to see how the app responds.

Terminal

curl http://localhost:8080/state/pageviews/999

Expected Output

{"key":"999","count":0}

Key Concept

Interactive queries let you ask a live Kafka Streams app for its current data state directly, enabling real-time insights without extra storage.

Common Mistakes

Trying to query a state store without enabling interactive queries in the Kafka Streams app.

The app won't expose the state store, so queries will fail or return errors.

Configure the Kafka Streams app with interactive queries enabled and expose a REST endpoint.

Querying a key that does not exist without handling empty or default responses.

You may get null or errors, causing your app or UI to crash.

Handle missing keys gracefully by returning default values like zero or empty objects.

Not matching the topic and store names correctly between the Kafka Streams app and the query URL.

The query will not find the correct store and fail.

Ensure the topic and state store names used in queries exactly match those in the Kafka Streams app.

Summary

Create a Kafka topic to hold streaming data.

Send data to the topic using a producer.

Query the Kafka Streams app's REST endpoint to get live state data for specific keys.

Handle missing keys and ensure the app is configured for interactive queries.