KStream and KTable concepts in Kafka - Mini Project: Build & Apply

Understanding KStream and KTable Concepts

📖 Scenario: You are working with Kafka Streams to process real-time data from a store's sales system. You want to understand how to use KStream and KTable to handle sales events and maintain the latest product inventory.

🎯 Goal: Build a simple Kafka Streams application that creates a KStream for sales events and a KTable for product inventory, then join them to enrich sales with current inventory data.

📋 What You'll Learn

Create a KStream named salesStream with sample sales data

Create a KTable named inventoryTable with product inventory data

Join salesStream and inventoryTable on product ID

Print the joined results showing sales enriched with inventory info

💡 Why This Matters

🌍 Real World

Retail companies use Kafka Streams to process sales and inventory data in real time to make quick decisions about stock and promotions.

💼 Career

Understanding KStream and KTable is essential for roles in data engineering and real-time analytics using Kafka.

1

Create the salesStream with sample sales data

Create a KStream<String, Integer> called salesStream with these entries: ("product1", 3), ("product2", 5), and ("product3", 2).

Kafka

# Create salesStream with sample sales data
# Your code here

Need a hint?

Use builder.stream() to create the KStream from the topic "sales-topic".

2

Create the inventoryTable with product inventory data

Create a KTable<String, Integer> called inventoryTable with these entries: ("product1", 10), ("product2", 20), and ("product3", 15).

Kafka

KStream salesStream = builder.stream("sales-topic");
// For this exercise, assume salesStream contains: ("product1", 3), ("product2", 5), ("product3", 2)

# Create inventoryTable with product inventory data
# Your code here

Need a hint?

Use builder.table() to create the KTable from the topic "inventory-topic".

3

Join salesStream with inventoryTable

Use salesStream.join(inventoryTable, (saleQty, inventoryQty) -> saleQty + inventoryQty) to create a joined stream called enrichedSales that adds sale quantity and inventory quantity.

Kafka

KStream salesStream = builder.stream("sales-topic");
// For this exercise, assume salesStream contains: ("product1", 3), ("product2", 5), ("product3", 2)

KTable inventoryTable = builder.table("inventory-topic");
// For this exercise, assume inventoryTable contains: ("product1", 10), ("product2", 20), ("product3", 15)

# Join salesStream with inventoryTable to create enrichedSales
# Your code here

Need a hint?

Use the join method on salesStream with inventoryTable and a lambda that adds the two quantities.

4

Print the joined results

Use enrichedSales.foreach((product, totalQty) -> System.out.println(product + ": " + totalQty)) to print each product and its total quantity.

Kafka

KStream salesStream = builder.stream("sales-topic");
// For this exercise, assume salesStream contains: ("product1", 3), ("product2", 5), ("product3", 2)

KTable inventoryTable = builder.table("inventory-topic");
// For this exercise, assume inventoryTable contains: ("product1", 10), ("product2", 20), ("product3", 15)

KStream enrichedSales = salesStream.join(inventoryTable, (saleQty, inventoryQty) -> saleQty + inventoryQty);

# Print the enrichedSales stream
# Your code here

Need a hint?

Use foreach on enrichedSales to print each product and its total quantity.