Subscribing to topics in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does it mean to subscribe to a topic in Kafka?

Subscribing to a topic means telling the Kafka consumer which topics it should listen to for new messages. The consumer will then receive messages from those topics.

beginner

How do you subscribe to multiple topics in Kafka using the consumer API?

You provide a list of topic names to the consumer's subscribe method, like consumer.subscribe(Arrays.asList("topic1", "topic2")). This makes the consumer listen to all listed topics.

intermediate

What is the difference between subscribing to topics and assigning partitions in Kafka?

Subscribing lets Kafka manage which partitions the consumer reads from automatically. Assigning partitions means you manually tell the consumer exactly which partitions to read, giving more control but requiring manual management.

intermediate

What happens if a consumer subscribes to a topic that does not exist yet?

The consumer will keep trying to find the topic. If the topic is created later, the consumer will start receiving messages from it. This allows dynamic topic creation.

beginner

Why is it important to call consumer.poll() after subscribing to topics?

Calling consumer.poll() triggers the consumer to fetch messages from the subscribed topics. Without polling, the consumer won't receive any messages.

What method is used to subscribe a Kafka consumer to topics?

A listen()

B assign()

C subscribe()

D connect()

The subscribe() method is used to tell the consumer which topics to listen to.

If you want Kafka to automatically manage partition assignment, you should:

A Use subscribe() with topic names

B Use assign() with partitions

C Manually poll each partition

D Create topics manually

subscribe() lets Kafka handle partition assignment automatically.

What happens if you subscribe to a topic that does not exist yet?

A Consumer throws an error immediately

B Consumer subscribes to a random topic

C Consumer ignores the topic

D Consumer waits and starts receiving when topic is created

Kafka consumers wait for the topic to be created and then start receiving messages.

Which method must be called regularly to receive messages after subscribing?

A connect()

B poll()

C subscribe()

D commit()

poll() fetches messages from the subscribed topics.

To listen to multiple topics, you should:

A Pass a list of topic names to subscribe()

B Call subscribe() multiple times

C Use assign() with multiple partitions

D Create multiple consumers

You pass a list of topic names to subscribe() to listen to multiple topics.

Explain how subscribing to topics works in Kafka and why it is useful.

Think about how the consumer knows what to listen to.

Describe the difference between subscribing to topics and assigning partitions in Kafka.

Consider who manages the partitions in each case.

Practice

1. What is the main purpose of the subscribe() method in Kafka consumers?

easy

A. To close the Kafka consumer connection

B. To send messages to Kafka topics

C. To listen to one or more Kafka topics and receive messages automatically

D. To create new Kafka topics

Solution

1. Step 1: Understand the role of subscribe()

   The subscribe() method is used by Kafka consumers to specify which topics they want to listen to.

2. Step 2: Identify what subscribe() enables

   Once subscribed, the consumer automatically receives messages from those topics when calling poll().

3. Final Answer:

   To listen to one or more Kafka topics and receive messages automatically -> Option C

4. Quick Check:

   subscribe() = listen to topics [OK]

Hint: subscribe() means start listening to topics [OK]

Common Mistakes:

• Confusing subscribe() with sending messages
• Thinking subscribe() creates topics
• Assuming subscribe() closes connections

2. Which of the following is the correct way to subscribe a Kafka consumer to multiple topics named "topicA" and "topicB"?

easy

A. consumer.subscribe(["topicA", "topicB"])

B. consumer.subscribe("topicA", "topicB")

C. consumer.subscribe("topicA" + "," + "topicB")

D. consumer.subscribe({"topicA", "topicB"})

Solution

1. Step 1: Recall the subscribe() method signature

   In Kafka, subscribe() takes a list (or collection) of topic names as an argument.

2. Step 2: Identify correct syntax for multiple topics

   Passing a list like ["topicA", "topicB"] is the correct way to subscribe to multiple topics.

3. Final Answer:

   consumer.subscribe(["topicA", "topicB"]) -> Option A

4. Quick Check:

   subscribe() needs a list for multiple topics [OK]

Hint: Use a list to subscribe to multiple topics [OK]

Common Mistakes:

• Passing multiple string arguments instead of a list
• Concatenating topic names into one string
• Using curly braces instead of square brackets

3. Given the following Kafka consumer code snippet, what will be the output if the consumer is subscribed to topic "news" and there are two messages "Hello" and "World" in the topic?

consumer.subscribe(["news"])
records = consumer.poll(timeout_ms=1000)
for record in records:
    print(record.value)

medium

A. ['Hello', 'World']

B. Hello\nWorld

C. HelloWorld

D. No output because poll() is missing a loop

Solution

1. Step 1: Understand subscribe() and poll() behavior

   The consumer subscribes to "news" and calls poll() once, which fetches available messages.

2. Step 2: Analyze the for loop printing each record

   The loop prints each message value on its own line, so "Hello" and "World" appear separately.

3. Final Answer:

   Hello\nWorld -> Option B

4. Quick Check:

   poll() returns messages, loop prints each [OK]

Hint: poll() returns messages; loop prints each on new line [OK]

Common Mistakes:

• Thinking poll() returns a list printed directly
• Assuming messages print concatenated without newline
• Believing poll() needs a loop to work at all

4. What is wrong with the following Kafka consumer code snippet?

consumer.subscribe("topic1")
while True:
    records = consumer.poll(1000)
    for record in records:
        print(record.value)

medium

A. subscribe() expects a list, not a single string

B. poll() must be called outside the loop

C. print() cannot be used inside the loop

D. The loop should be a for loop, not while

Solution

1. Step 1: Check subscribe() argument type

   The subscribe() method requires a list of topics, even if subscribing to one topic.

2. Step 2: Identify the error in argument

   Passing a single string instead of a list causes a type error or unexpected behavior.

3. Final Answer:

   subscribe() expects a list, not a single string -> Option A

4. Quick Check:

   subscribe() needs list argument [OK]

Hint: Always pass a list to subscribe(), even for one topic [OK]

Common Mistakes:

• Passing a string instead of a list to subscribe()
• Thinking poll() can't be inside a loop
• Misunderstanding loop type requirements

5. You want to subscribe a Kafka consumer to all topics starting with "logs-" dynamically, without listing each topic name. Which approach correctly achieves this?

hard

A. Use consumer.assign() with topic partitions matching "logs-"

B. Manually list all topics starting with "logs-" in a list and pass to subscribe()

C. Call subscribe() multiple times, once per topic starting with "logs-"

D. Use consumer.subscribe(Pattern.compile("logs-.*")) to subscribe by regex pattern

Solution

1. Step 1: Understand dynamic topic subscription

   Kafka consumers can subscribe to topics matching a regex pattern using subscribe(Pattern).

2. Step 2: Identify correct method for pattern subscription

   Using consumer.subscribe(Pattern.compile("logs-.*")) subscribes to all topics starting with "logs-" automatically.

3. Final Answer:

   Use consumer.subscribe(Pattern.compile("logs-.*")) to subscribe by regex pattern -> Option D

4. Quick Check:

   subscribe() supports regex patterns [OK]

Hint: Use regex pattern with subscribe() for dynamic topic matching [OK]

Common Mistakes:

• Listing topics manually instead of using regex
• Calling subscribe() multiple times incorrectly
• Using assign() which requires partitions, not topics