Error handling in clients in Kafka - Step-by-Step Execution

Process Flow - Error handling in clients

Start Client

↓

Send/Receive Message

↓

Error Occurs?

No→Continue Normal Flow

Yes↓

Catch Error

↓

Handle Error

↓

Retry or Log or Alert

↓

Decide to Continue or Exit

↩Back to Send/Receive or Exit

The client sends or receives messages, checks for errors, catches them, handles them by retrying, logging, or alerting, then decides to continue or stop.

Execution Sample

Kafka

try {
  consumer.poll();
} catch (Exception e) {
  log.error("Error consuming message", e);
  // retry or alert
}

This code tries to poll messages from Kafka, catches any exceptions, logs the error, and can retry or alert.

Process Table

Step	Action	Error Occurs?	Error Caught?	Handling Action	Next Step
1	Start client and poll messages	No	No	None	Continue normal flow
2	Poll messages	Yes (e.g., network error)	Yes	Log error and retry	Retry polling
3	Retry polling	No	No	None	Continue normal flow
4	Poll messages	Yes (e.g., deserialization error)	Yes	Log error and alert	Decide to continue or exit
5	Decide to exit due to repeated errors	N/A	N/A	Exit client	Stop execution

💡 Client stops after deciding to exit due to repeated errors

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
errorOccurred	false	false	true	false	true	true
retryCount	0	0	1	1	1	1
clientRunning	true	true	true	true	true	false

Key Moments - 3 Insights

Why do we catch errors immediately after polling messages?

Because polling can fail due to network or data issues, catching errors right after polling lets us handle them before continuing, as shown in execution_table step 2.

What happens if the error keeps occurring after retries?

The client logs the error, alerts if needed, and may decide to stop running, as seen in execution_table step 4 and 5.

Why track retryCount variable?

To limit how many times the client retries polling before giving up, preventing infinite loops, as shown in variable_tracker.

Visual Quiz - 3 Questions

Look at the execution_table, at which step does the client first catch an error?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Check the 'Error Caught?' column in execution_table rows

According to variable_tracker, what is the value of retryCount after step 3?

A 1

B 0

C 2

D Undefined

💡 Hint

Look at retryCount row and After Step 3 column in variable_tracker

If the client never catches an error, what would happen to clientRunning variable?

A It would become false immediately

B It would toggle between true and false

C It would stay true and continue running

D It would cause an error

💡 Hint

Refer to variable_tracker clientRunning values when no errors occur

Concept Snapshot

try { poll messages } catch(error) { handle error }
Handle errors by logging, retrying, or alerting
Track retries to avoid infinite loops
Decide to continue or stop client based on error severity
Keep client running if no errors occur

Full Transcript

This visual execution shows how a Kafka client handles errors during message polling. The client starts polling messages. If no error occurs, it continues normally. If an error occurs, it is caught immediately. The client then logs the error and may retry polling. If errors persist, it alerts and decides whether to continue or stop. Variables like errorOccurred, retryCount, and clientRunning track the client's state through these steps. Key moments include catching errors right after polling, handling repeated errors by stopping, and tracking retries to prevent infinite loops.

Practice

1. What is the main purpose of error handling in Kafka clients?

easy

A. To increase the speed of message processing

B. To keep the client stable and responsive during failures

C. To reduce the size of messages sent

D. To automatically create Kafka topics

Solution

1. Step 1: Understand error handling role

   Error handling helps the client manage unexpected problems without crashing.

2. Step 2: Identify the main goal in Kafka clients

   Keeping the client stable and responsive ensures smooth operation even if errors occur.

3. Final Answer:

   To keep the client stable and responsive during failures -> Option B

4. Quick Check:

   Error handling = stability and responsiveness [OK]

Hint: Error handling keeps your client stable and responsive [OK]

Common Mistakes:

• Thinking error handling speeds up processing
• Confusing error handling with message size control
• Assuming error handling creates topics automatically

2. Which syntax correctly wraps a Kafka producer send call to catch exceptions in Java?

easy

A. producer.send(record).catch(e => console.log(e));

B. if (producer.send(record)) { } else { handleError(); }

C. try { producer.send(record); } catch (Exception e) { e.printStackTrace(); }

D. try catch producer.send(record);

Solution

1. Step 1: Identify correct Java try-catch syntax

   Java requires try { ... } catch (Exception e) { ... } to handle exceptions.

2. Step 2: Match syntax with Kafka producer send call

   try { producer.send(record); } catch (Exception e) { e.printStackTrace(); } correctly wraps the send call in try-catch block.

3. Final Answer:

   try { producer.send(record); } catch (Exception e) { e.printStackTrace(); } -> Option C

4. Quick Check:

   Java try-catch syntax = try { producer.send(record); } catch (Exception e) { e.printStackTrace(); } [OK]

Hint: Use try-catch blocks in Java to catch Kafka errors [OK]

Common Mistakes:

• Using if-else instead of try-catch for exceptions
• Using JavaScript syntax in Java code
• Missing braces in try-catch block

3. Given this Java Kafka consumer code snippet, what will be printed if a network error occurs during poll?

try {
  ConsumerRecords records = consumer.poll(Duration.ofMillis(100));
  for (var record : records) {
    System.out.println(record.value());
  }
} catch (WakeupException e) {
  System.out.println("Consumer interrupted");
} catch (Exception e) {
  System.out.println("Error during poll");
}

medium

A. Error during poll

B. Consumer interrupted

C. No output

D. Exception stack trace

Solution

1. Step 1: Identify exception types caught

   WakeupException prints "Consumer interrupted"; all other exceptions print "Error during poll".

2. Step 2: Determine exception for network error

   Network errors throw general exceptions, not WakeupException.

3. Final Answer:

   Error during poll -> Option A

4. Quick Check:

   Network error triggers general Exception = Error during poll [OK]

Hint: General exceptions print "Error during poll" [OK]

Common Mistakes:

• Confusing WakeupException with network errors
• Expecting no output on error
• Assuming stack trace prints automatically

4. Identify the error in this Kafka producer error handling code snippet:

try {
  producer.send(record);
} catch (KafkaException e) {
  System.out.println("Error sending message");
}
producer.close();

medium

A. producer.send() does not throw exceptions

B. Catch block should catch Exception, not KafkaException

C. Missing return statement after catch

D. producer.close() should be inside finally block

Solution

1. Step 1: Check resource cleanup

   producer.close() must run regardless of exceptions to free resources.

2. Step 2: Use finally block for cleanup

   Placing close() inside finally ensures it always executes.

3. Final Answer:

   producer.close() should be inside finally block -> Option D

4. Quick Check:

   Always close resources in finally block [OK]

Hint: Put cleanup code in finally block to always run [OK]

Common Mistakes:

• Closing producer only on success
• Catching too broad or wrong exceptions
• Assuming send() never throws exceptions

5. You want to build a Kafka consumer that retries polling up to 3 times on recoverable errors, then logs and stops. Which approach correctly implements this error handling?

hard

A. Use a loop with try-catch around poll; on exception increment retry count; break after 3 retries; finally close consumer

B. Call consumer.poll() once; if exception, immediately close consumer without retry

C. Wrap poll in try-catch but ignore exceptions and continue polling forever

D. Use multiple nested try-catch blocks without retry logic

Solution

1. Step 1: Understand retry logic requirements

   Retry polling up to 3 times on recoverable errors, then stop and log.

2. Step 2: Identify correct control flow

   Loop with try-catch, count retries, break after 3 attempts, and close consumer in finally block is correct.

3. Final Answer:

   Use a loop with try-catch around poll; on exception increment retry count; break after 3 retries; finally close consumer -> Option A

4. Quick Check:

   Retry with loop and try-catch = Use a loop with try-catch around poll; on exception increment retry count; break after 3 retries; finally close consumer [OK]

Hint: Use loop with try-catch and retry count for controlled retries [OK]

Common Mistakes:

• Not retrying on recoverable errors
• Ignoring exceptions and continuing endlessly
• Closing consumer too early without retries