Consumer lag monitoring in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the command to list consumer groups in Kafka.

Kafka

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list --[1]

Drag options to blanks, or click blank then click option'

Adescribe

Ball

Cgroup

Dverbose

Attempts:

3 left

💡 Hint

Common Mistakes

Using '--list' with '--describe' together incorrectly

Omitting the '--group' option when describing a group

The correct option is 'group' to specify the consumer group when listing or describing groups.

2fill in blank

medium

Complete the command to describe a specific consumer group named 'my-group'.

Kafka

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group [1]

Drag options to blanks, or click blank then click option'

Amy-group

Btest-group

Cdefault

Dgroup1

Attempts:

3 left

💡 Hint

Common Mistakes

Using a wrong or default group name

Omitting the group name

The consumer group to describe is 'my-group' as specified in the instruction.

3fill in blank

hard

Fix the error in the command to monitor consumer lag by completing the missing option.

Kafka

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group --[1]

Drag options to blanks, or click blank then click option'

Alag

Boffsets

Creset-offsets

Dstate

Attempts:

3 left

💡 Hint

Common Mistakes

Using '--lag' which is not a valid option

Using '--reset-offsets' which changes offsets instead of showing lag

The '--offsets' option is used to show the current offsets and lag for the consumer group.

4fill in blank

hard

Fill both blanks to create a command that resets the consumer group offsets to the earliest position.

Kafka

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-group --[1] --to-[2] earliest --execute

Drag options to blanks, or click blank then click option'

Areset-offsets

Blatest

Cearliest

Ddescribe

Attempts:

3 left

💡 Hint

Common Mistakes

Using '--describe' instead of '--reset-offsets'

Using '--to-latest' instead of '--to-earliest'

The command uses '--reset-offsets' to reset offsets and '--to-earliest' to set them to the earliest position.

5fill in blank

hard

Fill all three blanks to create a command that describes consumer group 'analytics', shows lag, and uses a specific bootstrap server.

Kafka

kafka-consumer-groups.sh --bootstrap-server [1] --describe --group [2] --[3]

Drag options to blanks, or click blank then click option'

Alocalhost:9092

Banalytics

Coffsets

Dmy-group

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong bootstrap server address

Using wrong group name

Omitting the '--offsets' option

The command uses 'localhost:9092' as the bootstrap server, 'analytics' as the group, and '--offsets' to show lag.

Practice

1. What does consumer lag in Kafka indicate?

easy

A. The delay between the latest message produced and the message consumed

B. The number of consumers in a consumer group

C. The total number of messages produced to a topic

D. The size of the Kafka topic in bytes

Solution

1. Step 1: Understand consumer lag meaning

   Consumer lag measures how far behind a consumer is from the latest message produced in the topic partition.

2. Step 2: Differentiate from other metrics

   It is not about consumer count, total messages, or topic size, but about delay in consumption.

3. Final Answer:

   The delay between the latest message produced and the message consumed -> Option A

4. Quick Check:

   Consumer lag = delay behind latest message [OK]

Hint: Lag means delay between produced and consumed messages [OK]

Common Mistakes:

• Confusing lag with consumer count
• Thinking lag is topic size
• Mixing lag with total messages produced

2. Which command correctly shows consumer lag for a group named my-group?

easy

A. kafka-consumer-groups.sh --describe --group my-group

B. kafka-consumer-groups.sh --list --group my-group

C. kafka-topics.sh --describe --group my-group

D. kafka-producer-perf-test.sh --group my-group

Solution

1. Step 1: Identify correct command for consumer lag

   The command to describe consumer group lag is kafka-consumer-groups.sh --describe --group <group-name>.

2. Step 2: Eliminate incorrect commands

   --list lists groups but does not show lag; kafka-topics.sh is for topics, not groups; kafka-producer-perf-test.sh is unrelated.

3. Final Answer:

   kafka-consumer-groups.sh --describe --group my-group -> Option A

4. Quick Check:

   Describe group shows lag [OK]

Hint: Use --describe with --group to see lag [OK]

Common Mistakes:

• Using --list instead of --describe
• Confusing topic commands with consumer group commands
• Using producer commands for consumer lag

3. Given this output from kafka-consumer-groups.sh --describe --group my-group:

TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG  CONSUMER-ID
my-topic       0          100             105             5    consumer-1
my-topic       1          200             200             0    consumer-1

What is the total lag for the consumer group?

medium

A. 105

B. 5

C. 0

D. 205

Solution

1. Step 1: Identify lag per partition

   Partition 0 lag is 5, partition 1 lag is 0.

2. Step 2: Calculate total lag

   Total lag = 5 + 0 = 5 messages behind.

3. Final Answer:

   5 -> Option B

4. Quick Check:

   Total lag = sum of partition lags = 5 [OK]

Hint: Add lag values from all partitions for total lag [OK]

Common Mistakes:

• Using LOG-END-OFFSET as lag
• Confusing CURRENT-OFFSET with lag
• Adding offsets instead of lag values

4. You run kafka-consumer-groups.sh --describe --group my-group but see no lag column. What is the likely cause?

medium

A. The Kafka broker is down

B. The topic does not exist

C. The command syntax is incorrect

D. The consumer group has no committed offsets yet

Solution

1. Step 1: Understand missing lag column reason

   If the consumer group has no committed offsets, lag cannot be calculated and may not show.

2. Step 2: Exclude other causes

   Broker down or wrong syntax usually cause errors, not missing lag column; topic existence affects consumption but not lag display here.

3. Final Answer:

   The consumer group has no committed offsets yet -> Option D

4. Quick Check:

   No committed offsets = no lag shown [OK]

Hint: No committed offsets means lag won't appear [OK]

Common Mistakes:

• Assuming broker down causes missing lag column
• Thinking syntax error causes missing lag column
• Confusing topic existence with lag display

5. You notice high consumer lag for a group processing a topic with 3 partitions. The lag per partition is 100, 0, and 50 respectively. What is the best approach to reduce lag quickly?

hard

A. Delete and recreate the consumer group

B. Increase the topic partition count to 10

C. Add more consumers to the group to increase parallel processing

D. Restart the Kafka broker to clear lag

Solution

1. Step 1: Analyze lag distribution

   Partitions have uneven lag: 100, 0, 50. Some partitions are behind.

2. Step 2: Choose action to reduce lag

   Adding consumers can help process partitions faster by parallel consumption.

3. Step 3: Evaluate other options

   Increasing partitions alone won't reduce lag immediately; restarting broker or deleting group won't fix lag directly.

4. Final Answer:

   Add more consumers to the group to increase parallel processing -> Option C

5. Quick Check:

   More consumers = faster lag reduction [OK]

Hint: Add consumers to process partitions faster [OK]

Common Mistakes:

• Thinking restarting broker clears lag
• Believing increasing partitions instantly reduces lag
• Deleting group removes offsets but not lag solution