Geo-replication strategies in Kafka - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the replication factor for a Kafka topic.

Kafka

kafka-topics --create --topic my-topic --partitions 3 --replication-factor [1] --bootstrap-server localhost:9092

Drag options to blanks, or click blank then click option'

A3

B5

C0

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Setting replication factor to 0 disables replication and causes errors.

Using a replication factor higher than the number of brokers.

The replication factor defines how many copies of the data are kept across Kafka brokers. Setting it to 3 means the data is replicated on 3 brokers for fault tolerance.

2fill in blank

medium

Complete the code to configure MirrorMaker 2 to replicate topics from source to target cluster.

Kafka

bin/kafka-mirror-maker.sh --consumer.config consumer.properties --producer.config producer.properties --whitelist [1]

Drag options to blanks, or click blank then click option'

Atarget.*

Btopic1,topic2

Csource.*

D.*

Attempts:

3 left

💡 Hint

Common Mistakes

Using a whitelist that does not match any topics.

Confusing whitelist with blacklist.

The whitelist '.*' means all topics will be replicated from the source cluster to the target cluster.

3fill in blank

hard

Fix the error in the MirrorMaker 2 configuration to enable active-active geo-replication.

Kafka

clusters = [1]
replication.policy.class = org.apache.kafka.connect.mirror.DefaultReplicationPolicy

Drag options to blanks, or click blank then click option'

Asource,target

Bsource

Csource;target

Dsource target

Attempts:

3 left

💡 Hint

Common Mistakes

Using semicolons or spaces instead of commas.

Listing only one cluster disables replication.

The clusters property must list all clusters separated by commas without spaces or other delimiters.

4fill in blank

hard

Fill both blanks to create a topic configuration that enables geo-replication with idempotent producer and exactly-once semantics.

Kafka

enable.idempotence=[1]
processing.guarantee=[2]

Drag options to blanks, or click blank then click option'

Atrue

Bfalse

Cexactly_once

Dat_least_once

Attempts:

3 left

💡 Hint

Common Mistakes

Disabling idempotence causes duplicate messages.

Using at_least_once processing guarantee loses exactly-once benefits.

Setting enable.idempotence to true ensures no duplicate messages. Setting processing.guarantee to exactly_once enables exactly-once semantics for geo-replication.

5fill in blank

hard

Fill all three blanks to configure MirrorMaker 2 with source cluster alias, target cluster alias, and replication factor.

Kafka

clusters = [1]
source.cluster.alias = [2]
target.cluster.alias = [3]

Drag options to blanks, or click blank then click option'

Asource,target

Bus-east

Cus-west

Dsource;target

Attempts:

3 left

💡 Hint

Common Mistakes

Using semicolons instead of commas in clusters.

Mixing up source and target aliases.

The clusters property lists both clusters separated by a comma. The source and target cluster aliases are set to 'us-east' and 'us-west' respectively to identify clusters in geo-replication.

Practice

1. What is the main purpose of geo-replication in Kafka?

easy

A. To encrypt Kafka topics for security

B. To compress Kafka messages for faster transmission

C. To copy data between Kafka clusters in different locations for safety and availability

D. To delete old Kafka data automatically

Solution

1. Step 1: Understand geo-replication concept

   Geo-replication means copying data between clusters in different places to keep data safe and accessible.

2. Step 2: Identify the main goal

   The main goal is to keep data safe and close to users worldwide by copying it across locations.

3. Final Answer:

   To copy data between Kafka clusters in different locations for safety and availability -> Option C

4. Quick Check:

   Geo-replication = Data copying across locations [OK]

Hint: Geo-replication means copying data across locations [OK]

Common Mistakes:

• Confusing geo-replication with data compression
• Thinking geo-replication encrypts data
• Assuming geo-replication deletes data

2. Which Kafka tool is commonly used for geo-replication?

easy

A. MirrorMaker 2

B. Kafka Connect Sink

C. Kafka Streams

D. Kafka Producer API

Solution

1. Step 1: Recall Kafka geo-replication tools

   MirrorMaker 2 is designed to replicate data between Kafka clusters in different locations.

2. Step 2: Differentiate from other tools

   Kafka Streams processes data streams, Kafka Connect Sink moves data to external systems, and Producer API sends data to Kafka but does not replicate clusters.

3. Final Answer:

   MirrorMaker 2 -> Option A

4. Quick Check:

   Geo-replication tool = MirrorMaker 2 [OK]

Hint: MirrorMaker 2 is for replicating Kafka clusters [OK]

Common Mistakes:

• Choosing Kafka Streams which is for processing, not replication
• Confusing Kafka Connect Sink with replication tool
• Selecting Producer API which only sends data

3. Given the following MirrorMaker 2 configuration snippet, what is the effect?

{
  "clusters": {
    "A": {"bootstrap.servers": "a.kafka:9092"},
    "B": {"bootstrap.servers": "b.kafka:9092"}
  },
  "mirrors": [{"source.cluster": "A", "target.cluster": "B", "topics": ["orders"]}]
}

medium

A. Data from cluster B is copied to cluster A for topic 'orders'

B. Data from cluster A is copied to cluster B for topic 'orders'

C. Data from both clusters A and B are merged into a new topic 'orders'

D. No data is copied because topics list is empty

Solution

1. Step 1: Analyze source and target clusters

   The config shows source.cluster as A and target.cluster as B, meaning data flows from A to B.

2. Step 2: Check topics list

   The topics list includes "orders", so only the 'orders' topic is replicated from A to B.

3. Final Answer:

   Data from cluster A is copied to cluster B for topic 'orders' -> Option B

4. Quick Check:

   Source A to target B for 'orders' topic [OK]

Hint: Source cluster data replicates to target cluster [OK]

Common Mistakes:

• Reversing source and target clusters
• Thinking topics list is empty
• Assuming data merges instead of copies

4. You configured MirrorMaker 2 but no data is replicating. Which is the most likely error in this config?

{
  "clusters": {
    "source": {"bootstrap.servers": "source.kafka:9092"},
    "target": {"bootstrap.servers": "target.kafka:9092"}
  },
  "mirrors": [{"source.cluster": "source", "target.cluster": "target", "topics": []}]
}

medium

A. Bootstrap servers are incorrectly formatted

B. MirrorMaker 2 requires only one cluster in config

C. Cluster names must be numeric, not strings

D. Empty topics list means no topics are selected for replication

Solution

1. Step 1: Check topics list in mirror config

   The topics list is empty [], so no topics are selected to replicate.

2. Step 2: Validate other config parts

   Bootstrap servers are correctly formatted, cluster names as strings are valid, and multiple clusters are allowed.

3. Final Answer:

   Empty topics list means no topics are selected for replication -> Option D

4. Quick Check:

   Empty topics list = no replication [OK]

Hint: Empty topics list disables replication [OK]

Common Mistakes:

• Assuming bootstrap.servers format is wrong
• Thinking cluster names must be numbers
• Believing MirrorMaker 2 supports only one cluster

5. You want to set up geo-replication between three Kafka clusters (US, EU, Asia) so that each cluster has all topics from the others. Which strategy should you use?

hard

A. Set up MirrorMaker 2 with a full mesh replication where each cluster replicates to the other two

B. Use a single MirrorMaker 2 instance replicating only from US to EU and Asia

C. Configure MirrorMaker 2 to replicate only the largest topic across clusters

D. Use Kafka Connect Sink to move data between clusters

Solution

1. Step 1: Understand full data availability goal

   Each cluster should have all topics from the others, meaning data must flow both ways between all clusters.

2. Step 2: Choose replication strategy

   A full mesh replication with MirrorMaker 2 means each cluster replicates to the other two, ensuring complete data presence everywhere.

3. Final Answer:

   Set up MirrorMaker 2 with a full mesh replication where each cluster replicates to the other two -> Option A

4. Quick Check:

   Full mesh replication = all clusters have all topics [OK]

Hint: Full mesh replicates all topics between all clusters [OK]

Common Mistakes:

• Using one-way replication only
• Replicating only largest topic, missing others
• Confusing Kafka Connect Sink with cluster replication