Geo-replication strategies in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is geo-replication in Kafka?

Geo-replication in Kafka means copying data across different geographic locations to improve availability and disaster recovery.

beginner

Name two common Kafka geo-replication tools or methods.

Two common methods are MirrorMaker 2 and Confluent Replicator, which help copy topics between Kafka clusters in different regions.

intermediate

What is the main difference between active-active and active-passive geo-replication?

Active-active means multiple clusters can read and write data simultaneously, while active-passive means only one cluster writes and others are backups.

intermediate

Why is data consistency a challenge in geo-replication?

Because data is copied across distant locations, network delays and conflicts can cause data to be out of sync or inconsistent temporarily.

advanced

How does MirrorMaker 2 handle topic replication?

MirrorMaker 2 uses the Kafka Connect framework to replicate topics, consumer offsets, and configurations between clusters, supporting automatic failover.

What is the purpose of geo-replication in Kafka?

A To delete old Kafka logs

B To compress Kafka messages

C To encrypt Kafka topics

D To copy data across regions for availability and disaster recovery

Geo-replication copies data across regions to keep it safe and available even if one location fails.

Which tool is commonly used for Kafka geo-replication?

A MirrorMaker 2

B Kafka Streams

C Kafka Connect Sink

D ZooKeeper

MirrorMaker 2 is designed specifically to replicate Kafka topics between clusters.

In active-passive geo-replication, which cluster handles writes?

A Both clusters simultaneously

B Only the passive cluster

C Only the active cluster

D Neither cluster

Only the active cluster accepts writes; the passive cluster is a backup.

What challenge does network latency cause in geo-replication?

A Data inconsistency

B Faster replication

C More storage space

D Automatic topic creation

Network delays can cause data to be temporarily out of sync between clusters.

MirrorMaker 2 is built on which Kafka framework?

A Kafka Streams

B Kafka Connect

C Kafka Producer API

D Kafka Consumer API

MirrorMaker 2 uses Kafka Connect to manage replication tasks.

Explain the difference between active-active and active-passive geo-replication strategies in Kafka.

Think about how many clusters can write data at the same time.

Describe how MirrorMaker 2 helps with Kafka geo-replication.

Focus on the framework MirrorMaker 2 is built on and what it replicates.

Practice

1. What is the main purpose of geo-replication in Kafka?

easy

A. To encrypt Kafka topics for security

B. To compress Kafka messages for faster transmission

C. To copy data between Kafka clusters in different locations for safety and availability

D. To delete old Kafka data automatically

Solution

1. Step 1: Understand geo-replication concept

   Geo-replication means copying data between clusters in different places to keep data safe and accessible.

2. Step 2: Identify the main goal

   The main goal is to keep data safe and close to users worldwide by copying it across locations.

3. Final Answer:

   To copy data between Kafka clusters in different locations for safety and availability -> Option C

4. Quick Check:

   Geo-replication = Data copying across locations [OK]

Hint: Geo-replication means copying data across locations [OK]

Common Mistakes:

• Confusing geo-replication with data compression
• Thinking geo-replication encrypts data
• Assuming geo-replication deletes data

2. Which Kafka tool is commonly used for geo-replication?

easy

A. MirrorMaker 2

B. Kafka Connect Sink

C. Kafka Streams

D. Kafka Producer API

Solution

1. Step 1: Recall Kafka geo-replication tools

   MirrorMaker 2 is designed to replicate data between Kafka clusters in different locations.

2. Step 2: Differentiate from other tools

   Kafka Streams processes data streams, Kafka Connect Sink moves data to external systems, and Producer API sends data to Kafka but does not replicate clusters.

3. Final Answer:

   MirrorMaker 2 -> Option A

4. Quick Check:

   Geo-replication tool = MirrorMaker 2 [OK]

Hint: MirrorMaker 2 is for replicating Kafka clusters [OK]

Common Mistakes:

• Choosing Kafka Streams which is for processing, not replication
• Confusing Kafka Connect Sink with replication tool
• Selecting Producer API which only sends data

3. Given the following MirrorMaker 2 configuration snippet, what is the effect?

{
  "clusters": {
    "A": {"bootstrap.servers": "a.kafka:9092"},
    "B": {"bootstrap.servers": "b.kafka:9092"}
  },
  "mirrors": [{"source.cluster": "A", "target.cluster": "B", "topics": ["orders"]}]
}

medium

A. Data from cluster B is copied to cluster A for topic 'orders'

B. Data from cluster A is copied to cluster B for topic 'orders'

C. Data from both clusters A and B are merged into a new topic 'orders'

D. No data is copied because topics list is empty

Solution

1. Step 1: Analyze source and target clusters

   The config shows source.cluster as A and target.cluster as B, meaning data flows from A to B.

2. Step 2: Check topics list

   The topics list includes "orders", so only the 'orders' topic is replicated from A to B.

3. Final Answer:

   Data from cluster A is copied to cluster B for topic 'orders' -> Option B

4. Quick Check:

   Source A to target B for 'orders' topic [OK]

Hint: Source cluster data replicates to target cluster [OK]

Common Mistakes:

• Reversing source and target clusters
• Thinking topics list is empty
• Assuming data merges instead of copies

4. You configured MirrorMaker 2 but no data is replicating. Which is the most likely error in this config?

{
  "clusters": {
    "source": {"bootstrap.servers": "source.kafka:9092"},
    "target": {"bootstrap.servers": "target.kafka:9092"}
  },
  "mirrors": [{"source.cluster": "source", "target.cluster": "target", "topics": []}]
}

medium

A. Bootstrap servers are incorrectly formatted

B. MirrorMaker 2 requires only one cluster in config

C. Cluster names must be numeric, not strings

D. Empty topics list means no topics are selected for replication

Solution

1. Step 1: Check topics list in mirror config

   The topics list is empty [], so no topics are selected to replicate.

2. Step 2: Validate other config parts

   Bootstrap servers are correctly formatted, cluster names as strings are valid, and multiple clusters are allowed.

3. Final Answer:

   Empty topics list means no topics are selected for replication -> Option D

4. Quick Check:

   Empty topics list = no replication [OK]

Hint: Empty topics list disables replication [OK]

Common Mistakes:

• Assuming bootstrap.servers format is wrong
• Thinking cluster names must be numbers
• Believing MirrorMaker 2 supports only one cluster

5. You want to set up geo-replication between three Kafka clusters (US, EU, Asia) so that each cluster has all topics from the others. Which strategy should you use?

hard

A. Set up MirrorMaker 2 with a full mesh replication where each cluster replicates to the other two

B. Use a single MirrorMaker 2 instance replicating only from US to EU and Asia

C. Configure MirrorMaker 2 to replicate only the largest topic across clusters

D. Use Kafka Connect Sink to move data between clusters

Solution

1. Step 1: Understand full data availability goal

   Each cluster should have all topics from the others, meaning data must flow both ways between all clusters.

2. Step 2: Choose replication strategy

   A full mesh replication with MirrorMaker 2 means each cluster replicates to the other two, ensuring complete data presence everywhere.

3. Final Answer:

   Set up MirrorMaker 2 with a full mesh replication where each cluster replicates to the other two -> Option A

4. Quick Check:

   Full mesh replication = all clusters have all topics [OK]

Hint: Full mesh replicates all topics between all clusters [OK]

Common Mistakes:

• Using one-way replication only
• Replicating only largest topic, missing others
• Confusing Kafka Connect Sink with cluster replication