MirrorMaker 2 concept in Kafka - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is MirrorMaker 2 in Kafka?

MirrorMaker 2 is a tool in Kafka used to replicate data between Kafka clusters. It helps keep data synchronized across different environments or data centers.

intermediate

How does MirrorMaker 2 differ from MirrorMaker 1?

MirrorMaker 2 is built on the Kafka Connect framework, offering better scalability, fault tolerance, and easier configuration compared to MirrorMaker 1, which was a simpler, standalone tool.

intermediate

What are the main components of MirrorMaker 2?

MirrorMaker 2 uses Kafka Connect source and sink connectors to read from the source cluster and write to the target cluster. It also manages offsets and consumer groups to ensure reliable replication.

advanced

Why is offset translation important in MirrorMaker 2?

Offset translation allows MirrorMaker 2 to map consumer offsets from the source cluster to the target cluster, so consumers can continue reading messages seamlessly after failover or migration.

advanced

Can MirrorMaker 2 handle bi-directional replication?

Yes, MirrorMaker 2 supports active-active setups where data is replicated in both directions between clusters, helping with high availability and disaster recovery.

What framework is MirrorMaker 2 built on?

A Kafka Connect

B Zookeeper

C Kafka Streams

D Apache Flink

MirrorMaker 2 uses Kafka Connect framework to manage connectors for replication.

What is the main purpose of MirrorMaker 2?

A To monitor Kafka brokers

B To process streaming data

C To replicate data between Kafka clusters

D To store data permanently

MirrorMaker 2 replicates data between Kafka clusters to keep them synchronized.

Which feature helps MirrorMaker 2 maintain consumer position after failover?

A Offset translation

B Topic compaction

C Partition reassignment

D Consumer lag monitoring

Offset translation maps consumer offsets from source to target cluster.

MirrorMaker 2 supports which type of replication?

A Uni-directional only

B Bi-directional

C File-based

D Database replication

MirrorMaker 2 supports bi-directional replication for active-active cluster setups.

Which of these is NOT a component of MirrorMaker 2?

A Source connector

B Sink connector

C Offset translator

D Kafka Streams processor

MirrorMaker 2 does not use Kafka Streams processors; it uses Kafka Connect connectors.

Explain how MirrorMaker 2 replicates data between Kafka clusters.

Think about how data moves from one cluster to another and how consumer positions are kept.

Describe the benefits of using MirrorMaker 2 over MirrorMaker 1.

Consider improvements in architecture and reliability.

Practice

1. What is the main purpose of Kafka MirrorMaker 2?

easy

A. To copy data between Kafka clusters for backup and availability

B. To create new Kafka topics automatically

C. To monitor Kafka cluster health

D. To compress Kafka messages for storage

Solution

1. Step 1: Understand MirrorMaker 2 role

   MirrorMaker 2 is designed to replicate data from one Kafka cluster to another to ensure data safety and availability.

2. Step 2: Compare options with MirrorMaker 2 function

   Only To copy data between Kafka clusters for backup and availability describes copying data between clusters, which matches MirrorMaker 2's purpose.

3. Final Answer:

   To copy data between Kafka clusters for backup and availability -> Option A

4. Quick Check:

   MirrorMaker 2 purpose = Data replication [OK]

Hint: MirrorMaker 2 copies data between clusters for safety [OK]

Common Mistakes:

• Confusing MirrorMaker 2 with monitoring tools
• Thinking it creates topics automatically
• Assuming it compresses messages

2. Which configuration is required to specify the source Kafka cluster in MirrorMaker 2?

easy

A. producer.bootstrap.servers

B. replica.bootstrap.servers

C. mirror.bootstrap.servers

D. consumer.bootstrap.servers

Solution

1. Step 1: Identify source cluster connection

   MirrorMaker 2 uses consumer configs to read from the source Kafka cluster.

2. Step 2: Match config key to source cluster

   The correct config key for source cluster is consumer.bootstrap.servers, which tells MirrorMaker 2 where to consume data from.

3. Final Answer:

   consumer.bootstrap.servers -> Option D

4. Quick Check:

   Source cluster = consumer.bootstrap.servers [OK]

Hint: Source cluster uses consumer.bootstrap.servers config [OK]

Common Mistakes:

• Using producer.bootstrap.servers for source cluster
• Confusing config keys with mirror.bootstrap.servers
• Assuming replica.bootstrap.servers is valid

3. Given this MirrorMaker 2 config snippet:

{
  "source.cluster.bootstrap.servers": "source:9092",
  "target.cluster.bootstrap.servers": "target:9092",
  "topics": "topicA,topicB",
  "num.streams": 3
}

How many parallel streams will MirrorMaker 2 use to copy data?

medium

A. 3

B. 2

C. 1

D. 4

Solution

1. Step 1: Locate num.streams setting

   The config sets "num.streams" to 3, which controls how many parallel streams MirrorMaker 2 uses.

2. Step 2: Understand effect of num.streams

   More streams mean more parallelism in copying data, so MirrorMaker 2 will use 3 streams here.

3. Final Answer:

   3 -> Option A

4. Quick Check:

   num.streams = 3 [OK]

Hint: Check num.streams value for parallelism count [OK]

Common Mistakes:

• Assuming default 1 stream if not read carefully
• Confusing topics count with streams count
• Ignoring num.streams config

4. You configured MirrorMaker 2 with:

topics = "topic1,topic2"
num.streams = 0

What is the problem with this configuration?

medium

A. topics must be a single topic, not multiple

B. num.streams cannot be zero; it must be at least 1

C. topics should be configured as a list, not a string

D. num.streams must be an even number

Solution

1. Step 1: Check num.streams value validity

   num.streams controls parallel streams and must be at least 1 to work properly.

2. Step 2: Identify error in config

   Setting num.streams to 0 disables streams, causing MirrorMaker 2 to fail or not copy data.

3. Final Answer:

   num.streams cannot be zero; it must be at least 1 -> Option B

4. Quick Check:

   num.streams ≥ 1 required [OK]

Hint: num.streams must be ≥ 1, zero disables copying [OK]

Common Mistakes:

• Thinking topics must be single only
• Assuming num.streams can be zero
• Believing num.streams must be even

5. You want MirrorMaker 2 to replicate only topics starting with "logs-" from source to target cluster. Which config setting achieves this?

hard

A. "topics": "logs-*"

B. "topics": "logs-"

C. "topics.regex": "logs-.*"

D. "topics.regex": "^logs$"

Solution

1. Step 1: Understand topic filtering in MirrorMaker 2

   MirrorMaker 2 supports filtering topics by regex using the "topics.regex" config.

2. Step 2: Match regex to desired pattern

   To match all topics starting with "logs-", the regex "logs-.*" correctly matches any topic beginning with "logs-".

3. Step 3: Compare options

   "topics": "logs-*" uses "topics" with wildcard which is not supported; "topics": "logs-" matches exact topic name; "topics.regex": "^logs$" matches only "logs" exactly.

4. Final Answer:

   "topics.regex": "logs-.*" -> Option C

5. Quick Check:

   Use topics.regex with proper regex for filtering [OK]

Hint: Use topics.regex with regex pattern to filter topics [OK]

Common Mistakes:

• Using wildcard * in topics instead of regex
• Confusing exact match with regex match
• Using incorrect regex that matches wrong topics