MirrorMaker 2 concept in Kafka - Mini Project: Build & Apply

Kafka MirrorMaker 2 Basic Setup

📖 Scenario: You work for a company that has two Kafka clusters in different data centers. You want to keep the data in these clusters synchronized automatically.Kafka MirrorMaker 2 is a tool that helps copy data between Kafka clusters, so they stay in sync.

🎯 Goal: Set up a simple MirrorMaker 2 configuration to replicate topics from a source Kafka cluster to a target Kafka cluster.

📋 What You'll Learn

Create a MirrorMaker 2 properties file with source and target cluster details

Configure replication of all topics

Start MirrorMaker 2 with the created configuration

Verify that data is being mirrored from source to target

💡 Why This Matters

🌍 Real World

Companies use MirrorMaker 2 to keep Kafka data synchronized across data centers for disaster recovery and data locality.

💼 Career

Kafka administrators and DevOps engineers often configure MirrorMaker 2 to ensure data availability and consistency across distributed systems.

1

Create MirrorMaker 2 configuration file

Create a file named mm2.properties with these exact lines:
clusters = source, target
source.bootstrap.servers = localhost:9092
target.bootstrap.servers = localhost:9093

Kafka

# Create mm2.properties file with cluster names and bootstrap servers
# Your code here

Need a hint?

Define two clusters named source and target with their bootstrap servers.

2

Configure topic replication

Add these exact lines to mm2.properties to replicate all topics:
source->target.enabled = true
source->target.topics = .*

Kafka

clusters = source, target
source.bootstrap.servers = localhost:9092
target.bootstrap.servers = localhost:9093
# Enable replication from source to target for all topics
# Your code here

Need a hint?

Enable replication and use .* to match all topics.

3

Start MirrorMaker 2

Run this exact command in your terminal to start MirrorMaker 2:
connect-mirror-maker mm2.properties

Kafka

clusters = source, target
source.bootstrap.servers = localhost:9092
target.bootstrap.servers = localhost:9093
source->target.enabled = true
source->target.topics = .*
# Run the MirrorMaker 2 start command below
# Your code here

Need a hint?

Use the connect-mirror-maker command with the mm2.properties file.

4

Verify data replication

Write this exact command to check topics on the target cluster:
kafka-topics --bootstrap-server localhost:9093 --list

Kafka

clusters = source, target
source.bootstrap.servers = localhost:9092
target.bootstrap.servers = localhost:9093
source->target.enabled = true
source->target.topics = .*
connect-mirror-maker mm2.properties
# Check topics on target cluster
# Your code here

Need a hint?

Use kafka-topics with the target cluster's bootstrap server to list topics.