Replication factor in Kafka - Mini Project: Build & Apply

Kafka Topic Replication Factor Setup

📖 Scenario: You are setting up a Kafka topic for a small business application. To ensure data safety, you want to configure the replication factor properly.

🎯 Goal: Create a Kafka topic named orders with 3 partitions and a replication factor of 2. Then verify the topic configuration.

📋 What You'll Learn

Create a Kafka topic named orders with 3 partitions and replication factor 2

Store the topic name in a variable called TOPIC_NAME

Use the kafka-topics.sh command to create the topic with the specified replication factor

Use the kafka-topics.sh command to describe the topic and verify the replication factor

Print the output of the topic description

💡 Why This Matters

🌍 Real World

Kafka topics with replication factor ensure data is copied across multiple servers. This protects against data loss if one server fails.

💼 Career

Kafka administrators and DevOps engineers often configure replication factors to balance data safety and resource use in production systems.

1

Set the Kafka topic name

Create a variable called TOPIC_NAME and set it to the string orders.

Kafka

# Create a variable called TOPIC_NAME and set it to 'orders'
# Your code here

Need a hint?

Use the syntax VAR_NAME="value" to create a variable in bash.

2

Set the number of partitions and replication factor

Create two variables: PARTITIONS set to 3 and REPLICATION_FACTOR set to 2.

Kafka

TOPIC_NAME="orders"
# Create PARTITIONS=3 and REPLICATION_FACTOR=2
# Your code here

Need a hint?

Use the syntax VAR_NAME=value without spaces for integer variables.

3

Create the Kafka topic with replication factor

Use the kafka-topics.sh command with --create, --topic $TOPIC_NAME, --partitions $PARTITIONS, and --replication-factor $REPLICATION_FACTOR to create the topic.

Kafka

TOPIC_NAME="orders"
PARTITIONS=3
REPLICATION_FACTOR=2
# Create the Kafka topic using kafka-topics.sh
# Your code here

Need a hint?

Use --bootstrap-server localhost:9092 to connect to the Kafka server.

4

Describe the topic and print the replication factor

Use the kafka-topics.sh --describe --topic $TOPIC_NAME --bootstrap-server localhost:9092 command and print the output to verify the replication factor.

Kafka

TOPIC_NAME="orders"
PARTITIONS=3
REPLICATION_FACTOR=2
kafka-topics.sh --create --topic "$TOPIC_NAME" --partitions "$PARTITIONS" --replication-factor "$REPLICATION_FACTOR" --bootstrap-server localhost:9092
# Describe the topic and print the output
# Your code here

Need a hint?

The --describe command shows topic details including replication factor.