Why Replication factor in Kafka? - Purpose & Use Cases

The Big Idea

What if your important data vanished in a blink? Replication factor stops that from happening.

The Scenario

Imagine you run a small library where you keep only one copy of each book. If that book gets lost or damaged, your readers can't access it anymore.

The Problem

Keeping just one copy means if something goes wrong--like a fire or a misplaced book--you lose all access. Manually making copies and tracking them is slow and easy to mess up.

The Solution

Replication factor in Kafka automatically keeps multiple copies of your data across different servers. If one copy fails, others are ready to serve, so your data stays safe and available without extra work.

Before vs After

✗ Before

store data on one server only
if server fails, data lost

✓ After

set replication.factor=3
Kafka keeps 3 copies automatically

What It Enables

You can trust your data is safe and always accessible, even if some servers fail.

Real Life Example

Think of a photo album saved on three different phones. If one phone breaks, you still have the photos on the other two.

Key Takeaways

Manual single copies risk data loss.

Replication factor creates multiple automatic copies.

This keeps data safe and available without extra effort.

Practice

1.

What does the replication factor in Kafka control?

easy

A. The size limit of each partition

B. The number of copies of each partition stored across brokers

C. The number of partitions in a topic

D. The number of consumers in a consumer group

Solution

1. Step 1: Understand replication factor meaning

   The replication factor defines how many copies of each partition Kafka keeps on different brokers.

2. Step 2: Differentiate from other Kafka settings

   Partitions count and consumer group size are unrelated to replication factor.

3. Final Answer:

   The number of copies of each partition stored across brokers -> Option B

4. Quick Check:

   Replication factor = number of copies [OK]

Hint: Replication factor = copies of data stored [OK]

Common Mistakes:

• Confusing replication factor with partition count
• Thinking replication factor controls consumer groups
• Assuming replication factor limits partition size

2.

Which of the following is the correct way to set the replication factor when creating a Kafka topic using the command line?

kafka-topics.sh --create --topic my-topic --partitions 3 --replication-factor ? --bootstrap-server localhost:9092

easy

A. --replication-factor 3

B. --replication-factor=3

C. --replication_factor 3

D. --replicationFactor 3

Solution

1. Step 1: Check Kafka CLI syntax for replication factor

   The correct flag is --replication-factor with a space before the value.

2. Step 2: Identify incorrect options

   Options with underscores or camelCase are invalid flags; equals sign is not used here.

3. Final Answer:

   --replication-factor 3 -> Option A

4. Quick Check:

   Use double dash and hyphen for replication factor [OK]

Hint: Use double dash and hyphen for flags in Kafka CLI [OK]

Common Mistakes:

• Using underscore instead of hyphen
• Using equals sign instead of space
• Using camelCase flag names

3.

Given a Kafka topic created with --partitions 2 and --replication-factor 3, how many total replicas will Kafka store across the cluster?

medium

A. 6

B. 3

C. 5

D. 2

Solution

1. Step 1: Calculate replicas per partition

   Each partition has 3 replicas because replication factor is 3.

2. Step 2: Multiply by number of partitions

   2 partitions x 3 replicas each = 6 total replicas.

3. Final Answer:

   6 -> Option A

4. Quick Check:

   Partitions x Replication factor = Total replicas [OK]

Hint: Multiply partitions by replication factor [OK]

Common Mistakes:

• Confusing replication factor with total partitions
• Adding instead of multiplying partitions and replicas
• Ignoring replication factor in calculation

4.

What is wrong with this Kafka topic creation command?

kafka-topics.sh --create --topic logs --partitions 3 --replication-factor 5 --bootstrap-server localhost:9092

Assuming the Kafka cluster has only 3 brokers.

medium

A. Partitions must be equal to replication factor

B. Topic name cannot be 'logs'

C. Bootstrap server address is incorrect

D. Replication factor cannot be greater than the number of brokers

Solution

1. Step 1: Understand replication factor limits

   Replication factor must be less than or equal to the number of brokers to ensure each replica is on a different broker.

2. Step 2: Check cluster size vs replication factor

   Replication factor 5 is greater than 3 brokers, which is invalid.

3. Final Answer:

   Replication factor cannot be greater than the number of brokers -> Option D

4. Quick Check:

   Replication factor ≤ brokers [OK]

Hint: Replication factor must not exceed broker count [OK]

Common Mistakes:

• Thinking partitions must equal replication factor
• Ignoring cluster size limits
• Assuming topic names have restrictions here

5.

You want to create a Kafka topic with 4 partitions and a replication factor of 3 for high availability. However, your cluster has only 2 brokers. What is the best solution?

hard

A. Create the topic with 1 partition and replication factor 3

B. Set replication factor to 4 to match partitions

C. Increase the number of brokers to at least 3

D. Create the topic with replication factor 2 instead

Solution

1. Step 1: Check replication factor vs brokers

   Replication factor 3 requires at least 3 brokers to store replicas on different machines.

2. Step 2: Evaluate options

   Reducing replication factor lowers availability; increasing brokers meets requirement.

3. Final Answer:

   Increase the number of brokers to at least 3 -> Option C

4. Quick Check:

   Replication factor ≤ brokers for high availability [OK]

Hint: Match replication factor to broker count for safety [OK]

Common Mistakes:

• Setting replication factor higher than brokers
• Reducing partitions instead of brokers
• Ignoring availability requirements