SASL authentication in Kafka - Mini Project: Build & Apply

SASL Authentication Setup for Kafka Client

📖 Scenario: You are setting up a Kafka client that needs to connect securely to a Kafka broker using SASL authentication. SASL helps verify the client's identity before allowing access.

🎯 Goal: Configure a Kafka client with SASL authentication using the PLAIN mechanism to connect to a Kafka broker.

📋 What You'll Learn

Create a Kafka client configuration dictionary with bootstrap servers

Add SASL authentication configuration with mechanism and credentials

Use the configuration to create a Kafka consumer

Print the SASL mechanism used to confirm setup

💡 Why This Matters

🌍 Real World

Kafka clients often need SASL authentication to securely connect to brokers in real-world applications like messaging systems and data pipelines.

💼 Career

Understanding SASL authentication setup is important for roles involving Kafka administration, backend development, and secure data streaming.

1

Create Kafka client configuration

Create a dictionary called config with the key 'bootstrap.servers' set to 'localhost:9092'.

Kafka

# Create a dictionary called config with bootstrap.servers set to 'localhost:9092'
# Your code here

Hint

The bootstrap.servers key tells the client where the Kafka broker is located.

2

Add SASL authentication configuration

Add the following keys to the config dictionary: 'security.protocol' set to 'SASL_PLAINTEXT', 'sasl.mechanism' set to 'PLAIN', 'sasl.username' set to 'user1', and 'sasl.password' set to 'pass123'.

Kafka

config = {'bootstrap.servers': 'localhost:9092'}
# Add SASL authentication keys to config
# Your code here

Hint

These keys configure the client to use SASL with the PLAIN mechanism and provide the username and password.

3

Create Kafka consumer with SASL config

Import KafkaConsumer from kafka and create a consumer called consumer using the config dictionary.

Kafka

config = {'bootstrap.servers': 'localhost:9092'}
config['security.protocol'] = 'SASL_PLAINTEXT'
config['sasl.mechanism'] = 'PLAIN'
config['sasl.username'] = 'user1'
config['sasl.password'] = 'pass123'

# Import KafkaConsumer and create consumer with config
# Your code here

Hint

Use the KafkaConsumer constructor with the SASL config keys as parameters.

4

Print SASL mechanism used

Print the string 'SASL mechanism used: ' followed by the value of config['sasl.mechanism'].

Kafka

from kafka import KafkaConsumer

config = {'bootstrap.servers': 'localhost:9092'}
config['security.protocol'] = 'SASL_PLAINTEXT'
config['sasl.mechanism'] = 'PLAIN'
config['sasl.username'] = 'user1'
config['sasl.password'] = 'pass123'

consumer = KafkaConsumer(bootstrap_servers=config['bootstrap.servers'],
                         security_protocol=config['security.protocol'],
                         sasl_mechanism=config['sasl.mechanism'],
                         sasl_plain_username=config['sasl.username'],
                         sasl_plain_password=config['sasl.password'])

# Print the SASL mechanism used
# Your code here

Hint

Use a print statement with an f-string to show the SASL mechanism.

Practice

1. What is the main purpose of SASL authentication in Kafka?

easy

A. To compress messages for faster transmission

B. To verify user identity before allowing access

C. To balance load between Kafka brokers

D. To encrypt data stored on disk

Solution

1. Step 1: Understand SASL role in Kafka

   SASL is a protocol used to verify who is connecting to Kafka, ensuring only authorized users can access it.

2. Step 2: Differentiate from other Kafka features

   Compression, load balancing, and disk encryption are unrelated to SASL authentication.

3. Final Answer:

   To verify user identity before allowing access -> Option B

4. Quick Check:

   SASL = user verification [OK]

Hint: SASL is about user identity, not data or load [OK]

Common Mistakes:

• Confusing SASL with encryption
• Thinking SASL manages data compression
• Mixing SASL with load balancing

2. Which of the following is the correct way to specify SASL mechanism in Kafka client configuration?

easy

A. security.protocol = SASL_PLAINTEXT

B. sasl.config = PLAIN

C. sasl.mechanism = PLAIN

D. security.mechanism = SASL

Solution

1. Step 1: Identify correct property for SASL mechanism

   The Kafka client uses sasl.mechanism to specify the SASL mechanism like PLAIN, SCRAM, etc.

2. Step 2: Check other options for correctness

   security.protocol sets protocol type, not mechanism; sasl.config is invalid; security.mechanism is not a valid property.

3. Final Answer:

   sasl.mechanism = PLAIN -> Option C

4. Quick Check:

   sasl.mechanism sets SASL type [OK]

Hint: Use sasl.mechanism to set SASL type like PLAIN [OK]

Common Mistakes:

• Using security.protocol instead of sasl.mechanism
• Writing sasl.config instead of sasl.mechanism
• Confusing property names

3. Given this Kafka client configuration snippet:

props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "SCRAM-SHA-256");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user1\" password=\"pass1\";");

What will happen when the client tries to connect?

medium

A. Client connects using PLAINTEXT mechanism

B. Client connects but without encryption

C. Client fails due to missing SSL configuration

D. Client connects securely using SCRAM-SHA-256 with encrypted channel

Solution

1. Step 1: Analyze security.protocol setting

   Setting security.protocol to SASL_SSL means the client uses SASL authentication over SSL encrypted connection.

2. Step 2: Check SASL mechanism and JAAS config

   The mechanism is SCRAM-SHA-256 with proper JAAS config providing username and password, so authentication will succeed if credentials are correct.

3. Final Answer:

   Client connects securely using SCRAM-SHA-256 with encrypted channel -> Option D

4. Quick Check:

   SASL_SSL + SCRAM = secure auth [OK]

Hint: SASL_SSL means secure auth with encryption [OK]

Common Mistakes:

• Assuming SASL_SSL means no encryption
• Ignoring JAAS config for credentials
• Confusing SCRAM with PLAINTEXT

4. You have this Kafka client config:

props.put("security.protocol", "SASL_PLAINTEXT");
props.put("sasl.mechanism", "PLAIN");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin123\";");

The client fails to authenticate. What is the most likely cause?

medium

A. Broker does not support SASL_PLAINTEXT

B. Incorrect sasl.mechanism value

C. Wrong format in sasl.jaas.config string

D. Missing SSL configuration causes failure

Solution

1. Step 1: Check security.protocol compatibility

   SASL_PLAINTEXT means SASL without encryption. Many brokers disable this for security reasons.

2. Step 2: Consider broker support

   If the broker is configured to reject SASL_PLAINTEXT connections, authentication will fail even if credentials and mechanism are correct.

3. Final Answer:

   Broker does not support SASL_PLAINTEXT -> Option A

4. Quick Check:

   Broker must allow SASL_PLAINTEXT [OK]

Hint: Check if broker allows SASL_PLAINTEXT connections [OK]

Common Mistakes:

• Assuming SSL is required for SASL_PLAINTEXT
• Misformatting sasl.jaas.config string
• Using wrong sasl.mechanism value

5. You want to configure a Kafka client to use SASL_SSL with SCRAM-SHA-512 mechanism. Which of the following is the correct minimal configuration snippet in Java?

hard

A. props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "SCRAM-SHA-512");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";");

B. props.put("security.protocol", "SSL");
props.put("sasl.mechanism", "SCRAM-SHA-512");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";");

C. props.put("security.protocol", "SASL_PLAINTEXT");
props.put("sasl.mechanism", "SCRAM-SHA-256");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";");

D. props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "PLAIN");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"user\" password=\"pass\";");

Solution

1. Step 1: Match security.protocol with SASL_SSL

   To use encrypted SASL, security.protocol must be SASL_SSL.

2. Step 2: Confirm sasl.mechanism and JAAS config

   For SCRAM-SHA-512, sasl.mechanism must be SCRAM-SHA-512 and JAAS config must use ScramLoginModule with username and password.

3. Step 3: Eliminate incorrect options

   props.put("security.protocol", "SSL"); props.put("sasl.mechanism", "SCRAM-SHA-512"); props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";"); uses SSL without SASL, C uses wrong mechanism and protocol, D uses PLAIN mechanism instead of SCRAM-SHA-512.

4. Final Answer:

   Correct SASL_SSL with SCRAM-SHA-512 and proper JAAS config -> Option A

5. Quick Check:

   SASL_SSL + SCRAM-SHA-512 + JAAS = props.put("security.protocol", "SASL_SSL"); props.put("sasl.mechanism", "SCRAM-SHA-512"); props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";"); [OK]

Hint: Match protocol, mechanism, and JAAS module exactly [OK]

Common Mistakes:

• Using SSL instead of SASL_SSL
• Mismatching mechanism and JAAS module
• Using wrong SASL mechanism version