SASL authentication in Kafka - Commands & Configuration

Introduction

SASL authentication helps secure Kafka by requiring users to prove their identity before connecting. It stops unauthorized users from accessing your Kafka servers.

When you want to protect your Kafka cluster from unauthorized access.

When multiple teams share the same Kafka cluster and need separate access controls.

When you want to encrypt user credentials during login to Kafka.

When you need to comply with security policies requiring authentication.

When running Kafka in a production environment where security is important.

Config File - server.properties

server.properties

listeners=SASL_PLAINTEXT://:9092
advertised.listeners=SASL_PLAINTEXT://localhost:9092
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
security.inter.broker.protocol=SASL_PLAINTEXT
listener.name.sasl_plaintext.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";

This configuration file sets Kafka to use SASL with the PLAIN mechanism over plaintext connections.

listeners: Defines Kafka to listen for SASL_PLAINTEXT connections on port 9092.

advertised.listeners: Tells clients how to connect to Kafka.

sasl.enabled.mechanisms: Enables the PLAIN SASL mechanism.

sasl.mechanism.inter.broker.protocol: Sets the mechanism brokers use to talk to each other.

security.inter.broker.protocol: Uses SASL_PLAINTEXT for broker communication.

listener.name.sasl_plaintext.plain.sasl.jaas.config: Provides the username and password for authentication.

Commands

Starts the Kafka server with SASL authentication enabled using the provided configuration file.

Terminal

kafka-server-start.sh server.properties

Expected Output

[2024-06-01 12:00:00,000] INFO KafkaConfig values:

listeners = SASL_PLAINTEXT://:9092

advertised.listeners = SASL_PLAINTEXT://localhost:9092

sasl.enabled.mechanisms = [PLAIN]

[2024-06-01 12:00:01,000] INFO Kafka started (kafka.server.KafkaServer)

Starts a Kafka producer that connects using SASL PLAIN authentication to send messages to 'test-topic'.

Terminal

kafka-console-producer.sh --broker-list localhost:9092 --topic test-topic --producer-property sasl.mechanism=PLAIN --producer-property security.protocol=SASL_PLAINTEXT --producer-property sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

Expected Output

No output (command runs silently)

→

--producer-property sasl.mechanism=PLAIN - Specifies the SASL mechanism to use.

→

--producer-property security.protocol=SASL_PLAINTEXT - Specifies the security protocol for the connection.

→

--producer-property sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";' - Provides the username and password for authentication.

Starts a Kafka consumer that connects using SASL PLAIN authentication to read messages from 'test-topic' from the beginning.

Terminal

kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic --from-beginning --consumer-property sasl.mechanism=PLAIN --consumer-property security.protocol=SASL_PLAINTEXT --consumer-property sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

Expected Output

No output (command runs silently)

→

--consumer-property sasl.mechanism=PLAIN - Specifies the SASL mechanism to use.

→

--consumer-property security.protocol=SASL_PLAINTEXT - Specifies the security protocol for the connection.

→

--consumer-property sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";' - Provides the username and password for authentication.

Key Concept

If you remember nothing else from this pattern, remember: SASL authentication requires both Kafka server and clients to be configured with matching security settings and credentials.

Common Mistakes

Not setting the sasl.jaas.config property on the client side.

The client cannot authenticate without the username and password, so connection fails.

Always provide sasl.jaas.config with correct credentials in client commands or config files.

Using PLAINTEXT instead of SASL_PLAINTEXT in listeners or client security.protocol.

Kafka expects SASL authentication but the connection is not secured for SASL, causing errors.

Use SASL_PLAINTEXT or SASL_SSL consistently for listeners and client security.protocol.

Mismatching SASL mechanisms between server and client (e.g., server uses PLAIN but client uses SCRAM).

Authentication fails because server and client do not agree on the method.

Ensure both server and client use the same sasl.enabled.mechanisms and sasl.mechanism settings.

Summary

Configure Kafka server with SASL settings in server.properties to enable authentication.

Start Kafka server with the SASL-enabled configuration file.

Use client commands with sasl.jaas.config and security.protocol flags to authenticate when producing or consuming messages.

Practice

1. What is the main purpose of SASL authentication in Kafka?

easy

A. To compress messages for faster transmission

B. To verify user identity before allowing access

C. To balance load between Kafka brokers

D. To encrypt data stored on disk

Solution

1. Step 1: Understand SASL role in Kafka

   SASL is a protocol used to verify who is connecting to Kafka, ensuring only authorized users can access it.

2. Step 2: Differentiate from other Kafka features

   Compression, load balancing, and disk encryption are unrelated to SASL authentication.

3. Final Answer:

   To verify user identity before allowing access -> Option B

4. Quick Check:

   SASL = user verification [OK]

Hint: SASL is about user identity, not data or load [OK]

Common Mistakes:

• Confusing SASL with encryption
• Thinking SASL manages data compression
• Mixing SASL with load balancing

2. Which of the following is the correct way to specify SASL mechanism in Kafka client configuration?

easy

A. security.protocol = SASL_PLAINTEXT

B. sasl.config = PLAIN

C. sasl.mechanism = PLAIN

D. security.mechanism = SASL

Solution

1. Step 1: Identify correct property for SASL mechanism

   The Kafka client uses sasl.mechanism to specify the SASL mechanism like PLAIN, SCRAM, etc.

2. Step 2: Check other options for correctness

   security.protocol sets protocol type, not mechanism; sasl.config is invalid; security.mechanism is not a valid property.

3. Final Answer:

   sasl.mechanism = PLAIN -> Option C

4. Quick Check:

   sasl.mechanism sets SASL type [OK]

Hint: Use sasl.mechanism to set SASL type like PLAIN [OK]

Common Mistakes:

• Using security.protocol instead of sasl.mechanism
• Writing sasl.config instead of sasl.mechanism
• Confusing property names

3. Given this Kafka client configuration snippet:

props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "SCRAM-SHA-256");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user1\" password=\"pass1\";");

What will happen when the client tries to connect?

medium

A. Client connects using PLAINTEXT mechanism

B. Client connects but without encryption

C. Client fails due to missing SSL configuration

D. Client connects securely using SCRAM-SHA-256 with encrypted channel

Solution

1. Step 1: Analyze security.protocol setting

   Setting security.protocol to SASL_SSL means the client uses SASL authentication over SSL encrypted connection.

2. Step 2: Check SASL mechanism and JAAS config

   The mechanism is SCRAM-SHA-256 with proper JAAS config providing username and password, so authentication will succeed if credentials are correct.

3. Final Answer:

   Client connects securely using SCRAM-SHA-256 with encrypted channel -> Option D

4. Quick Check:

   SASL_SSL + SCRAM = secure auth [OK]

Hint: SASL_SSL means secure auth with encryption [OK]

Common Mistakes:

• Assuming SASL_SSL means no encryption
• Ignoring JAAS config for credentials
• Confusing SCRAM with PLAINTEXT

4. You have this Kafka client config:

props.put("security.protocol", "SASL_PLAINTEXT");
props.put("sasl.mechanism", "PLAIN");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin123\";");

The client fails to authenticate. What is the most likely cause?

medium

A. Broker does not support SASL_PLAINTEXT

B. Incorrect sasl.mechanism value

C. Wrong format in sasl.jaas.config string

D. Missing SSL configuration causes failure

Solution

1. Step 1: Check security.protocol compatibility

   SASL_PLAINTEXT means SASL without encryption. Many brokers disable this for security reasons.

2. Step 2: Consider broker support

   If the broker is configured to reject SASL_PLAINTEXT connections, authentication will fail even if credentials and mechanism are correct.

3. Final Answer:

   Broker does not support SASL_PLAINTEXT -> Option A

4. Quick Check:

   Broker must allow SASL_PLAINTEXT [OK]

Hint: Check if broker allows SASL_PLAINTEXT connections [OK]

Common Mistakes:

• Assuming SSL is required for SASL_PLAINTEXT
• Misformatting sasl.jaas.config string
• Using wrong sasl.mechanism value

5. You want to configure a Kafka client to use SASL_SSL with SCRAM-SHA-512 mechanism. Which of the following is the correct minimal configuration snippet in Java?

hard

A. props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "SCRAM-SHA-512");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";");

B. props.put("security.protocol", "SSL");
props.put("sasl.mechanism", "SCRAM-SHA-512");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";");

C. props.put("security.protocol", "SASL_PLAINTEXT");
props.put("sasl.mechanism", "SCRAM-SHA-256");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";");

D. props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "PLAIN");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"user\" password=\"pass\";");

Solution

1. Step 1: Match security.protocol with SASL_SSL

   To use encrypted SASL, security.protocol must be SASL_SSL.

2. Step 2: Confirm sasl.mechanism and JAAS config

   For SCRAM-SHA-512, sasl.mechanism must be SCRAM-SHA-512 and JAAS config must use ScramLoginModule with username and password.

3. Step 3: Eliminate incorrect options

   props.put("security.protocol", "SSL"); props.put("sasl.mechanism", "SCRAM-SHA-512"); props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";"); uses SSL without SASL, C uses wrong mechanism and protocol, D uses PLAIN mechanism instead of SCRAM-SHA-512.

4. Final Answer:

   Correct SASL_SSL with SCRAM-SHA-512 and proper JAAS config -> Option A

5. Quick Check:

   SASL_SSL + SCRAM-SHA-512 + JAAS = props.put("security.protocol", "SASL_SSL"); props.put("sasl.mechanism", "SCRAM-SHA-512"); props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";"); [OK]

Hint: Match protocol, mechanism, and JAAS module exactly [OK]

Common Mistakes:

• Using SSL instead of SASL_SSL
• Mismatching mechanism and JAAS module
• Using wrong SASL mechanism version