Client authentication configuration in Kafka - Mini Project: Build & Apply

Client authentication configuration

📖 Scenario: You are setting up a Kafka client to securely connect to a Kafka broker. To do this, you need to configure client authentication using SASL/PLAIN mechanism.

🎯 Goal: Configure a Kafka client with SASL/PLAIN authentication by creating the necessary configuration properties step-by-step.

📋 What You'll Learn

Create a dictionary called client_config with basic Kafka client settings

Add a configuration variable sasl_mechanism set to PLAIN

Add SASL authentication settings to client_config using the sasl_mechanism and given credentials

Print the final client_config dictionary

💡 Why This Matters

🌍 Real World

Kafka clients often need secure authentication to connect to brokers in production environments. Configuring SASL/PLAIN is a common way to authenticate users.

💼 Career

Understanding how to configure Kafka client authentication is important for roles like backend developers, DevOps engineers, and data engineers working with Kafka clusters.

1

Create initial Kafka client configuration

Create a dictionary called client_config with these exact entries: 'bootstrap.servers': 'localhost:9092' and 'security.protocol': 'SASL_PLAINTEXT'.

Kafka

# Create the client_config dictionary with bootstrap.servers and security.protocol
# Your code here

Hint

Use a Python dictionary with the exact keys and values as shown.

2

Add SASL mechanism configuration

Create a variable called sasl_mechanism and set it to the string 'PLAIN'.

Kafka

client_config = {
    'bootstrap.servers': 'localhost:9092',
    'security.protocol': 'SASL_PLAINTEXT'
}
# Create sasl_mechanism variable and set it to 'PLAIN'
# Your code here

Hint

Assign the string 'PLAIN' to the variable sasl_mechanism.

3

Add SASL authentication details to client configuration

Add these exact entries to the client_config dictionary: 'sasl.mechanism' set to sasl_mechanism, 'sasl.username' set to 'user1', and 'sasl.password' set to 'pass123'.

Kafka

client_config = {
    'bootstrap.servers': 'localhost:9092',
    'security.protocol': 'SASL_PLAINTEXT'
}
sasl_mechanism = 'PLAIN'
# Add sasl.mechanism, sasl.username, and sasl.password to client_config
# Your code here

Hint

Use dictionary key assignment to add the new entries.

4

Print the final client configuration

Write a print statement to display the client_config dictionary.

Kafka

client_config = {
    'bootstrap.servers': 'localhost:9092',
    'security.protocol': 'SASL_PLAINTEXT'
}
sasl_mechanism = 'PLAIN'
client_config['sasl.mechanism'] = sasl_mechanism
client_config['sasl.username'] = 'user1'
client_config['sasl.password'] = 'pass123'
# Print the client_config dictionary
# Your code here

Hint

Use print(client_config) to show the dictionary.

Practice

1. What is the main purpose of client authentication in Kafka?

easy

A. To verify the identity of clients connecting to the Kafka broker

B. To increase the speed of message delivery

C. To compress messages before sending

D. To store messages permanently on the client side

Solution

1. Step 1: Understand client authentication

   Client authentication is about confirming who is connecting to Kafka to ensure security.

2. Step 2: Identify the purpose in Kafka context

   Kafka uses client authentication to verify client identity before allowing access.

3. Final Answer:

   To verify the identity of clients connecting to the Kafka broker -> Option A

4. Quick Check:

   Client authentication = verify identity [OK]

Hint: Authentication means verifying who you are [OK]

Common Mistakes:

• Confusing authentication with message compression
• Thinking authentication speeds up delivery
• Mixing authentication with data storage

2. Which Kafka client configuration property is used to enable SSL authentication?

easy

A. enable.ssl.auth=true

B. ssl.enabled=true

C. security.protocol=SSL

D. auth.method=ssl

Solution

1. Step 1: Recall Kafka SSL config property

   The correct property to enable SSL is 'security.protocol' set to 'SSL'.

2. Step 2: Check other options for correctness

   Other options are not valid Kafka config keys for SSL authentication.

3. Final Answer:

   security.protocol=SSL -> Option C

4. Quick Check:

   SSL enabled by security.protocol=SSL [OK]

Hint: Use security.protocol=SSL to enable SSL [OK]

Common Mistakes:

• Using non-existent properties like ssl.enabled
• Confusing property names with auth.method
• Adding extra words like enable.ssl.auth

3. Given this Kafka client config snippet:

security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="user" password="pass";

What type of client authentication is configured?

medium

A. SSL without SASL

B. SASL with PLAIN mechanism over SSL

C. SASL with SCRAM mechanism

D. No authentication

Solution

1. Step 1: Analyze security.protocol and sasl.mechanism

   security.protocol=SASL_SSL means SASL authentication over SSL. sasl.mechanism=PLAIN means using PLAIN SASL mechanism.

2. Step 2: Confirm authentication type

   The config uses SASL PLAIN mechanism secured by SSL transport.

3. Final Answer:

   SASL with PLAIN mechanism over SSL -> Option B

4. Quick Check:

   SASL_SSL + PLAIN = SASL PLAIN over SSL [OK]

Hint: SASL_SSL means SASL over SSL, check sasl.mechanism [OK]

Common Mistakes:

• Ignoring sasl.mechanism and assuming only SSL
• Confusing PLAIN with SCRAM mechanism
• Thinking no authentication is set

4. You have this Kafka client config:

security.protocol=SSL
ssl.keystore.location=/path/to/keystore
ssl.keystore.password=wrongpass

The client fails to authenticate. What is the most likely cause?

medium

A. Incorrect keystore password

B. Missing sasl.mechanism property

C. security.protocol should be PLAINTEXT

D. ssl.keystore.location path is not needed

Solution

1. Step 1: Check SSL keystore config

   The keystore password is set to 'wrongpass', which likely does not match the actual password.

2. Step 2: Evaluate other options

   Missing sasl.mechanism is irrelevant for SSL-only config. PLAINTEXT disables security. Keystore location is required.

3. Final Answer:

   Incorrect keystore password -> Option A

4. Quick Check:

   Wrong password causes SSL auth failure [OK]

Hint: Check passwords carefully for SSL keystore [OK]

Common Mistakes:

• Assuming sasl.mechanism is needed for SSL-only
• Changing security.protocol to PLAINTEXT by mistake
• Ignoring keystore location requirement

5. You want to configure a Kafka client to authenticate using SASL SCRAM-SHA-512 over SSL. Which configuration snippet is correct?

hard

A. security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="user" password="pass";

B. security.protocol=SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass";

C. security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass";

D. security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass";

Solution

1. Step 1: Identify correct security.protocol for SASL SCRAM over SSL

   It must be 'SASL_SSL' to use SASL authentication over SSL.

2. Step 2: Match sasl.mechanism and jaas config

   For SCRAM-SHA-512, sasl.mechanism must be 'SCRAM-SHA-512' and jaas config must use ScramLoginModule.

3. Step 3: Verify options

   security.protocol=SASL_SSL sasl.mechanism=SCRAM-SHA-512 sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass"; matches all requirements. security.protocol=SSL sasl.mechanism=SCRAM-SHA-512 sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass"; misses SASL in protocol. security.protocol=SASL_PLAINTEXT sasl.mechanism=SCRAM-SHA-512 sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass"; uses PLAINTEXT transport. security.protocol=SASL_SSL sasl.mechanism=PLAIN sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="user" password="pass"; uses PLAIN mechanism, not SCRAM.

4. Final Answer:

   security.protocol=SASL_SSL sasl.mechanism=SCRAM-SHA-512 sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass"; -> Option D

5. Quick Check:

   SASL_SSL + SCRAM-SHA-512 + ScramLoginModule = security.protocol=SASL_SSL sasl.mechanism=SCRAM-SHA-512 sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass"; [OK]

Hint: Use SASL_SSL with matching SCRAM mechanism and login module [OK]

Common Mistakes:

• Using SSL without SASL for SCRAM
• Mixing PLAIN mechanism with SCRAM config
• Setting security.protocol to PLAINTEXT