Security best practices in Kafka - Commands & Configuration

Introduction

Kafka is a system that moves messages between programs. Security best practices help keep these messages safe from hackers and mistakes.

When you want to make sure only trusted users can send or read messages.

When you need to protect sensitive data moving through Kafka from being seen by others.

When you want to stop outsiders from connecting to your Kafka servers.

When you want to keep track of who accessed or changed Kafka data.

When you want to make sure messages are not changed or lost during transfer.

Config File - server.properties

server.properties

listeners=SASL_SSL://:9093
advertised.listeners=SASL_SSL://localhost:9093
security.inter.broker.protocol=SASL_SSL
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password=changeit
ssl.key.password=changeit
ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password=changeit
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=false
super.users=User:admin

listeners: Defines how Kafka accepts connections securely using SASL_SSL.
advertised.listeners: How Kafka tells clients to connect.
security.inter.broker.protocol: Secures communication between Kafka servers.
sasl.enabled.mechanisms: Enables PLAIN authentication.
ssl.keystore.location/password: Location and password for server keys.
ssl.truststore.location/password: Location and password for trusted certificates.
authorizer.class.name: Enables access control lists (ACLs) for permissions.
allow.everyone.if.no.acl.found: Denies access if no ACL is set.
super.users: Defines admin users with full access.

Commands

This command adds permission for user 'alice' to read messages from 'my-topic'. It controls who can access the topic.

Terminal

kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:alice --operation Read --topic my-topic

Expected Output

Added ACLs.

→

--add - Adds a new ACL rule

→

--allow-principal - Specifies the user allowed

→

--operation - Defines the allowed action (Read)

→

--topic - Specifies the topic for the ACL

This command sets the password for user 'alice' to authenticate securely with Kafka.

Terminal

kafka-configs --zookeeper localhost:2181 --entity-type users --entity-name alice --alter --add-config 'sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="alice" password="alice-secret";'

Expected Output

Updated config for user alice.

→

--alter - Changes existing user config

→

--add-config - Adds SASL authentication details

This command checks the details of 'my-topic' to verify it exists and is accessible.

Terminal

kafka-topics --bootstrap-server localhost:9093 --describe --topic my-topic

Expected Output

Topic: my-topic	PartitionCount: 1	ReplicationFactor: 1	Configs: 

	Topic: my-topic	Partition: 0	Leader: 1	Replicas: 1	Isr: 1

→

--describe - Shows topic details

→

--bootstrap-server - Connects to Kafka securely

Key Concept

If you remember nothing else from this pattern, remember: securing Kafka means controlling who connects, encrypting data, and setting clear permissions.

Common Mistakes

Not enabling SSL or SASL, leaving Kafka open without encryption or authentication.

This allows anyone to connect and read or write messages, risking data leaks or tampering.

Always enable SSL for encryption and SASL for authentication in Kafka configuration.

Setting allow.everyone.if.no.acl.found=true, which lets anyone access topics without ACLs.

This defeats the purpose of access control and can expose sensitive data.

Set allow.everyone.if.no.acl.found=false to deny access if no ACL is defined.

Using weak or default passwords for SASL users.

Weak passwords can be guessed, allowing attackers to impersonate users.

Use strong, unique passwords and rotate them regularly.

Summary

Configure Kafka to use SSL and SASL for encrypted and authenticated connections.

Use ACLs to control which users can read or write to specific topics.

Verify topic access and user permissions with Kafka CLI commands.

Practice

1. Which of the following is the primary purpose of using SSL/TLS in Kafka security?

easy

A. To manage access permissions with ACLs

B. To authenticate users with usernames and passwords

C. To encrypt data transmitted between Kafka clients and brokers

D. To compress messages for faster transmission

Solution

1. Step 1: Understand SSL/TLS role in Kafka

   SSL/TLS is used to encrypt data sent over the network to protect it from eavesdropping.

2. Step 2: Differentiate from other security features

   Authentication uses SASL, and ACLs manage permissions, not encryption.

3. Final Answer:

   To encrypt data transmitted between Kafka clients and brokers -> Option C

4. Quick Check:

   SSL/TLS = Encryption [OK]

Hint: SSL/TLS encrypts data in transit, not authentication or permissions [OK]

Common Mistakes:

• Confusing SSL/TLS with authentication methods
• Thinking ACLs handle encryption
• Assuming SSL/TLS compresses data

2. Which of the following is the correct way to enable SASL authentication in Kafka's server.properties file?

easy

A. acl.enable=true

B. security.protocol=PLAINTEXT

C. ssl.enabled=true

D. sasl.enabled.mechanisms=PLAIN

Solution

1. Step 1: Identify SASL configuration property

   The property sasl.enabled.mechanisms specifies which SASL mechanisms Kafka accepts.

2. Step 2: Check other options for correctness

   security.protocol=PLAINTEXT disables security, ssl.enabled is not a valid property, and acl.enable is unrelated to SASL.

3. Final Answer:

   sasl.enabled.mechanisms=PLAIN -> Option D

4. Quick Check:

   SASL config uses sasl.enabled.mechanisms [OK]

Hint: SASL uses sasl.enabled.mechanisms property in server.properties [OK]

Common Mistakes:

• Using PLAINTEXT disables security
• Confusing SSL properties with SASL
• Assuming acl.enable controls authentication

3. Given the following ACL command, what permission does it grant?

kafka-acls --add --allow-principal User:Alice --operation Read --topic sales-data

medium

A. Alice can read messages from the sales-data topic

B. Alice can create new topics

C. Alice can delete the sales-data topic

D. Alice can write messages to the sales-data topic

Solution

1. Step 1: Analyze the ACL command parameters

   The command adds an ACL allowing User:Alice the Read operation on the topic sales-data.

2. Step 2: Understand the Read operation

   Read permission allows consuming or reading messages from the topic, not writing or deleting.

3. Final Answer:

   Alice can read messages from the sales-data topic -> Option A

4. Quick Check:

   ACL Read = Permission to consume messages [OK]

Hint: Read operation in ACL means permission to consume messages [OK]

Common Mistakes:

• Confusing Read with Write permission
• Assuming Read allows topic deletion
• Mixing topic creation with ACL permissions

4. Identify the error in the following Kafka security configuration snippet:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256

medium

A. security.protocol should be SASL_SSL to enable encryption

B. The property should be sasl.mechanism instead of sasl.mechanisms

C. SCRAM-SHA-256 is not a valid SASL mechanism

D. No error, configuration is correct

Solution

1. Step 1: Check security.protocol value

   SASL_PLAINTEXT enables SASL but does not encrypt traffic, which is insecure for sensitive data.

2. Step 2: Recommend secure protocol

   Using SASL_SSL enables SASL authentication with SSL encryption, which is a best practice.

3. Final Answer:

   security.protocol should be SASL_SSL to enable encryption -> Option A

4. Quick Check:

   Use SASL_SSL for encrypted SASL connections [OK]

Hint: SASL_PLAINTEXT lacks encryption; use SASL_SSL for security [OK]

Common Mistakes:

• Thinking SASL_PLAINTEXT encrypts data
• Confusing sasl.mechanism property name
• Believing SCRAM-SHA-256 is invalid

5. You want to secure a Kafka cluster so that only authenticated users can produce or consume messages, and all traffic is encrypted. Which combination of settings achieves this best?

hard

A. Use PLAINTEXT protocol with ACLs to restrict access, no encryption needed

B. Enable SSL encryption with security.protocol=SASL_SSL, configure SASL authentication, and set ACLs for user permissions

C. Enable SASL_PLAINTEXT for authentication and rely on network firewall for encryption

D. Use SSL only without SASL or ACLs to simplify configuration

Solution

1. Step 1: Ensure encryption and authentication

   Using security.protocol=SASL_SSL enables both encryption (SSL) and authentication (SASL).

2. Step 2: Control access with ACLs

   ACLs restrict which authenticated users can produce or consume messages, enforcing permissions.

3. Final Answer:

   Enable SSL encryption with security.protocol=SASL_SSL, configure SASL authentication, and set ACLs for user permissions -> Option B

4. Quick Check:

   Encryption + Authentication + ACLs = Secure Kafka [OK]

Hint: Combine SASL_SSL, SASL auth, and ACLs for full security [OK]

Common Mistakes:

• Using PLAINTEXT disables encryption
• Relying on firewall instead of encryption
• Skipping ACLs allows unauthorized access