Why securing Kafka protects data - Why It Works

Introduction

Kafka is a system that moves data between programs. If it is not secured, bad people can see or change the data. Securing Kafka keeps data safe and private.

When you want to stop strangers from reading your messages in Kafka.

When you need to make sure only trusted programs can send or get data.

When you want to protect sensitive information like passwords or personal details.

When you want to prevent attackers from changing or deleting your data.

When you want to follow rules that require data protection.

Config File - server.properties

server.properties

listeners=SASL_SSL://:9093
advertised.listeners=SASL_SSL://localhost:9093
security.inter.broker.protocol=SASL_SSL
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
listener.name.sasl_ssl.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";
ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password=keystore-password
ssl.key.password=key-password
ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password=truststore-password

This file sets up Kafka to use secure connections with SSL and user login with SASL PLAIN.

listeners and advertised.listeners tell Kafka to use secure ports.

security.inter.broker.protocol makes sure Kafka brokers talk securely.

sasl.enabled.mechanisms and related settings enable username/password login.

ssl.keystore.location and ssl.truststore.location point to files that hold security certificates.

Commands

This command starts the Kafka server using the secure settings in the configuration file.

Terminal

kafka-server-start.sh /usr/local/kafka/config/server.properties

Expected Output

[2024-06-01 12:00:00,000] INFO Kafka version : 3.5.0 (org.apache.kafka.common.utils.AppInfoParser)

[2024-06-01 12:00:00,100] INFO Kafka started (kafka.server.KafkaServer)

This command sends messages to Kafka securely using username and password over SSL.

Terminal

kafka-console-producer.sh --broker-list localhost:9093 --topic secure-topic --producer-property security.protocol=SASL_SSL --producer-property sasl.mechanism=PLAIN --producer-property sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

Expected Output

No output (command runs silently)

→

--producer-property security.protocol=SASL_SSL - Use secure SSL connection

→

--producer-property sasl.mechanism=PLAIN - Use username/password login

→

--producer-property sasl.jaas.config - Provide login credentials

This command reads messages from Kafka securely using the same login and SSL settings.

Terminal

kafka-console-consumer.sh --bootstrap-server localhost:9093 --topic secure-topic --from-beginning --consumer-property security.protocol=SASL_SSL --consumer-property sasl.mechanism=PLAIN --consumer-property sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

Expected Output

Hello secure world

→

--consumer-property security.protocol=SASL_SSL - Use secure SSL connection

→

--consumer-property sasl.mechanism=PLAIN - Use username/password login

→

--consumer-property sasl.jaas.config - Provide login credentials

Key Concept

If you remember nothing else, remember: securing Kafka with SSL and login stops outsiders from reading or changing your data.

Common Mistakes

Not enabling SSL and SASL in Kafka configuration

Kafka will accept unencrypted connections, exposing data to anyone on the network.

Always set listeners and security protocols to use SSL and SASL in server.properties.

Using wrong or missing credentials when connecting

Kafka will reject the connection, and your app cannot send or receive data.

Provide correct username and password in the client command using sasl.jaas.config.

Not using the secure port (9093) when connecting

Clients will try to connect insecurely and fail or send data without encryption.

Always use the secure port defined in listeners for SSL connections.

Summary

Configure Kafka to use SSL and SASL for encrypted and authenticated communication.

Start Kafka with the secure configuration to protect data in transit.

Use client commands with proper security settings and credentials to send and receive data safely.