How to Encrypt Data in Kafka: Simple Steps and Examples

To encrypt data in Kafka, use SSL/TLS for encrypting data in transit between clients and brokers, and enable encryption at rest by configuring disk encryption on Kafka brokers. SSL/TLS requires setting up certificates and configuring Kafka properties like ssl.keystore.location and ssl.truststore.location.

Syntax

Kafka encryption involves configuring SSL properties in the broker and client configuration files. Key properties include:

• ssl.keystore.location: Path to the broker's keystore file containing its private key.
• ssl.keystore.password: Password for the keystore.
• ssl.key.password: Password for the private key in the keystore.
• ssl.truststore.location: Path to the truststore file containing trusted certificates.
• ssl.truststore.password: Password for the truststore.
• security.protocol: Set to SSL to enable SSL encryption.

These settings go into server.properties for brokers and producer.properties or consumer.properties for clients.

properties

security.protocol=SSL
ssl.keystore.location=/path/to/kafka.server.keystore.jks
ssl.keystore.password=yourKeystorePassword
ssl.key.password=yourKeyPassword
ssl.truststore.location=/path/to/kafka.server.truststore.jks
ssl.truststore.password=yourTruststorePassword

Example

This example shows how to configure a Kafka broker and a producer client to use SSL encryption for data in transit.

bash

# Broker server.properties snippet
security.protocol=SSL
ssl.keystore.location=/etc/kafka/secrets/kafka.server.keystore.jks
ssl.keystore.password=changeit
ssl.key.password=changeit
ssl.truststore.location=/etc/kafka/secrets/kafka.server.truststore.jks
ssl.truststore.password=changeit

# Producer client.properties snippet
security.protocol=SSL
ssl.truststore.location=/etc/kafka/secrets/kafka.client.truststore.jks
ssl.truststore.password=changeit

# Command to produce a message using SSL
kafka-console-producer --broker-list localhost:9093 --topic test-topic --producer.config client.properties
>Hello encrypted Kafka!

Output

Message 'Hello encrypted Kafka!' sent securely over SSL to topic 'test-topic'.

Common Pitfalls

Common mistakes when encrypting Kafka data include:

• Using incorrect file paths for keystore or truststore files.
• Not matching passwords between keystore and configuration.
• Forgetting to set security.protocol=SSL on both broker and client.
• Not restarting Kafka brokers after changing SSL settings.
• Ignoring encryption at rest, which requires disk-level encryption outside Kafka.

Always verify certificates and test connectivity after configuration.

properties

## Wrong: Missing security.protocol
ssl.keystore.location=/path/to/keystore.jks

## Right: Include security.protocol
security.protocol=SSL
ssl.keystore.location=/path/to/keystore.jks

Quick Reference

Setting	Description
security.protocol	Protocol used; set to SSL for encryption
ssl.keystore.location	Path to broker's keystore file
ssl.keystore.password	Password for keystore
ssl.key.password	Password for private key in keystore
ssl.truststore.location	Path to truststore file with trusted certs
ssl.truststore.password	Password for truststore

Key Takeaways

Enable SSL/TLS in Kafka by setting security.protocol=SSL and configuring keystore and truststore files.

Both Kafka brokers and clients must have matching SSL configurations for encryption to work.

Encryption at rest is handled outside Kafka, typically via disk encryption on broker machines.

Always verify file paths and passwords to avoid connection failures.

Restart Kafka brokers after SSL configuration changes to apply encryption settings.