Security best practices in Kafka - Mini Project: Build & Apply

Kafka Security Best Practices

📖 Scenario: You are setting up a Kafka cluster for a company that handles sensitive customer data. To protect this data, you need to apply security best practices in Kafka configuration.

🎯 Goal: Configure Kafka with basic security settings including authentication and authorization to secure the data flow.

📋 What You'll Learn

Create a Kafka server properties dictionary with default settings

Add a security protocol configuration variable

Apply SASL authentication and ACL authorization settings

Print the final Kafka security configuration

💡 Why This Matters

🌍 Real World

Kafka is widely used for real-time data streaming in industries like finance, healthcare, and e-commerce where data security is critical.

💼 Career

Understanding Kafka security best practices is essential for roles like DevOps engineers, data engineers, and system administrators to protect sensitive data and comply with regulations.

1

Create Kafka server properties dictionary

Create a dictionary called kafka_config with these exact entries: 'broker.id': '1', 'log.dirs': '/tmp/kafka-logs', and 'zookeeper.connect': 'localhost:2181'.

Kafka

# Create the kafka_config dictionary with broker.id, log.dirs, and zookeeper.connect
# Your code here

Need a hint?

Use curly braces to create a dictionary and include the exact keys and values as strings.

2

Add security protocol configuration

Add a new entry to the kafka_config dictionary with key 'security.protocol' and value 'SASL_PLAINTEXT'.

Kafka

kafka_config = {
    'broker.id': '1',
    'log.dirs': '/tmp/kafka-logs',
    'zookeeper.connect': 'localhost:2181'
}
# Add 'security.protocol' with value 'SASL_PLAINTEXT' to kafka_config
# Your code here

Need a hint?

Add the key and value inside the existing dictionary with a comma separating entries.

3

Apply SASL authentication and ACL authorization settings

Add these exact entries to kafka_config: 'sasl.mechanism.inter.broker.protocol': 'PLAIN' and 'authorizer.class.name': 'kafka.security.auth.SimpleAclAuthorizer'.

Kafka

kafka_config = {
    'broker.id': '1',
    'log.dirs': '/tmp/kafka-logs',
    'zookeeper.connect': 'localhost:2181',
    'security.protocol': 'SASL_PLAINTEXT'
}
# Add SASL mechanism and authorizer class name to kafka_config
# Your code here

Need a hint?

Add both entries separated by commas inside the dictionary.

4

Print the final Kafka security configuration

Write a print statement to display the kafka_config dictionary.

Kafka

kafka_config = {
    'broker.id': '1',
    'log.dirs': '/tmp/kafka-logs',
    'zookeeper.connect': 'localhost:2181',
    'security.protocol': 'SASL_PLAINTEXT',
    'sasl.mechanism.inter.broker.protocol': 'PLAIN',
    'authorizer.class.name': 'kafka.security.auth.SimpleAclAuthorizer'
}
# Print the kafka_config dictionary
# Your code here

Need a hint?

Use print(kafka_config) to show the dictionary.