Kafka on Kubernetes (Strimzi) - Mini Project: Build & Apply

Kafka on Kubernetes with Strimzi

📖 Scenario: You are working in a team that wants to run Kafka on Kubernetes. To make this easier, you will use Strimzi, a tool that helps manage Kafka clusters on Kubernetes.This project will guide you step-by-step to create a Kafka cluster using Strimzi on Kubernetes.

🎯 Goal: Set up a Kafka cluster on Kubernetes using Strimzi by creating the necessary Kubernetes resources step-by-step.

📋 What You'll Learn

Have access to a Kubernetes cluster

kubectl command-line tool installed and configured

Strimzi Operator installed in the Kubernetes cluster

💡 Why This Matters

🌍 Real World

Running Kafka on Kubernetes is common in modern cloud-native applications to handle real-time data streams efficiently.

💼 Career

DevOps engineers and platform engineers often deploy and manage Kafka clusters on Kubernetes using tools like Strimzi.

1

Create a Kafka namespace

Create a Kubernetes namespace called kafka-project using the command kubectl create namespace kafka-project.

Kafka

# Run the command to create the namespace
# Your code here

Need a hint?

Use kubectl create namespace kafka-project to create the namespace.

2

Deploy the Strimzi Kafka Operator

Apply the Strimzi Operator installation YAML in the kafka-project namespace using kubectl apply -f https://strimzi.io/install/latest?namespace=kafka-project -n kafka-project.

Kafka

kubectl create namespace kafka-project
# Apply the Strimzi Operator YAML below
# Your code here

Need a hint?

Use kubectl apply -f with the Strimzi URL and specify the namespace with -n kafka-project.

3

Create a Kafka cluster custom resource

Create a YAML file named kafka-cluster.yaml with the exact content below to define a Kafka cluster with 1 Kafka broker and 1 Zookeeper node in the kafka-project namespace:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
  namespace: kafka-project
spec:
  kafka:
    replicas: 1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
    storage:
      type: ephemeral
  zookeeper:
    replicas: 1
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

Then apply it with kubectl apply -f kafka-cluster.yaml.

Kafka

kubectl create namespace kafka-project
kubectl apply -f https://strimzi.io/install/latest?namespace=kafka-project -n kafka-project
# Create kafka-cluster.yaml with the given content and apply it
# Your code here

Need a hint?

Create the YAML file exactly as shown and apply it with kubectl apply -f kafka-cluster.yaml.

4

Check Kafka cluster status

Run kubectl get kafka my-cluster -n kafka-project -o yaml to see the status of the Kafka cluster and confirm it is ready.

Kafka

kubectl create namespace kafka-project
kubectl apply -f https://strimzi.io/install/latest?namespace=kafka-project -n kafka-project

kubectl apply -f kafka-cluster.yaml
# Run the command to check Kafka cluster status
# Your code here

Need a hint?

Use kubectl get kafka my-cluster -n kafka-project -o yaml to check the cluster status.