Kafka on Kubernetes (Strimzi) - Time & Space Complexity
When running Kafka on Kubernetes using Strimzi, it's important to understand how the system handles tasks as the number of Kafka topics or partitions grows.
We want to know how the time to manage Kafka resources changes as we add more topics or partitions.
Analyze the time complexity of the following Kafka resource reconciliation loop in Strimzi.
for (Topic topic : kafkaCluster.getTopics()) {
reconcileTopic(topic);
for (Partition partition : topic.getPartitions()) {
assignPartition(partition);
}
}
This code loops through all Kafka topics and then through all partitions of each topic to assign them properly in the Kubernetes cluster.
Look at the loops that repeat work:
- Primary operation: Looping through all topics and then all partitions inside each topic.
- How many times: The outer loop runs once per topic, and the inner loop runs once per partition per topic.
As the number of topics and partitions grows, the work grows too.
| Input Size (topics * partitions) | Approx. Operations |
|---|---|
| 10 topics * 5 partitions | 50 operations |
| 100 topics * 5 partitions | 500 operations |
| 1000 topics * 5 partitions | 5000 operations |
Pattern observation: The total work grows roughly in proportion to the number of topics times the number of partitions.
Time Complexity: O(n * m)
This means the time to complete the task grows in direct proportion to the number of topics (n) multiplied by the number of partitions per topic (m).
[X] Wrong: "The time to assign partitions grows only with the number of topics, not partitions."
[OK] Correct: Each partition needs separate handling, so the total work depends on both topics and partitions, not just topics alone.
Understanding how Kafka scales on Kubernetes helps you explain how systems handle growing workloads, a key skill in DevOps roles.
"What if we changed the code to assign partitions in parallel instead of sequentially? How would the time complexity change?"