Testing stream topologies in Kafka - Time & Space Complexity
When testing stream topologies, we want to know how the time to run tests changes as the data size grows.
We ask: How does the test execution time grow when we add more input records?
Analyze the time complexity of the following Kafka Streams test code snippet.
TopologyTestDriver testDriver = new TopologyTestDriver(topology, props);
TestInputTopic<String, String> inputTopic = testDriver.createInputTopic("input", stringSerializer, stringSerializer);
TestOutputTopic<String, String> outputTopic = testDriver.createOutputTopic("output", stringDeserializer, stringDeserializer);
for (int i = 0; i < n; i++) {
inputTopic.pipeInput("key" + i, "value" + i);
}
List<KeyValue<String, String>> results = outputTopic.readKeyValuesToList();
This code feeds n records into the input topic and reads all output records after processing.
Look for repeated actions that take time.
- Primary operation: Loop feeding n input records one by one.
- How many times: Exactly n times, once per input record.
- Secondary operation: Reading all output records after processing, which depends on n.
As you add more input records, the test runs longer because each record is processed.
| Input Size (n) | Approx. Operations |
|---|---|
| 10 | About 10 input operations + reading outputs |
| 100 | About 100 input operations + reading outputs |
| 1000 | About 1000 input operations + reading outputs |
Pattern observation: The time grows roughly in direct proportion to the number of input records.
Time Complexity: O(n)
This means the test time increases linearly as you add more input records.
[X] Wrong: "The test time stays the same no matter how many records I add."
[OK] Correct: Each input record triggers processing, so more records mean more work and longer test time.
Understanding how test time grows helps you write efficient tests and reason about system behavior under load.
"What if we batch input records instead of sending them one by one? How would the time complexity change?"