Incrementality testing in Digital Marketing - Time & Space Complexity

Time Complexity: Incrementality testing

O(n)

Understanding Time Complexity

Incrementality testing measures the true impact of a marketing action by comparing groups. We want to understand how the time to analyze results grows as we increase the size of the test data.

How does the effort to calculate incrementality change when we have more users or conversions?

Scenario Under Consideration

Analyze the time complexity of the following incrementality test analysis.

// Assume we have two groups: test and control
// Each group has a list of user conversions
function calculateIncrementality(testGroup, controlGroup) {
  let testSum = 0;
  for (let i = 0; i < testGroup.length; i++) {
    testSum += testGroup[i].conversions;
  }
  let controlSum = 0;
  for (let j = 0; j < controlGroup.length; j++) {
    controlSum += controlGroup[j].conversions;
  }
  return (testSum - controlSum) / controlSum;
}
    

This code sums conversions in both groups and calculates the relative lift to find incrementality.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Two separate loops that sum conversions in test and control groups.
• How many times: Each loop runs once for every user in its group, so total operations grow with the number of users.

How Execution Grows With Input

As the number of users in test and control groups increases, the time to sum conversions grows proportionally.

Input Size (n)	Approx. Operations
10 users per group	About 20 additions
100 users per group	About 200 additions
1000 users per group	About 2000 additions

Pattern observation: The total work grows directly with the number of users; doubling users roughly doubles the work.

Final Time Complexity

Time Complexity: O(n)

This means the time to calculate incrementality grows in a straight line with the number of users analyzed.

Common Mistake

[X] Wrong: "Since we have two groups, the time complexity is O(n²) because of two loops."

[OK] Correct: The loops run one after another, not nested inside each other, so their times add up, not multiply.

Interview Connect

Understanding how analysis time grows with data size helps you design scalable marketing tests and explain your approach clearly in discussions.

Self-Check

"What if we combined test and control data into one list and filtered inside the loop? How would that affect the time complexity?"