Correlation and cross-correlation in Signal Processing - Time & Space Complexity

Time Complexity: Correlation and cross-correlation

O(n²)

Understanding Time Complexity

We want to understand how the time needed to compute correlation and cross-correlation changes as the input signals get longer.

How does the work grow when we compare bigger signals?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

def cross_correlation(x, y):
    n = len(x)
    result = []
    for lag in range(n):
        sum_val = 0
        for i in range(n - lag):
            sum_val += x[i] * y[i + lag]
        result.append(sum_val)
    return result

This code calculates the cross-correlation of two signals by sliding one over the other and summing the products at each lag.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Nested loops multiplying and summing signal values.
• How many times: Outer loop runs n times; inner loop runs decreasing from n to 1 times.

How Execution Grows With Input

As the signal length n grows, the total number of multiplications and additions grows roughly with the square of n.

Input Size (n)	Approx. Operations
10	~55
100	~5,050
1000	~500,500

Pattern observation: Doubling the input size roughly quadruples the work needed.

Final Time Complexity

Time Complexity: O(n²)

This means the time to compute correlation grows quickly as the signals get longer, roughly like the square of their length.

Common Mistake

[X] Wrong: "Cross-correlation runs in linear time because it just slides one signal over the other once."

[OK] Correct: Each slide requires multiplying and summing many pairs of values, so the total work adds up to more than just one pass.

Interview Connect

Understanding how correlation scales helps you explain performance when working with signals or time series data in real projects.

Self-Check

"What if we used a fast Fourier transform (FFT) based method for correlation? How would the time complexity change?"

Practice

1. What does the cross-correlation between two signals primarily measure?

easy

A. The similarity between the signals at different time shifts

B. The sum of the signals' amplitudes

C. The difference in frequency between the signals

D. The product of the signals' maximum values

Solution

1. Step 1: Understand cross-correlation concept

   Cross-correlation compares two signals by sliding one over the other to find similarity at different shifts.

2. Step 2: Identify what cross-correlation measures

   It measures how well the signals match at each time delay, not just their sums or frequency differences.

3. Final Answer:

   The similarity between the signals at different time shifts -> Option A

4. Quick Check:

   Cross-correlation = similarity at shifts [OK]

Hint: Cross-correlation slides signals to find best match [OK]

Common Mistakes:

• Confusing correlation with frequency difference
• Thinking it sums amplitudes only
• Assuming it multiplies max values

2. Which Python function from NumPy computes the cross-correlation of two 1D signals correctly?

easy

A. numpy.convolve(signal1, signal2)

B. numpy.correlate(signal1, signal2, mode='full')

C. numpy.cross(signal1, signal2)

D. numpy.corrcoef(signal1, signal2)

Solution

1. Step 1: Identify NumPy functions for correlation

   numpy.correlate computes cross-correlation; numpy.cross is for vector cross product; numpy.convolve does convolution; numpy.corrcoef computes correlation coefficient matrix.

2. Step 2: Confirm correct function and parameters

   numpy.correlate with mode='full' returns cross-correlation at all shifts.

3. Final Answer:

   numpy.correlate(signal1, signal2, mode='full') -> Option B

4. Quick Check:

   Use numpy.correlate for cross-correlation [OK]

Hint: Use numpy.correlate with mode='full' for cross-correlation [OK]

Common Mistakes:

• Using numpy.cross which is vector product
• Confusing convolution with correlation
• Using numpy.corrcoef which gives correlation matrix

3. Given two signals x = [1, 2, 3] and y = [0, 1, 0.5], what is the cross-correlation output using numpy.correlate(x, y, mode='full')?

medium

A. [0.5, 2.5, 4, 3, 0]

B. [0, 1, 2, 3, 4]

C. [0.5, 2, 3.5, 3, 0]

D. [1, 2, 3, 0, 0]

Solution

1. Step 1: Calculate cross-correlation values manually

   Cross-correlation at lag -2: x[0]*y[2]=1*0.5=0.5; lag -1: x[0]*y[1]+x[1]*y[2]=1*1+2*0.5=1+1=2; lag 0: x[0]*y[0]+x[1]*y[1]+x[2]*y[2]=1*0+2*1+3*0.5=0+2+1.5=3.5; lag 1: x[1]*y[0]+x[2]*y[1]=2*0+3*1=3; lag 2: x[2]*y[0]=3*0=0

2. Step 2: Arrange results in full mode order

   Full mode output is [lag -2, -1, 0, 1, 2] = [0.5, 2, 3.5, 3, 0]

3. Final Answer:

   [0.5, 2, 3.5, 3, 0] -> Option C

4. Quick Check:

   Cross-correlation values match calculated array [OK]

Hint: Calculate lag by lag multiplying overlapping elements [OK]

Common Mistakes:

• Mixing convolution with correlation
• Incorrect lag order in output
• Forgetting zero padding in full mode

4. You run numpy.correlate(signal1, signal2) but get a shorter output than expected. What is the likely cause?

medium

A. The function requires 2D arrays, not 1D

B. signal1 and signal2 must be the same length

C. You need to normalize signals before correlation

D. The default mode is 'valid', which returns only overlapping parts

Solution

1. Step 1: Check default mode of numpy.correlate

   By default, numpy.correlate uses mode='valid', which returns only positions where signals fully overlap, resulting in shorter output.

2. Step 2: Understand effect of mode on output length

   To get full cross-correlation including partial overlaps, mode='full' must be specified.

3. Final Answer:

   The default mode is 'valid', which returns only overlapping parts -> Option D

4. Quick Check:

   Default mode='valid' shortens output [OK]

Hint: Specify mode='full' for complete cross-correlation output [OK]

Common Mistakes:

• Assuming signals must be same length
• Thinking normalization affects output length
• Believing function needs 2D arrays

5. You have two noisy signals recorded with a delay. How can cross-correlation help you find the delay between them?

hard

A. By finding the lag where cross-correlation is maximum, indicating best alignment

B. By subtracting the signals and measuring the difference

C. By computing the Fourier transform and comparing magnitudes

D. By averaging the signals and checking the mean value

Solution

1. Step 1: Understand cross-correlation use for delay detection

   Cross-correlation slides one signal over the other and measures similarity at each lag.

2. Step 2: Identify lag with maximum cross-correlation

   The lag where cross-correlation peaks shows the delay that best aligns the two signals despite noise.

3. Final Answer:

   By finding the lag where cross-correlation is maximum, indicating best alignment -> Option A

4. Quick Check:

   Max cross-correlation lag = signal delay [OK]

Hint: Max cross-correlation lag reveals signal delay [OK]

Common Mistakes:

• Using subtraction which ignores alignment
• Confusing frequency domain methods here
• Averaging signals loses timing info