Long to wide format conversion in Pandas - Time & Space Complexity

Time Complexity: Long to wide format conversion

O(n)

Understanding Time Complexity

When we change data from long to wide format, we rearrange rows into columns. Understanding how long this takes helps us work efficiently with bigger data.

We want to know how the time needed grows as the data gets larger.

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

df = pd.DataFrame({
    'id': [1, 1, 2, 2],
    'variable': ['A', 'B', 'A', 'B'],
    'value': [10, 20, 30, 40]
})

wide_df = df.pivot(index='id', columns='variable', values='value')

This code changes a table from long format to wide format using pivot, turning variable values into columns.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Scanning all rows to group by the index and columns.
• How many times: Once for each row in the data.

How Execution Grows With Input

As the number of rows grows, the time to rearrange grows roughly in the same way.

Input Size (n)	Approx. Operations
10	About 10 operations
100	About 100 operations
1000	About 1000 operations

Pattern observation: The work grows directly with the number of rows.

Final Time Complexity

Time Complexity: O(n)

This means the time to convert grows in a straight line as the data gets bigger.

Common Mistake

[X] Wrong: "Pivoting data takes the same time no matter how big the data is."

[OK] Correct: The process must look at each row to place it correctly, so more rows mean more work and more time.

Interview Connect

Knowing how data reshaping scales helps you handle real datasets smoothly and shows you understand how tools work behind the scenes.

Self-Check

"What if we used pivot_table with aggregation instead of pivot? How would the time complexity change?"