append equivalent with concat in Pandas - Time & Space Complexity

Time Complexity: append equivalent with concat

O(n)

Understanding Time Complexity

We want to understand how the time needed to combine data grows when using pandas concat instead of append.

How does the work change as the data size gets bigger?

Scenario Under Consideration

Analyze the time complexity of this pandas code that combines two dataframes.

import pandas as pd

n = 10  # example value for n
df1 = pd.DataFrame({'A': range(n)})
df2 = pd.DataFrame({'A': range(n, 2*n)})

result = pd.concat([df1, df2], ignore_index=True)

This code joins two dataframes with n rows each into one dataframe with 2n rows.

Identify Repeating Operations

Look at what repeats as data grows.

• Primary operation: copying rows from both dataframes into a new combined dataframe.
• How many times: once for each row in both dataframes, so about 2n times.

How Execution Grows With Input

As the number of rows n grows, the work to combine grows roughly in direct proportion.

Input Size (n)	Approx. Operations
10	About 20 operations (copying rows)
100	About 200 operations
1000	About 2000 operations

Pattern observation: doubling the input doubles the work needed.

Final Time Complexity

Time Complexity: O(n)

This means the time to combine data grows linearly with the number of rows.

Common Mistake

[X] Wrong: "Using concat is faster because it just links data without copying."

[OK] Correct: concat actually copies data to create a new dataframe, so time grows with data size.

Interview Connect

Understanding how data combining scales helps you write efficient code and explain your choices clearly.

Self-Check

What if we combined many small dataframes in a loop using concat each time? How would the time complexity change?

Practice

1. What is the main reason to use pandas.concat() instead of DataFrame.append()?

easy

A. concat() is faster for single DataFrame addition only

B. Because append() is deprecated and concat() can join multiple DataFrames at once

C. append() works only for columns, not rows

D. concat() automatically sorts columns alphabetically

Solution

1. Step 1: Understand the deprecation of append()

   The append() method is deprecated in pandas and will be removed in future versions.

2. Step 2: Recognize concat() advantages

   concat() can join multiple DataFrames at once, not just one, making it more flexible.

3. Final Answer:

   Because append() is deprecated and concat() can join multiple DataFrames at once -> Option B

4. Quick Check:

   append() deprecated, concat() multi-join [OK]

Hint: Remember: concat replaces append and handles many DataFrames [OK]

Common Mistakes:

• Thinking append is still recommended
• Believing concat only works for columns
• Assuming append is faster than concat

2. Which of the following is the correct syntax to replace df1.append(df2, ignore_index=True) using concat?

easy

A. pd.concat([df1, df2], ignore_index=True)

B. pd.concat([df1, df2], axis=0)

C. pd.concat(df1, df2, ignore_index=True)

D. pd.concat([df1, df2], axis=1, ignore_index=True)

Solution

1. Step 1: Identify the correct function call

   pd.concat() requires a list of DataFrames as the first argument, so [df1, df2] is correct.

2. Step 2: Use ignore_index=True to reset row numbers

   To mimic append(ignore_index=True), pass ignore_index=True to concat(). The default axis is 0 (rows), so no need to specify.

3. Final Answer:

   pd.concat([df1, df2], ignore_index=True) -> Option A

4. Quick Check:

   List of DataFrames + ignore_index=True = correct concat syntax [OK]

Hint: Use list of DataFrames and ignore_index=True with concat [OK]

Common Mistakes:

• Passing DataFrames without list brackets
• Using axis=1 to add rows (wrong axis)
• Omitting ignore_index when needed

3. Given the code:

import pandas as pd

df1 = pd.DataFrame({'A': [1, 2]})
df2 = pd.DataFrame({'A': [3, 4]})
result = pd.concat([df1, df2], ignore_index=True)
print(result)

What will be the output?

medium

A. A
0 1
1 2
0 3
1 4

B. A
0 3
1 4

C. A
0 1
1 2
2 3
3 4

D. Error because ignore_index is not valid

Solution

1. Step 1: Understand concat with ignore_index=True

   concat joins rows from both DataFrames and resets the index to 0,1,2,...

2. Step 2: Combine rows and reset index

   Rows from df1 (1,2) and df2 (3,4) stack vertically with new index 0 to 3.

3. Final Answer:

   A 0 1 1 2 2 3 3 4 -> Option C

4. Quick Check:

   concat stacks rows and resets index with ignore_index=True [OK]

Hint: concat stacks rows; ignore_index resets numbering [OK]

Common Mistakes:

• Expecting original indexes to remain
• Confusing axis parameter
• Thinking concat drops rows

4. What is wrong with this code snippet?

import pandas as pd

df1 = pd.DataFrame({'A': [1, 2]})
df2 = pd.DataFrame({'A': [3, 4]})
result = pd.concat(df1, df2, ignore_index=True)
print(result)

medium

A. The DataFrames should be passed as a list inside concat

B. ignore_index is not a valid argument for concat

C. concat cannot join DataFrames with same columns

D. The code will run without errors and print combined DataFrame

Solution

1. Step 1: Check concat argument format

   pd.concat() expects a list or iterable of DataFrames as the first argument, but here two DataFrames are passed as separate arguments.

2. Step 2: Identify correct usage

   Correct usage is pd.concat([df1, df2], ignore_index=True) with square brackets around DataFrames.

3. Final Answer:

   The DataFrames should be passed as a list inside concat -> Option A

4. Quick Check:

   concat needs list of DataFrames [OK]

Hint: Always pass DataFrames as a list to concat [OK]

Common Mistakes:

• Passing DataFrames as separate arguments
• Forgetting brackets around DataFrames
• Misusing ignore_index parameter

5. You have three DataFrames:

df1 = pd.DataFrame({'X': [1, 2]})
df2 = pd.DataFrame({'X': [3], 'Y': [5]})
df3 = pd.DataFrame({'X': [4], 'Y': [6]})

You want to combine them vertically using concat so that missing columns fill with NaN and the index resets. Which code achieves this?

hard

A. pd.concat(df1, df2, df3, ignore_index=True)

B. pd.concat([df1, df2, df3], axis=1, ignore_index=True)

C. pd.concat([df1, df2, df3], join='inner', ignore_index=True)

D. pd.concat([df1, df2, df3], ignore_index=True)

Solution

1. Step 1: Combine DataFrames vertically with concat

   Use pd.concat() with a list of DataFrames to stack rows.

2. Step 2: Handle missing columns and reset index

   Default join='outer' fills missing columns with NaN. Use ignore_index=True to reset row numbers.

3. Final Answer:

   pd.concat([df1, df2, df3], ignore_index=True) -> Option D

4. Quick Check:

   concat list, default outer join, ignore_index=True [OK]

Hint: Use concat list with ignore_index=True for clean vertical join [OK]

Common Mistakes:

• Using axis=1 to join rows (actually joins columns)
• Using join='inner' which drops columns
• Passing DataFrames without list brackets