Dropping columns and rows in Pandas - Time & Space Complexity

Time Complexity: Dropping columns and rows

O(n x m)

Understanding Time Complexity

When we drop columns or rows in pandas, we want to know how the time it takes changes as the data grows.

We ask: How does the work increase when the table gets bigger?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'A': range(1000),
    'B': range(1000, 2000),
    'C': range(2000, 3000)
})

# Drop one column and one row
result = data.drop(columns=['B']).drop(index=[0])

This code creates a table with 3 columns and 1000 rows, then removes one column and one row.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Copying data except the dropped parts.
• How many times: Once for each drop operation, touching all remaining rows or columns.

How Execution Grows With Input

When the table grows, dropping a column or row means copying almost all data except the dropped parts.

Input Size (n rows)	Approx. Operations
10	About 20 (copying 2 columns x 10 rows)
100	About 200 (copying 2 columns x 100 rows)
1000	About 2000 (copying 2 columns x 1000 rows)

Pattern observation: The work grows roughly in direct proportion to the number of rows times columns kept.

Final Time Complexity

Time Complexity: O(n x m)

This means the time to drop columns or rows grows roughly with the size of the data kept, where n is rows and m is columns.

Common Mistake

[X] Wrong: "Dropping a single column or row is always very fast and constant time."

[OK] Correct: Even dropping one column or row requires copying the rest of the data, so time grows with the size of what remains.

Interview Connect

Understanding how data operations scale helps you explain your choices clearly and shows you know what happens behind the scenes.

Self-Check

What if we dropped multiple columns at once? How would the time complexity change?

Practice

1. What does the drop() function do in pandas?

easy

A. Removes specified rows or columns from a DataFrame

B. Adds new rows or columns to a DataFrame

C. Sorts the DataFrame by index or column values

D. Calculates summary statistics of the DataFrame

Solution

1. Step 1: Understand the purpose of drop()

   The drop() function is used to remove data, either rows or columns, from a DataFrame.

2. Step 2: Identify what drop() removes

   It specifically removes rows or columns based on labels, not adding or sorting data.

3. Final Answer:

   Removes specified rows or columns from a DataFrame -> Option A

4. Quick Check:

   drop() removes rows/columns [OK]

Hint: Remember: drop means remove rows or columns [OK]

Common Mistakes:

• Thinking drop adds data instead of removing
• Confusing drop with sort or describe functions
• Assuming drop changes data without specifying inplace

2. Which of the following is the correct syntax to drop the column named 'Age' from a DataFrame df?

easy

A. df.drop('Age', axis=0)

B. df.drop('Age', axis=1)

C. df.drop(column='Age')

D. df.drop('Age')

Solution

1. Step 1: Identify axis for columns

   In pandas, axis=1 means columns, so to drop a column, use axis=1.

2. Step 2: Check syntax correctness

   The syntax df.drop('Age', axis=1) correctly drops the column named 'Age'. df.drop(column='Age') is close but incorrect syntax; it should be df.drop(columns=['Age']) with a list.

3. Final Answer:

   df.drop('Age', axis=1) -> Option B

4. Quick Check:

   Drop column uses axis=1 [OK]

Hint: Use axis=1 to drop columns, axis=0 for rows [OK]

Common Mistakes:

• Using axis=0 to drop columns
• Forgetting to specify axis parameter
• Using drop with wrong parameter names

3. Given the DataFrame df below:

   Name  Age  City
0  John   25  NY
1  Anna   30  LA
2  Mike   22  SF

What is the output of df.drop(1, axis=0)?

medium

A. Name Age City
0 John 25 NY
1 Anna 30 LA

B. Name Age City
1 Anna 30 LA

C. Name Age City
0 John 25 NY
2 Mike 22 SF

D. Name Age City
0 John 25 NY
1 Anna 30 LA
2 Mike 22 SF

Solution

1. Step 1: Understand dropping rows by index

   Using drop(1, axis=0) removes the row with index 1, which is the row for Anna.

2. Step 2: Identify remaining rows

   Rows with index 0 (John) and 2 (Mike) remain, so the output excludes Anna's row.

3. Final Answer:

   Name Age City 0 John 25 NY 2 Mike 22 SF -> Option C

4. Quick Check:

   Drop row index 1 removes Anna [OK]

Hint: Drop row by index with axis=0 [OK]

Common Mistakes:

• Confusing axis=0 and axis=1
• Expecting row to be removed by label instead of index
• Not realizing drop returns a new DataFrame by default

4. What is wrong with this code snippet to drop the column 'Salary' from DataFrame df?

df.drop('Salary')

medium

A. drop() cannot remove columns

B. Should use inplace=True to drop the column

C. Column name should be in a list like ['Salary']

D. Missing axis=1 to specify dropping a column

Solution

1. Step 1: Check default axis in drop()

   By default, drop() assumes axis=0, which means it tries to drop a row, not a column.

2. Step 2: Specify axis=1 to drop a column

   To drop a column, you must specify axis=1. Without it, pandas looks for a row named 'Salary' and fails if none exists.

3. Final Answer:

   Missing axis=1 to specify dropping a column -> Option D

4. Quick Check:

   Drop columns requires axis=1 [OK]

Hint: Always specify axis=1 to drop columns [OK]

Common Mistakes:

• Forgetting axis=1 when dropping columns
• Assuming drop changes DataFrame without inplace=True
• Using drop with a single string but wrong axis

5. You have a DataFrame df with columns ['Name', 'Age', 'City', 'Salary']. You want to remove the rows with index 0 and 2, and also drop the column 'Salary' in one line without changing the original DataFrame. Which code does this correctly?

hard

A. df.drop([0, 2], axis=0).drop('Salary', axis=1)

B. df.drop([0, 2, 'Salary'], axis=0)

C. df.drop(['Salary'], axis=1).drop([0, 2], axis=0, inplace=True)

D. df.drop([0, 2], axis=1).drop('Salary', axis=0)

Solution

1. Step 1: Drop rows with index 0 and 2

   Use df.drop([0, 2], axis=0) to remove rows 0 and 2 without changing original DataFrame.

2. Step 2: Drop column 'Salary' from the result

   Chain .drop('Salary', axis=1) to remove the column from the intermediate DataFrame.

3. Step 3: Verify no inplace=True used

   Since we want to keep original DataFrame unchanged, do not use inplace=True. The chained drops return a new DataFrame.

4. Final Answer:

   df.drop([0, 2], axis=0).drop('Salary', axis=1) -> Option A

5. Quick Check:

   Chain drops with correct axis, no inplace [OK]

Hint: Chain drop with axis=0 for rows, axis=1 for columns [OK]

Common Mistakes:

• Mixing axis values for rows and columns
• Using inplace=True which changes original DataFrame
• Trying to drop rows and columns in one drop call incorrectly