Adding and removing columns in Data Analysis Python - Time & Space Complexity

Time Complexity: Adding and removing columns

O(n)

Understanding Time Complexity

When we add or remove columns in a data table, the time it takes depends on how many rows we have.

We want to know how the work grows as the table gets bigger.

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

n = 10  # Example value for n
data = pd.DataFrame({
    'A': range(n),
    'B': range(n, 2*n)
})

# Adding a new column
 data['C'] = data['A'] + data['B']

# Removing a column
 data.drop('B', axis=1, inplace=True)

This code creates a table with n rows, adds a new column by combining two columns, then removes one column.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Adding a column involves going through each row once to compute values.
• How many times: Each row is visited once for the addition, and once for removing the column.

How Execution Grows With Input

As the number of rows grows, the time to add or remove a column grows roughly the same way.

Input Size (n)	Approx. Operations
10	About 10 operations to add and remove
100	About 100 operations to add and remove
1000	About 1000 operations to add and remove

Pattern observation: The work grows in a straight line with the number of rows.

Final Time Complexity

Time Complexity: O(n)

This means the time to add or remove a column grows directly with the number of rows.

Common Mistake

[X] Wrong: "Adding or removing a column is instant and does not depend on data size."

[OK] Correct: Even though it looks simple, the computer must update every row, so it takes more time as the table grows.

Interview Connect

Understanding how data operations scale helps you explain your code choices clearly and shows you know how data size affects performance.

Self-Check

"What if we add multiple columns at once instead of one? How would the time complexity change?"

Practice

1. What is the correct way to add a new column named 'age' with all values set to 30 in a pandas DataFrame df?

easy

A. df.append('age', 30)

B. df.add_column('age', 30)

C. df['age'] = 30

D. df.insert_column('age', 30)

Solution

1. Step 1: Understand how to add columns in pandas

   In pandas, you add a new column by assigning a value or list to df['column_name'].

2. Step 2: Apply the correct syntax to add a column named 'age'

   Using df['age'] = 30 adds a new column 'age' with all values set to 30.

3. Final Answer:

   df['age'] = 30 -> Option C

4. Quick Check:

   Adding column = df['new_col'] = value [OK]

Hint: Use df['col_name'] = value to add columns fast [OK]

Common Mistakes:

• Using non-existent methods like add_column or insert_column
• Trying to use append which is for rows, not columns
• Forgetting to use square brackets for column name

2. Which of the following is the correct syntax to remove a column named 'salary' from a pandas DataFrame df?

easy

A. df.drop('salary', axis=1)

B. df.remove('salary')

C. df.delete_column('salary')

D. df.drop_column('salary')

Solution

1. Step 1: Recall pandas method to remove columns

   The pandas method to remove columns is drop() with axis=1 to specify columns.

2. Step 2: Use correct syntax to drop 'salary' column

   Using df.drop('salary', axis=1) removes the 'salary' column from the DataFrame.

3. Final Answer:

   df.drop('salary', axis=1) -> Option A

4. Quick Check:

   Remove column = df.drop('col', axis=1) [OK]

Hint: Use df.drop('col', axis=1) to remove columns [OK]

Common Mistakes:

• Using methods like remove or delete_column which don't exist
• Forgetting axis=1 to specify column removal
• Using drop without parentheses or wrong parameters

3. Given the DataFrame df below:

   name  score
0  Amy     90
1  Bob     80
2  Cara    85

What will be the output of this code?

df['passed'] = df['score'] >= 85
print(df)

medium

A. name score passed
0 Amy 90 False
1 Bob 80 True
2 Cara 85 False

B. name score passed
0 Amy 90 True
1 Bob 80 False
2 Cara 85 True

C. name score
0 Amy 90
1 Bob 80
2 Cara 85

D. name score passed
0 Amy 90 90
1 Bob 80 80
2 Cara 85 85

Solution

1. Step 1: Understand the condition applied to 'score'

   The expression df['score'] >= 85 creates a boolean series: True where score is 85 or more, False otherwise.

2. Step 2: Assign boolean series to new column 'passed'

   Adding df['passed'] = ... adds this boolean series as a new column.

3. Final Answer:

   DataFrame with 'passed' column showing True for scores >= 85 -> Option B

4. Quick Check:

   Boolean condition adds True/False column [OK]

Hint: Compare column with condition to add True/False column [OK]

Common Mistakes:

• Expecting numeric values instead of booleans in new column
• Not using >= operator correctly
• Printing df before adding the new column

4. What is wrong with this code snippet to remove the column 'height' from DataFrame df?

df.drop('height')

medium

A. Missing axis=1 to specify column removal

B. The column name should be in a list like ['height']

C. The method drop does not exist in pandas

D. You must assign the result back to df like df = df.drop('height')

Solution

1. Step 1: Check the drop method parameters

   By default, drop() removes rows. To remove columns, you must specify axis=1.

2. Step 2: Identify the missing axis parameter

   The code misses axis=1, so it tries to remove a row named 'height', causing an error if no such row exists.

3. Final Answer:

   Missing axis=1 to specify column removal -> Option A

4. Quick Check:

   drop columns = drop('col', axis=1) [OK]

Hint: Always add axis=1 to drop columns, not rows [OK]

Common Mistakes:

• Forgetting axis=1 causes wrong row removal or error
• Thinking drop modifies df inplace without assignment
• Using wrong data type for column name

5. You have a DataFrame df with columns ['A', 'B', 'C']. You want to create a new column 'D' which is the sum of 'A' and 'B', then remove column 'C'. Which code correctly does this in one step?

hard

A. df['D'] = df['A'] + df['B']
df.remove('C')

B. df['D'] = df['A'] + df['B']
df.drop('C', axis=1, inplace=True)

C. df['D'] = df['A'] + df['B']
df = df.drop('C', axis=1)

D. df = df.assign(D=df['A'] + df['B']).drop('C', axis=1)

Solution

1. Step 1: Create new column 'D' as sum of 'A' and 'B'

   Using assign() allows adding a new column in a chainable way without modifying df inplace.

2. Step 2: Remove column 'C' using drop() chained after assign()

   Chaining drop('C', axis=1) removes column 'C' and returns the modified DataFrame.

3. Step 3: Assign the result back to df

   Assigning the chained operations back to df updates the DataFrame correctly in one step.

4. Final Answer:

   df = df.assign(D=df['A'] + df['B']).drop('C', axis=1) -> Option D

5. Quick Check:

   Use assign + drop chained and assign back [OK]

Hint: Chain assign and drop, then assign back to df [OK]

Common Mistakes:

• Using drop without axis=1 or inplace causing no change
• Using non-existent remove method
• Not assigning chained result back to df