apply() with lambda functions in Pandas - Time & Space Complexity

Time Complexity: apply() with lambda functions

O(n)

Understanding Time Complexity

We want to understand how the time needed to run apply() with lambda functions changes as the data grows.

How does the work increase when we have more rows in our data?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

df = pd.DataFrame({'A': range(1000)})
df['B'] = df['A'].apply(lambda x: x * 2)

This code creates a column 'B' by doubling each value in column 'A' using apply() with a lambda function.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: The lambda function is called once for each row in column 'A'.
• How many times: Exactly as many times as there are rows in the DataFrame.

How Execution Grows With Input

As the number of rows increases, the total work grows in direct proportion.

Input Size (n)	Approx. Operations
10	10 calls to lambda
100	100 calls to lambda
1000	1000 calls to lambda

Pattern observation: Doubling the input size doubles the number of lambda calls and total work.

Final Time Complexity

Time Complexity: O(n)

This means the time needed grows linearly with the number of rows in the DataFrame.

Common Mistake

[X] Wrong: "Using apply() with a lambda is always slow because it loops over the data multiple times."

[OK] Correct: The lambda function is called once per row, so the loop happens only once. The time grows linearly, not more.

Interview Connect

Understanding how apply() scales helps you explain your data processing choices clearly and confidently.

Self-Check

"What if we replaced the lambda with a vectorized operation instead? How would the time complexity change?"

Practice

1. What does the apply() function with a lambda do in pandas?

easy

A. It sorts the DataFrame by column values.

B. It applies a small function to each element in a DataFrame column or row.

C. It merges two DataFrames based on a key.

D. It creates a new DataFrame from scratch.

Solution

1. Step 1: Understand the purpose of apply() with lambda

   The apply() method lets you run a function on each element or row/column of a DataFrame or Series.

2. Step 2: Recognize what lambda does

   A lambda is a small anonymous function used inside apply() to specify what to do with each element.

3. Final Answer:

   It applies a small function to each element in a DataFrame column or row. -> Option B

4. Quick Check:

   apply() + lambda = element-wise function [OK]

Hint: apply() runs a function on each item quickly [OK]

Common Mistakes:

• Thinking apply() sorts data
• Confusing apply() with merge or join
• Believing apply() creates new DataFrames

2. Which of the following is the correct syntax to use apply() with a lambda on a pandas Series named data to double each value?

easy

A. data.apply(lambda x: x * 2)

B. data.apply(x => x * 2)

C. apply(data, lambda x: x * 2)

D. data.apply(lambda x x * 2)

Solution

1. Step 1: Recall correct lambda syntax in Python

   Python lambda functions use the format lambda x: expression, so lambda x: x * 2 is correct.

2. Step 2: Check apply() usage on Series

   The method call is data.apply(lambda x: x * 2). Other options have syntax errors or wrong order.

3. Final Answer:

   data.apply(lambda x: x * 2) -> Option A

4. Quick Check:

   Correct lambda syntax = data.apply(lambda x: x * 2) [OK]

Hint: Use colon after lambda parameter: lambda x: ... [OK]

Common Mistakes:

• Using arrow syntax (=>) from other languages
• Missing colon after lambda parameter
• Calling apply() as a standalone function

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3]})
df['B'] = df['A'].apply(lambda x: x + 5)
print(df)

What is the output?

medium

A. Error: lambda function syntax

B. A B
0 1 5
1 2 6
2 3 7

C. A B
0 1 6
1 2 7
2 3 8

D. A B
0 6 1
1 7 2
2 8 3

Solution

1. Step 1: Understand the lambda function applied

   The lambda adds 5 to each value in column 'A'. So 1+5=6, 2+5=7, 3+5=8.

2. Step 2: Check DataFrame after assignment

   Column 'B' gets these new values, while 'A' remains unchanged.

3. Final Answer:

   A B 0 1 6 1 2 7 2 3 8 -> Option C

4. Quick Check:

   Original + 5 = New column [OK]

Hint: apply() changes each value as per lambda [OK]

Common Mistakes:

• Mixing up columns A and B
• Forgetting to add 5 inside lambda
• Expecting error from correct syntax

4. What is wrong with this code snippet?

df['C'] = df['A'].apply(lambda x x * 3)

medium

A. Missing colon after lambda parameter causes syntax error.

B. apply() cannot be used on DataFrame columns.

C. lambda functions cannot multiply values.

D. Column 'C' does not exist yet.

Solution

1. Step 1: Check lambda function syntax

   Python lambda requires a colon after parameters: lambda x: x * 3. The code misses the colon.

2. Step 2: Understand error cause

   Missing colon causes a syntax error, so code won't run.

3. Final Answer:

   Missing colon after lambda parameter causes syntax error. -> Option A

4. Quick Check:

   lambda syntax error = missing colon [OK]

Hint: Always put colon after lambda parameters [OK]

Common Mistakes:

• Forgetting colon after lambda parameter
• Thinking apply() can't be used on columns
• Assuming new columns must exist before assignment

5. You have a DataFrame df with a column 'score' containing numbers. You want to create a new column 'grade' that says 'Pass' if the score is 50 or more, and 'Fail' otherwise. Which code correctly uses apply() with a lambda to do this?

hard

A. df['grade'] = df.apply(lambda x: 'Pass' if x['score'] >= 50 else 'Fail')

B. df['grade'] = df['score'].apply(lambda x: if x >= 50 then 'Pass' else 'Fail')

C. df['grade'] = df['score'].apply(lambda x: x >= 50 ? 'Pass' : 'Fail')

D. df['grade'] = df['score'].apply(lambda x: 'Pass' if x >= 50 else 'Fail')

Solution

1. Step 1: Understand conditional expression in lambda

   Python uses value_if_true if condition else value_if_false syntax inside lambda.

2. Step 2: Check application target and syntax

   Applying to the Series df['score'] with lambda x: 'Pass' if x >= 50 else 'Fail' is correct, as x is each scalar score. Applying to the whole df (without axis=1) passes column Series to lambda, where x['score'] causes KeyError. Syntaxes like ? : ternary or if-then are invalid in Python.

3. Final Answer:

   df['grade'] = df['score'].apply(lambda x: 'Pass' if x >= 50 else 'Fail') -> Option D

4. Quick Check:

   Correct if-else in lambda = df['grade'] = df['score'].apply(lambda x: 'Pass' if x >= 50 else 'Fail') [OK]

Hint: Use Python's if-else inside lambda: val_if_true if cond else val_if_false [OK]

Common Mistakes:

• Using ?: ternary operator from other languages
• Applying lambda on whole DataFrame without axis
• Writing if-else without proper Python syntax