apply() on columns in Pandas - Step-by-Step Execution

Concept Flow - apply() on columns

Start with DataFrame

↓

Select a column

↓

Apply function to each value

↓

Collect results into new Series

↓

Replace or add column in DataFrame

↓

End

The apply() method takes a function and runs it on each value in a DataFrame column, then collects the results back into a Series or DataFrame column.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 3]})
df['B'] = df['A'].apply(lambda x: x * 2)
print(df)

This code doubles each value in column 'A' and stores the result in a new column 'B'.

Execution Table

Step	Input Value (x)	Function Applied	Result	DataFrame State
1	1	x * 2	2	{'A': [1, 2, 3], 'B': [2]}
2	2	x * 2	4	{'A': [1, 2, 3], 'B': [2, 4]}
3	3	x * 2	6	{'A': [1, 2, 3], 'B': [2, 4, 6]}
4	-	-	-	Completed applying function to all rows

💡 All values in column 'A' processed; new column 'B' created with doubled values.

Variable Tracker

Variable	Start	After 1	After 2	After 3	Final
df['A']	[1, 2, 3]	[1, 2, 3]	[1, 2, 3]	[1, 2, 3]	[1, 2, 3]
df['B']	[]	[2]	[2, 4]	[2, 4, 6]	[2, 4, 6]

Key Moments - 2 Insights

Why does apply() run the function on each value separately instead of the whole column at once?

apply() is designed to apply the function element-wise on each value in the column, as shown in execution_table rows 1-3 where each input value is processed individually.

What happens if the function returns a different type or size than expected?

The result will be collected into a Series matching the original column length. If the function returns unexpected types, the new column may have mixed types or cause errors. This is why the function should return one value per input.

Visual Quiz - 3 Questions

Look at the execution table, what is the value of df['B'] after step 2?

A [4]

B [2]

C [2, 4]

D []

💡 Hint

Check the DataFrame State column at step 2 in the execution_table.

At which step does the apply() function finish processing all values?

A Step 4

B Step 3

C Step 2

D Step 1

💡 Hint

Look for the row in execution_table where the DataFrame State says 'Completed applying function to all rows'.

If the lambda function was changed to 'x + 1', what would be the value of df['B'] after step 3?

A [3, 4, 5]

B [2, 3, 4]

C [1, 2, 3]

D [0, 1, 2]

💡 Hint

Add 1 to each original value in df['A'] which are [1, 2, 3].

Concept Snapshot

apply() on columns:
- Use df['col'].apply(func) to run func on each value.
- func receives one value at a time.
- Returns a Series with results.
- Can create or replace columns.
- Useful for element-wise transformations.

Full Transcript

This visual execution shows how pandas apply() works on DataFrame columns. Starting with a DataFrame with column 'A', we apply a function that doubles each value. The function runs on each value one by one, producing results collected into a new column 'B'. The execution table traces each step, showing input, function, result, and DataFrame state. Variable tracker shows how df['B'] grows after each step. Key moments clarify why apply() works element-wise and what happens if function returns unexpected results. The quiz tests understanding of the stepwise changes and effects of changing the function. The snapshot summarizes the key points for quick recall.

Practice

1. What does the apply() function do when used on a pandas DataFrame column?

easy

A. It applies a function to each value in the column.

B. It deletes the column from the DataFrame.

C. It sorts the column values in ascending order.

D. It merges the column with another DataFrame.

Solution

1. Step 1: Understand the purpose of apply() on columns

   The apply() method runs a function on each element of the column, transforming or analyzing the data.

2. Step 2: Compare with other options

   Deleting, sorting, or merging are not actions performed by apply() on a column.

3. Final Answer:

   It applies a function to each value in the column. -> Option A

4. Quick Check:

   apply() runs function on each column value [OK]

Hint: Remember: apply() runs a function on each column value [OK]

Common Mistakes:

• Thinking apply() deletes or sorts data
• Confusing apply() with merge or drop functions
• Assuming apply() works on entire DataFrame without axis

2. Which of the following is the correct syntax to apply a lambda function that doubles each value in the 'age' column of a DataFrame df?

easy

A. df['age'].apply = lambda x: x * 2

B. df.apply['age'](lambda x: x * 2)

C. df['age'].apply(lambda x: x * 2)

D. apply(df['age'], lambda x: x * 2)

Solution

1. Step 1: Identify correct method call syntax

   The correct way to use apply on a column is df['column'].apply(function). df['age'].apply(lambda x: x * 2) matches this.

2. Step 2: Check lambda function usage

   df['age'].apply(lambda x: x * 2) correctly passes a lambda function that doubles the value. Other options misuse syntax or assignment.

3. Final Answer:

   df['age'].apply(lambda x: x * 2) -> Option C

4. Quick Check:

   Use df['col'].apply(lambda x: ...) [OK]

Hint: Use df['col'].apply(lambda x: ...) for column functions [OK]

Common Mistakes:

• Using square brackets with apply like a list
• Assigning apply instead of calling it
• Calling apply as a standalone function

3. Given the DataFrame df below:

   score
0     10
1     20
2     30
3     40

What is the output of df['score'].apply(lambda x: x // 15)?

medium

A. 0 10
1 20
2 30
3 40
Name: score, dtype: int64

B. 0 0
1 1
2 2
3 3
Name: score, dtype: int64

C. 0 0
1 0
2 1
3 2
Name: score, dtype: int64

D. 0 0
1 1
2 2
3 2
Name: score, dtype: int64

Solution

1. Step 1: Apply the lambda function to each value

   Divide each score by 15 using floor division:
   10 // 15 = 0
   20 // 15 = 1
   30 // 15 = 2
   40 // 15 = 2

2. Step 2: Write the resulting Series

   The output Series is:
   0 0
   1 1
   2 2
   3 2

3. Final Answer:

   0 0 1 1 2 2 3 2 Name: score, dtype: int64 -> Option D

4. Quick Check:

   Floor division on each value = output [OK]

Hint: Apply lambda to each value and calculate carefully [OK]

Common Mistakes:

• Using normal division instead of floor division
• Misreading floor division results
• Confusing output format with DataFrame instead of Series

4. What is wrong with this code snippet?

df['new_col'] = df['value'].apply(x: x + 5)

How can it be fixed?

medium

A. Missing lambda keyword; fix: df['value'].apply(lambda x: x + 5)

B. apply() cannot add numbers; use map() instead

C. Should use square brackets around x; fix: apply([x]: x + 5)

D. Function must be defined outside apply; fix: def f(x): return x+5

Solution

1. Step 1: Identify syntax error in lambda function

   The code uses x: x + 5 without the lambda keyword, which is required to define an anonymous function.

2. Step 2: Correct the syntax

   Add lambda before the function: df['value'].apply(lambda x: x + 5).

3. Final Answer:

   Missing lambda keyword; fix: df['value'].apply(lambda x: x + 5) -> Option A

4. Quick Check:

   Lambda keyword needed for inline functions [OK]

Hint: Always use 'lambda' before inline functions in apply() [OK]

Common Mistakes:

• Omitting the lambda keyword
• Using incorrect brackets or syntax for lambda
• Thinking apply() can't use inline functions

5. You have a DataFrame df with a column 'temperature' in Celsius. You want to create a new column 'temp_f' with Fahrenheit values using apply(). Which code correctly does this?

hard

A. df['temp_f'] = df.apply(lambda c: (c * 9/5) + 32)

B. df['temp_f'] = df['temperature'].apply(lambda c: (c * 9/5) + 32)

C. df['temp_f'] = df['temperature'].apply(lambda c: c + 32 * 9/5)

D. df['temp_f'] = df['temperature'].apply(lambda c: c * (9/5 + 32))

Solution

1. Step 1: Recall the formula for Celsius to Fahrenheit

   The correct formula is Fahrenheit = (Celsius * 9/5) + 32.

2. Step 2: Check each option's lambda function

   df['temp_f'] = df['temperature'].apply(lambda c: (c * 9/5) + 32) correctly applies the formula to each value in 'temperature'. df['temp_f'] = df.apply(lambda c: (c * 9/5) + 32) applies to whole DataFrame, C and D have incorrect formulas.

3. Final Answer:

   df['temp_f'] = df['temperature'].apply(lambda c: (c * 9/5) + 32) -> Option B

4. Quick Check:

   Use correct formula inside apply() on the column [OK]

Hint: Use correct formula inside lambda for each column value [OK]

Common Mistakes:

• Applying function to whole DataFrame instead of column
• Mixing order of operations in formula
• Forgetting parentheses in formula