apply() function for custom logic in Data Analysis Python - Step-by-Step Execution

Concept Flow - apply() function for custom logic

Start with DataFrame

↓

Define custom function

↓

Call apply() on DataFrame or Series

↓

apply() sends each row/column/value to custom function

↓

Custom function processes input and returns result

↓

apply() collects all results into new Series/DataFrame

↓

Use or store the transformed data

The apply() function takes a custom function and runs it on each row, column, or element, then collects the results.

Execution Sample

Data Analysis Python

import pandas as pd

def add_bonus(salary):
    return salary + 100

salaries = pd.Series([1000, 2000, 3000])
bonuses = salaries.apply(add_bonus)

This code adds 100 bonus to each salary using apply() with a custom function.

Execution Table

Step	Input to add_bonus	Function Output	bonuses Series after step
1	1000	1100	[1100, NaN, NaN]
2	2000	2100	[1100, 2100, NaN]
3	3000	3100	[1100, 2100, 3100]
4	End of apply()	All values processed	[1100, 2100, 3100]

💡 All elements processed, apply() returns new Series with updated values

Variable Tracker

Variable	Start	After 1	After 2	After 3	Final
salary	pd.Series([1000, 2000, 3000])	1000	2000	3000	3000
bonuses	Empty	[1100, NaN, NaN]	[1100, 2100, NaN]	[1100, 2100, 3100]	[1100, 2100, 3100]

Key Moments - 2 Insights

Why does apply() return a new Series instead of modifying the original?

apply() creates a new Series to keep original data unchanged, as shown in execution_table rows 1-4 where bonuses build up separately.

What exactly is passed to the custom function in apply()?

Each element of the Series is passed one by one to the custom function, as seen in execution_table column 'Input to add_bonus'.

Visual Quiz - 3 Questions

Look at the execution table, what is the output of add_bonus when input is 2000?

A 2000

B 2100

C 100

D 3000

💡 Hint

Check the row where Input to add_bonus is 2000 in the execution_table.

At which step does the bonuses Series first have all values filled?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Look at the 'bonuses Series after step' column in execution_table.

If the custom function returned salary * 2 instead of salary + 100, what would bonuses be after step 3?

A [2000, 4000, 6000]

B [1100, 2100, 3100]

C [100, 200, 300]

D [1000, 2000, 3000]

💡 Hint

Think about doubling each input value from the variable_tracker salaries.

Concept Snapshot

apply() runs a custom function on each element (or row/column) of a DataFrame or Series.
It returns a new Series or DataFrame with the results.
Original data stays unchanged.
Use apply() to add custom logic easily.
Example: series.apply(func) where func processes one value at a time.

Full Transcript

The apply() function in pandas lets you run your own function on each element of a Series or each row/column of a DataFrame. You start with your data, define a function that does something to one value, then call apply() with that function. Internally, apply() sends each value to your function, collects the results, and returns a new Series or DataFrame. This way, you can add custom logic without changing the original data. For example, adding a fixed bonus to salaries by defining add_bonus(salary) and applying it to each salary. The execution table shows each step: input value, function output, and how the new Series builds up. This helps beginners see how apply() works step-by-step.

Practice

1. What does the apply() function do in a pandas DataFrame?

easy

A. It merges two DataFrames together.

B. It sorts the DataFrame by a column.

C. It filters rows based on a condition.

D. It applies a custom function to each row or column.

Solution

1. Step 1: Understand the purpose of apply()

   The apply() function is used to run a custom function on each row or column of a DataFrame.

2. Step 2: Compare with other DataFrame methods

   Sorting, filtering, and merging are different operations and not done by apply().

3. Final Answer:

   It applies a custom function to each row or column. -> Option D

4. Quick Check:

   apply() = custom function application [OK]

Hint: Remember: apply() runs your function on rows or columns [OK]

Common Mistakes:

• Confusing apply() with sort or filter methods
• Thinking apply() merges DataFrames
• Assuming apply() only works on columns

2. Which of the following is the correct syntax to apply a function f row-wise on a DataFrame df?

easy

A. df.apply(f, axis=0)

B. df.apply(f)

C. df.apply(f, axis=1)

D. df.apply(f, axis=2)

Solution

1. Step 1: Recall axis parameter meaning

   In pandas, axis=1 means apply function row-wise, while axis=0 means column-wise.

2. Step 2: Identify correct axis for row-wise

   Since the question asks for row-wise application, axis=1 is correct.

3. Final Answer:

   df.apply(f, axis=1) -> Option C

4. Quick Check:

   Row-wise apply = axis=1 [OK]

Hint: Use axis=1 for rows, axis=0 for columns [OK]

Common Mistakes:

• Using axis=0 for row-wise apply
• Omitting axis parameter and expecting row-wise
• Using invalid axis values like 2

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})

What is the output of:

df['Sum'] = df.apply(lambda row: row['A'] + row['B'], axis=1)
print(df)

medium

A. A B Sum
0 1 3 3
1 2 4 4

B. A B Sum
0 1 3 4
1 2 4 6

C. A B Sum
0 1 3 1
1 2 4 2

D. Error: axis parameter missing

Solution

1. Step 1: Understand the lambda function

   The lambda adds values from columns 'A' and 'B' for each row.

2. Step 2: Apply row-wise addition

   Row 0: 1 + 3 = 4; Row 1: 2 + 4 = 6.

3. Final Answer:

   A B Sum 0 1 3 4 1 2 4 6 -> Option B

4. Quick Check:

   Sum column = A + B per row [OK]

Hint: Sum columns row-wise with apply and axis=1 [OK]

Common Mistakes:

• Forgetting axis=1 causes wrong operation
• Adding columns incorrectly (e.g., column-wise)
• Expecting apply to work without lambda

4. What is wrong with this code snippet?

df = pd.DataFrame({'X': [10, 20], 'Y': [30, 40]})
df['Result'] = df.apply(lambda x: x['X'] * 2 if x['Y'] > 35 else x['X'], axis=0)

medium

A. Using axis=0 applies function column-wise, causing a KeyError.

B. Lambda function syntax is incorrect.

C. DataFrame is missing required columns.

D. No error; code runs correctly.

Solution

1. Step 1: Check axis parameter usage

   axis=0 applies function to each column (Series), not rows.

2. Step 2: Understand lambda function context

   Lambda expects row with keys 'X' and 'Y', but column Series only has index labels, causing KeyError.

3. Final Answer:

   Using axis=0 applies function column-wise, causing a KeyError. -> Option A

4. Quick Check:

   axis=0 = columns, lambda expects rows [OK]

Hint: Use axis=1 for row-wise logic to avoid key errors [OK]

Common Mistakes:

• Confusing axis=0 and axis=1
• Expecting row keys when applying column-wise
• Ignoring error messages about missing keys

5. You have a DataFrame df with columns "score" and "passed" (boolean). You want to create a new column "grade" where:
- If passed is True and score >= 90, grade is 'A'
- If passed is True and score >= 75 but < 90, grade is 'B'
- Otherwise, grade is 'F'
Which apply() function correctly implements this logic?

hard

A. df['grade'] = df.apply(lambda r: 'A' if r['passed'] and r['score'] >= 90 else ('B' if r['passed'] and r['score'] >= 75 else 'F'), axis=1)

B. df['grade'] = df.apply(lambda r: 'A' if r['score'] >= 90 else 'B' if r['score'] >= 75 else 'F', axis=0)

C. df['grade'] = df.apply(lambda r: 'A' if r['passed'] or r['score'] >= 90 else 'F', axis=1)

D. df['grade'] = df.apply(lambda r: 'A' if r['passed'] and r['score'] > 90 else 'B', axis=1)

Solution

1. Step 1: Check axis parameter for row-wise logic

   axis=1 is needed to apply function on each row.

2. Step 2: Verify conditional logic matches requirements

   df['grade'] = df.apply(lambda r: 'A' if r['passed'] and r['score'] >= 90 else ('B' if r['passed'] and r['score'] >= 75 else 'F'), axis=1) correctly checks if passed is True and score thresholds for 'A' and 'B', else 'F'.

3. Step 3: Identify errors in other options

   df['grade'] = df.apply(lambda r: 'A' if r['score'] >= 90 else 'B' if r['score'] >= 75 else 'F', axis=0) uses axis=0 (wrong axis). df['grade'] = df.apply(lambda r: 'A' if r['passed'] or r['score'] >= 90 else 'F', axis=1) uses 'or' instead of 'and'. df['grade'] = df.apply(lambda r: 'A' if r['passed'] and r['score'] > 90 else 'B', axis=1) misses 'F' grade and uses wrong comparison for 90.

4. Final Answer:

   df['grade'] = df.apply(lambda r: 'A' if r['passed'] and r['score'] >= 90 else ('B' if r['passed'] and r['score'] >= 75 else 'F'), axis=1) -> Option A

5. Quick Check:

   Correct logic and axis=1 for row-wise apply [OK]

Hint: Use axis=1 and nested if-else for multi-condition grading [OK]

Common Mistakes:

• Using axis=0 instead of axis=1
• Using 'or' instead of 'and' in conditions
• Missing else condition for failing grade