apply() on columns in Pandas

Introduction

We use apply() on columns to quickly change or analyze data in each column of a table.

You want to add 10 to every number in a column.

You need to check if values in a column are above a certain limit.

You want to convert all text in a column to uppercase.

You want to calculate the length of each string in a column.

You want to replace missing values in a column with a fixed number.

Syntax

Pandas

DataFrame['column_name'].apply(function)

The function can be a built-in function, a lambda (small anonymous function), or a custom function.

This applies the function to each value in the specified column.

Examples

Adds 1 to every value in the 'age' column.

Pandas

df['age'].apply(lambda x: x + 1)

Converts all names in the 'name' column to uppercase letters.

Pandas

df['name'].apply(str.upper)

Marks each score as 'Pass' or 'Fail' based on the value.

Pandas

df['score'].apply(lambda x: 'Pass' if x >= 50 else 'Fail')

Sample Program

This code creates a small table with names, ages, and scores. Then it uses apply() on columns to change ages, names, and scores as explained.

Pandas

import pandas as pd

data = {'name': ['Alice', 'Bob', 'Charlie'], 'age': [25, 30, 35], 'score': [88, 45, 67]}
df = pd.DataFrame(data)

# Add 5 years to each age
new_ages = df['age'].apply(lambda x: x + 5)

# Convert names to uppercase
upper_names = df['name'].apply(str.upper)

# Mark scores as Pass or Fail
pass_fail = df['score'].apply(lambda x: 'Pass' if x >= 50 else 'Fail')

print('New ages:')
print(new_ages)
print('\nUppercase names:')
print(upper_names)
print('\nPass or Fail:')
print(pass_fail)

Important Notes

If you want to change the column in the original table, assign the result back like df['age'] = df['age'].apply(...).

Using apply() is slower than vectorized operations but easier for custom logic.

Summary

apply() runs a function on each value in a column.

It helps change or analyze data quickly without loops.

You can use built-in, lambda, or your own functions with apply().

Practice

1. What does the apply() function do when used on a pandas DataFrame column?

easy

A. It applies a function to each value in the column.

B. It deletes the column from the DataFrame.

C. It sorts the column values in ascending order.

D. It merges the column with another DataFrame.

Solution

1. Step 1: Understand the purpose of apply() on columns

   The apply() method runs a function on each element of the column, transforming or analyzing the data.

2. Step 2: Compare with other options

   Deleting, sorting, or merging are not actions performed by apply() on a column.

3. Final Answer:

   It applies a function to each value in the column. -> Option A

4. Quick Check:

   apply() runs function on each column value [OK]

Hint: Remember: apply() runs a function on each column value [OK]

Common Mistakes:

• Thinking apply() deletes or sorts data
• Confusing apply() with merge or drop functions
• Assuming apply() works on entire DataFrame without axis

2. Which of the following is the correct syntax to apply a lambda function that doubles each value in the 'age' column of a DataFrame df?

easy

A. df['age'].apply = lambda x: x * 2

B. df.apply['age'](lambda x: x * 2)

C. df['age'].apply(lambda x: x * 2)

D. apply(df['age'], lambda x: x * 2)

Solution

1. Step 1: Identify correct method call syntax

   The correct way to use apply on a column is df['column'].apply(function). df['age'].apply(lambda x: x * 2) matches this.

2. Step 2: Check lambda function usage

   df['age'].apply(lambda x: x * 2) correctly passes a lambda function that doubles the value. Other options misuse syntax or assignment.

3. Final Answer:

   df['age'].apply(lambda x: x * 2) -> Option C

4. Quick Check:

   Use df['col'].apply(lambda x: ...) [OK]

Hint: Use df['col'].apply(lambda x: ...) for column functions [OK]

Common Mistakes:

• Using square brackets with apply like a list
• Assigning apply instead of calling it
• Calling apply as a standalone function

3. Given the DataFrame df below:

   score
0     10
1     20
2     30
3     40

What is the output of df['score'].apply(lambda x: x // 15)?

medium

A. 0 10
1 20
2 30
3 40
Name: score, dtype: int64

B. 0 0
1 1
2 2
3 3
Name: score, dtype: int64

C. 0 0
1 0
2 1
3 2
Name: score, dtype: int64

D. 0 0
1 1
2 2
3 2
Name: score, dtype: int64

Solution

1. Step 1: Apply the lambda function to each value

   Divide each score by 15 using floor division:
   10 // 15 = 0
   20 // 15 = 1
   30 // 15 = 2
   40 // 15 = 2

2. Step 2: Write the resulting Series

   The output Series is:
   0 0
   1 1
   2 2
   3 2

3. Final Answer:

   0 0 1 1 2 2 3 2 Name: score, dtype: int64 -> Option D

4. Quick Check:

   Floor division on each value = output [OK]

Hint: Apply lambda to each value and calculate carefully [OK]

Common Mistakes:

• Using normal division instead of floor division
• Misreading floor division results
• Confusing output format with DataFrame instead of Series

4. What is wrong with this code snippet?

df['new_col'] = df['value'].apply(x: x + 5)

How can it be fixed?

medium

A. Missing lambda keyword; fix: df['value'].apply(lambda x: x + 5)

B. apply() cannot add numbers; use map() instead

C. Should use square brackets around x; fix: apply([x]: x + 5)

D. Function must be defined outside apply; fix: def f(x): return x+5

Solution

1. Step 1: Identify syntax error in lambda function

   The code uses x: x + 5 without the lambda keyword, which is required to define an anonymous function.

2. Step 2: Correct the syntax

   Add lambda before the function: df['value'].apply(lambda x: x + 5).

3. Final Answer:

   Missing lambda keyword; fix: df['value'].apply(lambda x: x + 5) -> Option A

4. Quick Check:

   Lambda keyword needed for inline functions [OK]

Hint: Always use 'lambda' before inline functions in apply() [OK]

Common Mistakes:

• Omitting the lambda keyword
• Using incorrect brackets or syntax for lambda
• Thinking apply() can't use inline functions

5. You have a DataFrame df with a column 'temperature' in Celsius. You want to create a new column 'temp_f' with Fahrenheit values using apply(). Which code correctly does this?

hard

A. df['temp_f'] = df.apply(lambda c: (c * 9/5) + 32)

B. df['temp_f'] = df['temperature'].apply(lambda c: (c * 9/5) + 32)

C. df['temp_f'] = df['temperature'].apply(lambda c: c + 32 * 9/5)

D. df['temp_f'] = df['temperature'].apply(lambda c: c * (9/5 + 32))

Solution

1. Step 1: Recall the formula for Celsius to Fahrenheit

   The correct formula is Fahrenheit = (Celsius * 9/5) + 32.

2. Step 2: Check each option's lambda function

   df['temp_f'] = df['temperature'].apply(lambda c: (c * 9/5) + 32) correctly applies the formula to each value in 'temperature'. df['temp_f'] = df.apply(lambda c: (c * 9/5) + 32) applies to whole DataFrame, C and D have incorrect formulas.

3. Final Answer:

   df['temp_f'] = df['temperature'].apply(lambda c: (c * 9/5) + 32) -> Option B

4. Quick Check:

   Use correct formula inside apply() on the column [OK]

Hint: Use correct formula inside lambda for each column value [OK]

Common Mistakes:

• Applying function to whole DataFrame instead of column
• Mixing order of operations in formula
• Forgetting parentheses in formula