Why custom functions matter in Pandas - The Real Reasons

The Big Idea

What if you could fix your data problems once and never worry about repeating the same work again?

The Scenario

Imagine you have a big spreadsheet with hundreds of rows, and you need to clean or change the data in many places. Doing this by hand or copying and pasting formulas over and over is tiring and confusing.

The Problem

Manually repeating the same steps takes a lot of time and is easy to mess up. If you make a mistake once, it can spread everywhere. Also, if you want to change the way you clean data, you have to redo everything again.

The Solution

Custom functions let you write your cleaning or calculation steps once, then use them again and again on any data. This saves time, reduces mistakes, and makes your work easy to update.

Before vs After

✗ Before

df['new'] = df['old'] * 2
# Repeat this for every new calculation

✓ After

def double(x):
    return x * 2

df['new'] = df['old'].apply(double)

What It Enables

With custom functions, you can quickly apply complex changes to your data, making your analysis faster and more reliable.

Real Life Example

A sales analyst needs to adjust prices based on different rules. Instead of rewriting formulas for each product, they create a custom function to apply the rules automatically to all products.

Key Takeaways

Manual data changes are slow and error-prone.

Custom functions let you reuse and update your work easily.

This makes data cleaning and analysis faster and safer.

Practice

1. Why do we use custom functions when working with pandas DataFrames?

easy

A. To reuse code and make data changes clear and simple

B. To make the DataFrame larger automatically

C. To delete all data quickly

D. To change the pandas version

Solution

1. Step 1: Understand the purpose of custom functions

   Custom functions let us write a piece of code once and use it many times, which saves effort.

2. Step 2: Connect to pandas usage

   In pandas, custom functions help apply the same change or calculation to many rows or columns easily and clearly.

3. Final Answer:

   To reuse code and make data changes clear and simple -> Option A

4. Quick Check:

   Custom functions improve code reuse and clarity [OK]

Hint: Custom functions simplify repeated data changes [OK]

Common Mistakes:

• Thinking custom functions automatically change DataFrame size
• Believing custom functions delete data
• Confusing custom functions with pandas version control

2. Which of the following is the correct way to define a simple custom function in Python to double a number?

easy

A. function double(x) { return x * 2 }

B. def double x: return x * 2

C. def double(x): return x * 2

D. double = x => x * 2

Solution

1. Step 1: Identify Python function syntax

   Python functions start with 'def', followed by the function name and parentheses with parameters.

2. Step 2: Check each option

   def double(x): return x * 2 uses correct Python syntax. function double(x) { return x * 2 } is JavaScript syntax. def double x: return x * 2 misses parentheses. double = x => x * 2 is JavaScript arrow function syntax.

3. Final Answer:

   def double(x): return x * 2 -> Option C

4. Quick Check:

   Python function syntax = def name(params): [OK]

Hint: Python functions start with 'def' and use parentheses [OK]

Common Mistakes:

• Using JavaScript syntax in Python
• Omitting parentheses in function definition
• Confusing arrow functions with Python syntax

3. Given the code below, what will be the output of df['new']?

import pandas as pd
def add_five(x):
    return x + 5
df = pd.DataFrame({'num': [1, 2, 3]})
df['new'] = df['num'].apply(add_five)
print(df['new'])

medium

A. [1, 2, 3]

B. [6, 7, 8]

C. [5, 6, 7]

D. Error: add_five is not defined

Solution

1. Step 1: Understand the function add_five

   The function adds 5 to the input number.

2. Step 2: Apply the function to each element in 'num'

   Applying add_five to [1, 2, 3] results in [6, 7, 8].

3. Final Answer:

   [6, 7, 8] -> Option B

4. Quick Check:

   1+5=6, 2+5=7, 3+5=8 [OK]

Hint: Apply adds function result to each element [OK]

Common Mistakes:

• Thinking apply changes original column without assignment
• Confusing function name or missing definition
• Expecting original values without addition

4. What is wrong with this code snippet?

import pandas as pd
def multiply_by_two(x):
    x * 2
df = pd.DataFrame({'val': [4, 5, 6]})
df['result'] = df['val'].apply(multiply_by_two)
print(df['result'])

medium

A. The DataFrame is not imported correctly

B. The DataFrame is missing a column

C. apply() cannot be used with custom functions

D. The function multiply_by_two does not return a value

Solution

1. Step 1: Check the function multiply_by_two

   The function calculates x * 2 but does not return the result.

2. Step 2: Understand apply behavior

   apply expects the function to return a value to create the new column. Without return, it returns None for each row.

3. Final Answer:

   The function multiply_by_two does not return a value -> Option D

4. Quick Check:

   Functions must return values for apply to work [OK]

Hint: Always return a value in your custom function [OK]

Common Mistakes:

• Forgetting the return statement in functions
• Assuming apply works without returned values
• Thinking DataFrame import affects apply

5. You have a DataFrame with a column 'score' containing numbers. You want to create a new column 'grade' that assigns 'Pass' if the score is 50 or more, and 'Fail' otherwise. Which custom function and pandas code correctly achieve this?

hard

A. def grade_func(x): return 'Pass' if x >= 50 else 'Fail'
df['grade'] = df['score'].apply(grade_func)

B. def grade_func(x): if x >= 50: 'Pass' else: 'Fail'
df['grade'] = df['score'].apply(grade_func)

C. def grade_func(x): return x >= 50 or 'Pass' and 'Fail'
df['grade'] = df['score'].apply(grade_func)

D. def grade_func(x): return 'Pass' if x > 50 else 'Fail'
df['grade'] = df['score'].apply(grade_func)

Solution

1. Step 1: Check the function logic for grading

   def grade_func(x): return 'Pass' if x >= 50 else 'Fail' df['grade'] = df['score'].apply(grade_func) correctly returns 'Pass' if x is 50 or more, else 'Fail'. def grade_func(x): if x >= 50: 'Pass' else: 'Fail' df['grade'] = df['score'].apply(grade_func) misses return statement. def grade_func(x): return x >= 50 or 'Pass' and 'Fail' df['grade'] = df['score'].apply(grade_func) uses incorrect logical operators, resulting in True or 'Fail'. def grade_func(x): return 'Pass' if x > 50 else 'Fail' df['grade'] = df['score'].apply(grade_func) uses > 50, which excludes 50.

2. Step 2: Confirm pandas apply usage

   All options use apply correctly, but only def grade_func(x): return 'Pass' if x >= 50 else 'Fail' df['grade'] = df['score'].apply(grade_func) has correct function logic and syntax.

3. Final Answer:

   def grade_func(x): return 'Pass' if x >= 50 else 'Fail' df['grade'] = df['score'].apply(grade_func) -> Option A

4. Quick Check:

   Use return with correct condition for grading [OK]

Hint: Use return and correct condition in custom function [OK]

Common Mistakes:

• Forgetting return in function
• Using wrong comparison operator
• Confusing logical expressions without return