When to use apply vs vectorized operations in Pandas - Hands-On Comparison

When to use apply vs vectorized operations

📖 Scenario: You work in a small online store. You have a list of product prices and want to calculate the final price after a discount. You want to learn the best way to do this using pandas.

🎯 Goal: Learn how to use vectorized operations and the apply method in pandas to calculate discounted prices. Understand when to use each method.

📋 What You'll Learn

Create a pandas DataFrame with product names and prices

Create a discount rate variable

Use vectorized operations to calculate discounted prices

Use the apply method to calculate discounted prices

Print the results to compare both methods

💡 Why This Matters

🌍 Real World

In real online stores or sales data, you often need to adjust prices or calculate new values based on existing data. Knowing when to use fast vectorized operations or flexible apply functions helps you write efficient and clear code.

💼 Career

Data analysts and data scientists frequently manipulate data tables. Understanding pandas vectorized operations and apply method is essential for cleaning, transforming, and analyzing data efficiently.

1

Create the product DataFrame

Create a pandas DataFrame called products with two columns: 'Product' and 'Price'. Add these exact rows: 'Pen', 1.20, 'Notebook', 2.50, 'Eraser', 0.50, 'Pencil', 0.80.

Pandas

# Create the products DataFrame with Product and Price columns
# Your code here

Hint

Use pd.DataFrame with a dictionary where keys are column names and values are lists of column values.

2

Set the discount rate

Create a variable called discount_rate and set it to 0.1 to represent a 10% discount.

Pandas

import pandas as pd

products = pd.DataFrame({
    'Product': ['Pen', 'Notebook', 'Eraser', 'Pencil'],
    'Price': [1.20, 2.50, 0.50, 0.80]
})

# Create discount_rate variable
# Your code here

Hint

Just assign 0.1 to the variable discount_rate.

3

Calculate discounted prices using vectorized operations

Create a new column in products called 'Discounted_Vectorized'. Calculate the discounted price by subtracting discount_rate times Price from Price using vectorized operations.

Pandas

import pandas as pd

products = pd.DataFrame({
    'Product': ['Pen', 'Notebook', 'Eraser', 'Pencil'],
    'Price': [1.20, 2.50, 0.50, 0.80]
})

discount_rate = 0.1

# Calculate discounted prices using vectorized operations
# Your code here

Hint

Use products['Discounted_Vectorized'] = products['Price'] - products['Price'] * discount_rate.

4

Calculate discounted prices using apply and print both results

Create a new column in products called 'Discounted_Apply'. Use the apply method with a lambda function that subtracts discount_rate times the price from the price. Then print the entire products DataFrame.

Pandas

import pandas as pd

products = pd.DataFrame({
    'Product': ['Pen', 'Notebook', 'Eraser', 'Pencil'],
    'Price': [1.20, 2.50, 0.50, 0.80]
})

discount_rate = 0.1

products['Discounted_Vectorized'] = products['Price'] - products['Price'] * discount_rate

# Calculate discounted prices using apply and print products
# Your code here

Hint

Use products['Discounted_Apply'] = products['Price'].apply(lambda price: price - price * discount_rate) and then print(products).

Practice

1. Which of the following best describes when to use apply in pandas?

easy

A. When you want to merge two DataFrames.

B. When you want to perform simple arithmetic on entire columns quickly.

C. When you want to sort a DataFrame by a column.

D. When you need to run a custom function on each row or column.

Solution

1. Step 1: Understand the purpose of apply

   apply is used to apply custom or complex functions to rows or columns in a DataFrame.

2. Step 2: Compare with vectorized operations

   Vectorized operations are for simple, fast math on entire columns, not custom row-wise functions.

3. Final Answer:

   When you need to run a custom function on each row or column. -> Option D

4. Quick Check:

   Custom function use = apply = A [OK]

Hint: Use apply for custom row/column functions, vectorized for simple math [OK]

Common Mistakes:

• Confusing apply with vectorized operations for simple math
• Using apply for sorting or merging
• Thinking apply is always faster

2. Which of these is the correct syntax to use apply on a DataFrame column df['col'] with a function func?

easy

A. apply(df['col'], func)

B. df['col'].apply(func)

C. df.apply['col'](func)

D. df['col'].apply = func

Solution

1. Step 1: Recall the method call syntax

   In pandas, apply is a method called on a Series or DataFrame with the function as argument.

2. Step 2: Check each option

   df['col'].apply(func) uses correct syntax: df['col'].apply(func). Others have syntax errors or wrong usage.

3. Final Answer:

   df['col'].apply(func) -> Option B

4. Quick Check:

   Method call with function argument = C [OK]

Hint: Use dot apply with parentheses and function inside [OK]

Common Mistakes:

• Using square brackets instead of parentheses for apply
• Trying to assign function to apply attribute
• Calling apply as a standalone function

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})

What is the output of:

df['C'] = df['A'] + df['B']
df['D'] = df['A'].apply(lambda x: x * 2)
df[['C', 'D']]

medium

A. C D
0 5 2
1 7 4
2 9 6

B. C D
0 5 1
1 7 2
2 9 3

C. C D
0 4 2
1 5 4
2 6 6

D. C D
0 5 4
1 7 6
2 9 8

Solution

1. Step 1: Calculate column C using vectorized addition

   Column C = A + B = [1+4, 2+5, 3+6] = [5, 7, 9]

2. Step 2: Calculate column D using apply with lambda

   Column D = A * 2 = [1*2, 2*2, 3*2] = [2, 4, 6]

3. Final Answer:

   C D 0 5 2 1 7 4 2 9 6 -> Option A

4. Quick Check:

   Vectorized + apply results = B [OK]

Hint: Add columns directly, use apply for custom row-wise ops [OK]

Common Mistakes:

• Mixing up which columns are added
• Forgetting to multiply by 2 in apply
• Confusing apply output with original values

4. You want to double the values in column 'A' of DataFrame df. Which code has an error?

medium

A. df['A'] = df['A'] * 2

B. df['A'] = df['A'].apply(lambda x: x * 2)

C. df['A'] = df.apply(lambda x: x['A'] * 2)

D. df['A'] = df['A'].map(lambda x: x * 2)

Solution

1. Step 1: Analyze df['A'] = df.apply(lambda x: x['A'] * 2)'s use of apply

   Using df.apply without specifying axis defaults to axis=0 (columns), so x['A'] fails because x is a column name string, not a row.

2. Step 2: Compare with other options

   Options B, C, and D correctly double values using apply on Series, vectorized, or map on Series.

3. Final Answer:

   df['A'] = df.apply(lambda x: x['A'] * 2) -> Option C

4. Quick Check:

   apply on DataFrame needs axis=1 for row-wise, else error = A [OK]

Hint: Use axis=1 with apply on DataFrame rows to avoid errors [OK]

Common Mistakes:

• Forgetting axis=1 in DataFrame apply
• Confusing apply on DataFrame vs Series
• Using map on DataFrame instead of Series

5. You have a DataFrame df with columns 'X' and 'Y'. You want to create a new column 'Z' that is 'X' squared if 'Y' is even, else 'X' cubed. Which is the best approach?

hard

A. Use vectorized operations with np.where and powers.

B. Use apply with a custom function checking Y and computing Z.

C. Use a for loop to iterate rows and assign Z values.

D. Use map on column 'X' only.

Solution

1. Step 1: Understand the condition and operations

   We want Z = X^2 if Y is even, else Z = X^3.

2. Step 2: Choose efficient method

   Using np.where with vectorized power operations is fast and clear. apply works but is slower. For loops are slow and verbose. map on X alone can't check Y.

3. Final Answer:

   Use vectorized operations with np.where and powers. -> Option A

4. Quick Check:

   Conditional vectorized ops > apply or loops = D [OK]

Hint: Use np.where for conditional vectorized operations [OK]

Common Mistakes:

• Using apply when vectorized is faster
• Using loops instead of vectorized code
• Trying to map without condition on another column