Vectorized operations vs loops in Pandas - Visual Side-by-Side Comparison

Concept Flow - Vectorized operations vs loops

Start with DataFrame

↓

Choose operation

↓

Use loop

↓

Iterate rows

↓

Update values

↓

Compare speed & simplicity

↓

End

This flow shows two ways to update data: looping row by row or using vectorized operations that work on whole columns at once.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 3]})

# Loop method
for i in range(len(df)):
    df.loc[i, 'B'] = df.loc[i, 'A'] * 2

# Vectorized method
df['C'] = df['A'] * 2

This code creates a DataFrame and adds two new columns B and C by doubling column A, first using a loop, then vectorized operation.

Execution Table

Step	i	df.loc[i, 'A']	Action	df.loc[i, 'B']	df['C']
1	0	1	Loop: Multiply 1 * 2	2
2	1	2	Loop: Multiply 2 * 2	4
3	2	3	Loop: Multiply 3 * 2	6
4			Vectorized: Multiply entire 'A' by 2		[2, 4, 6]
5			End of operations	B column filled	C column filled

💡 Loop ends after i=2 (last index). Vectorized operation applies to whole column at once.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After vectorized	Final
i	N/A	0	1	2	N/A	N/A
df.loc[i, 'B']	NaN	2	4	6	6	6
df['C']	NaN	NaN	NaN	NaN	[2, 4, 6]	[2, 4, 6]

Key Moments - 3 Insights

Why is the vectorized operation faster than the loop?

Vectorized operations work on the whole column at once using optimized C code inside pandas, while loops process one row at a time in Python, which is slower. See execution_table rows 1-3 vs 4.

Does the loop method create the new column 'B' immediately?

No, the loop assigns values one by one to 'B'. Initially, 'B' does not exist. After the first iteration, 'B' starts to form. See variable_tracker for 'df.loc[i, B]' changes.

Are the results from loop and vectorized operations the same?

Yes, both methods produce the same doubled values in new columns 'B' and 'C'. Check execution_table columns for 'df.loc[i, B]' and 'df["C"]'.

Visual Quiz - 3 Questions

Look at the execution table, what is the value of df.loc[1, 'B'] after step 2?

A 4

B 2

C 6

D NaN

💡 Hint

Check row with Step 2 in execution_table under 'df.loc[i, B]' column.

At which step does the vectorized operation update the entire 'C' column?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Look for the step mentioning vectorized operation in execution_table.

If we remove the loop and only use vectorized operation, what changes in variable_tracker?

A 'df["C"]' becomes NaN

B 'df.loc[i, B]' remains NaN throughout

C 'df.loc[i, B]' gets filled as in the loop

D 'i' variable increments more times

💡 Hint

Variable 'df.loc[i, B]' is only updated inside the loop, see variable_tracker.

Concept Snapshot

Vectorized operations apply functions to whole columns or arrays at once.
Loops process data row by row, which is slower.
Use vectorized methods in pandas for faster, simpler code.
Loops can be used but are less efficient.
Vectorized code is cleaner and leverages pandas optimizations.

Full Transcript

This lesson compares two ways to update data in pandas: using loops and vectorized operations. We start with a DataFrame with one column 'A'. The loop method goes row by row, multiplying each value by 2 and storing it in a new column 'B'. The vectorized method multiplies the entire 'A' column by 2 at once and stores it in 'C'. The execution table shows each step, variable values, and actions. The variable tracker follows how variables change after each step. Key moments clarify why vectorized operations are faster and how columns are created. The quiz tests understanding of values at specific steps and the difference between methods. The snapshot summarizes that vectorized operations are faster and simpler than loops in pandas.

Practice

1. Which of the following best describes vectorized operations in pandas?

easy

A. They require writing explicit for-loops to manipulate data.

B. They process data one row at a time using loops.

C. They operate on entire columns or arrays at once, making them faster.

D. They are slower than loops for large datasets.

Solution

1. Step 1: Understand vectorized operations

   Vectorized operations apply a function to whole columns or arrays simultaneously.

2. Step 2: Compare with loops

   Loops process data row by row, which is slower for large data.

3. Final Answer:

   They operate on entire columns or arrays at once, making them faster. -> Option C

4. Quick Check:

   Vectorized = whole columns at once [OK]

Hint: Vectorized means whole column at once, not row by row [OK]

Common Mistakes:

• Confusing vectorized operations with loops
• Thinking loops are faster for large data
• Believing vectorized operations require explicit loops

2. Which of the following is the correct syntax to add 10 to every element in a pandas Series s using vectorized operations?

easy

A. for i in s: s[i] = s[i] + 10

B. s = s + 10

C. s.apply(lambda x: x + 10)

D. s.add(10, inplace=True)

Solution

1. Step 1: Identify vectorized addition syntax

   Adding a scalar to a Series directly uses vectorized addition: s = s + 10.

2. Step 2: Check other options

   for i in s: s[i] = s[i] + 10 uses a loop incorrectly, C returns a new Series but does not assign, B is invalid because add does not have inplace.

3. Final Answer:

   s = s + 10 -> Option B

4. Quick Check:

   Use s + 10 for vectorized add [OK]

Hint: Use s + 10 for vectorized addition, no loops [OK]

Common Mistakes:

• Trying to modify Series inside a for-loop
• Using apply without assignment
• Using invalid inplace parameter with add()

3. What will be the output of this code?

import pandas as pd
s = pd.Series([1, 2, 3])
result = []
for val in s:
    result.append(val * 2)
print(result)

medium

A. Error: cannot multiply Series elements in loop

B. pd.Series([2, 4, 6])

C. [1, 2, 3]

D. [2, 4, 6]

Solution

1. Step 1: Understand the loop operation

   The loop goes through each value in the Series and multiplies it by 2, appending to a list.

2. Step 2: Check the printed output

   The printed output is the list [2, 4, 6], not a Series.

3. Final Answer:

   [2, 4, 6] -> Option D

4. Quick Check:

   Loop multiplies each value, list output [OK]

Hint: Loop appends multiplied values to list, not Series [OK]

Common Mistakes:

• Expecting a Series output instead of list
• Confusing loop with vectorized operation
• Thinking loop causes error multiplying values

4. Identify the error in this code that tries to double values in a pandas Series s:

result = []
for i in range(len(s)):
    result.append(s[i] * 2)
print(result)

medium

A. No error; code works correctly

B. Accessing Series elements by integer index is invalid

C. Using range(len(s)) causes an IndexError

D. Appending to list inside loop is not allowed

Solution

1. Step 1: Check indexing method

   Accessing Series elements by integer position with s[i] works if Series index is default 0-based.

2. Step 2: Verify loop and append

   The loop appends doubled values to list correctly; no error occurs.

3. Final Answer:

   No error; code works correctly -> Option A

4. Quick Check:

   Indexing by position works if default index [OK]

Hint: Default Series index allows integer access [OK]

Common Mistakes:

• Assuming s[i] always causes error
• Thinking append inside loop is invalid
• Confusing label-based and position-based indexing

5. You have a DataFrame df with a column 'price'. You want to increase all prices by 5% efficiently. Which method is best?

hard

A. Use df['price'] = df['price'] * 1.05 to apply vectorized multiplication.

B. Use a for-loop to multiply each price by 1.05 and update the DataFrame row by row.

C. Use df.apply(lambda row: row['price'] * 1.05, axis=1) without assignment.

D. Convert the column to a list, multiply each element by 1.05 in a loop, then assign back.

Solution

1. Step 1: Identify efficient vectorized method

   Multiplying the entire column by 1.05 directly uses vectorized operations and is fastest.

2. Step 2: Evaluate other options

   Use a for-loop to multiply each price by 1.05 and update the DataFrame row by row. uses slow loops, C does not assign result back, D is inefficient with list conversion.

3. Final Answer:

   Use df['price'] = df['price'] * 1.05 to apply vectorized multiplication. -> Option A

4. Quick Check:

   Vectorized column operation is fastest [OK]

Hint: Multiply whole column directly for speed [OK]

Common Mistakes:

• Using loops instead of vectorized operations
• Forgetting to assign result back to DataFrame
• Converting to list unnecessarily