Creating new columns in Pandas - Visual Walkthrough

Concept Flow - Creating new columns

Start with DataFrame

↓

Define new column values

↓

Assign new column to DataFrame

↓

DataFrame updated with new column

↓

End

We start with a DataFrame, define values for a new column, assign it, and update the DataFrame.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 3]})
df['B'] = df['A'] * 2
print(df)

This code creates a new column 'B' by doubling the values in column 'A'.

Execution Table

Step	Action	DataFrame State	New Column 'B' Values
1	Create DataFrame with column 'A'	{'A': [1, 2, 3]}	N/A
2	Calculate new column 'B' as df['A'] * 2	Same as step 1	[2, 4, 6]
3	Assign new column 'B' to DataFrame	{'A': [1, 2, 3], 'B': [2, 4, 6]}	[2, 4, 6]
4	Print DataFrame	Displays both columns	[2, 4, 6]

💡 New column 'B' added successfully; execution ends after printing DataFrame.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
df	undefined	{'A': [1, 2, 3]}	{'A': [1, 2, 3]}	{'A': [1, 2, 3], 'B': [2, 4, 6]}	{'A': [1, 2, 3], 'B': [2, 4, 6]}
df['B']	undefined	undefined	[2, 4, 6]	[2, 4, 6]	[2, 4, 6]

Key Moments - 2 Insights

Why does the new column 'B' appear only after assignment and not before?

Because calculating df['A'] * 2 creates a Series but does not add it to df until we assign it as df['B'] (see execution_table step 2 vs 3).

Can we create a new column using a list of values directly?

Yes, as long as the list length matches the DataFrame rows. Assignment like df['B'] = [2,4,6] works (similar to step 3).

Visual Quiz - 3 Questions

Look at the execution table at step 2, what are the values calculated for the new column 'B'?

A N/A

B [1, 2, 3]

C [2, 4, 6]

D [3, 6, 9]

💡 Hint

Check the 'New Column 'B' Values' column in execution_table row for step 2.

At which step does the DataFrame actually get updated with the new column 'B'?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look at the 'DataFrame State' column in execution_table to see when 'B' appears.

If we assign df['B'] = [10, 20], what will happen?

A Error due to length mismatch

B New column 'B' with values [10, 20] added

C Column 'B' will have NaN for missing rows

D Column 'B' will be empty

💡 Hint

Recall that the list length must match DataFrame rows (see key_moments about list length).

Concept Snapshot

Creating new columns in pandas:
- Use df['new_col'] = values
- Values can be Series, list, or calculations
- Length of values must match DataFrame rows
- New column appears after assignment
- Useful for adding derived data

Full Transcript

We start with a DataFrame having one column 'A'. We calculate new values by doubling 'A' and store them temporarily. Only after assigning these values to df['B'] does the DataFrame update to include the new column. Printing shows both columns. Beginners often wonder why the new column isn't visible before assignment and must ensure the new column values match the DataFrame length. This process allows adding new data columns easily.

Practice

1. What is the correct way to add a new column named age + 10 to a DataFrame df that adds 10 to the existing age column?

easy

A. df['age + 10'] == df['age'] + 10

B. df['age + 10'] = df['age'] + 10

C. df.add_column('age + 10', df['age'] + 10)

D. df.age + 10 = df.age + 10

Solution

1. Step 1: Understand how to create a new column in pandas

   In pandas, new columns are created by assigning a value to df['new_column_name'].

2. Step 2: Apply the addition operation to the existing column

   Adding 10 to the age column is done by df['age'] + 10. Assigning this to df['age + 10'] creates the new column.

3. Final Answer:

   df['age + 10'] = df['age'] + 10 -> Option B

4. Quick Check:

   New column assignment = df['age + 10'] = df['age'] + 10 [OK]

Hint: Use df['new_col'] = expression to add columns [OK]

Common Mistakes:

• Using double equals (==) instead of single equals (=) for assignment
• Trying to use a method like add_column which doesn't exist
• Using dot notation for new columns with spaces or special characters

2. Which of the following is the correct syntax to create a new column full_name by combining first_name and last_name columns with a space in between in DataFrame df?

easy

A. df['full_name'] = df['first_name'] + ' ' + df['last_name']

B. df['full_name'] = df['first_name'].append(' ').append(df['last_name'])

C. df['full_name'] = df['first_name'].join(' ', df['last_name'])

D. df['full_name'] = df['first_name'] & ' ' & df['last_name']

Solution

1. Step 1: Combine string columns with + operator

   In pandas, string columns can be combined using the + operator. To add a space, include ' ' between them.

2. Step 2: Assign the combined string to a new column

   Assign the result to df['full_name'] to create the new column.

3. Final Answer:

   df['full_name'] = df['first_name'] + ' ' + df['last_name'] -> Option A

4. Quick Check:

   String concat with + = df['full_name'] = df['first_name'] + ' ' + df['last_name'] [OK]

Hint: Use + to join string columns with spaces [OK]

Common Mistakes:

• Using append or join methods which don't work on pandas Series
• Using & operator which is for bitwise operations
• Forgetting to add space between names

3. Given the DataFrame df below:

   price  quantity
0    100         2
1    150         3
2    200         1

What will be the output of this code?

df['total'] = df['price'] * df['quantity']
df['total']

medium

A. [200, 450, 200]

B. [100, 150, 200]

C. [2, 3, 1]

D. [300, 450, 600]

Solution

1. Step 1: Multiply price and quantity element-wise

   Multiply each row's price by quantity: 100*2=200, 150*3=450, 200*1=200.

2. Step 2: Assign result to new column and output it

   The new column 'total' will have values [200, 450, 200]. Printing df['total'] shows this list.

3. Final Answer:

   [200, 450, 200] -> Option A

4. Quick Check:

   Element-wise multiply = [200, 450, 200] [OK]

Hint: Multiply columns element-wise for new column [OK]

Common Mistakes:

• Adding instead of multiplying columns
• Confusing row-wise with column-wise operations
• Expecting a scalar instead of a Series output

4. What is wrong with this code that tries to create a new column discounted_price by subtracting 10 from price in DataFrame df?

df.discounted_price = df['price'] - 10

medium

A. It will fail if the column name has spaces or special characters.

B. It will cause a syntax error because of dot notation.

C. It will create a new variable, not a DataFrame column.

D. It will create the column correctly.

Solution

1. Step 1: Understand dot notation for DataFrame columns

   Dot notation df.col = value works to create new columns only if 'col' is a valid Python identifier (no spaces, no special characters).

2. Step 2: Check the column name 'discounted_price'

   'discounted_price' consists of letters and underscores, which is valid. Therefore, the code creates the column correctly.

3. Final Answer:

   It will create the column correctly. -> Option D

4. Quick Check:

   Dot notation works for valid identifiers [OK]

Hint: Use df['col'] if column name has spaces or special chars [OK]

Common Mistakes:

• Thinking dot notation fails for valid column names
• Confusing DataFrame column assignment with local variable creation
• Believing underscores invalidate column names for dot notation

5. You have a DataFrame df with columns score and max_score. You want to create a new column percentage that shows the score as a percentage of max_score, but if max_score is zero, set percentage to None. Which code correctly does this?

hard

A. df['percentage'] = df['score'] / df['max_score'].replace(0, None) * 100

B. df.loc[df['max_score'] == 0, 'percentage'] = None
df['percentage'] = df['score'] / df['max_score'] * 100

C. df['percentage'] = df.apply(lambda row: row['score'] / row['max_score'] * 100 if row['max_score'] != 0 else None, axis=1)

D. df['percentage'] = df['score'] / df['max_score'] * 100
df['percentage'][df['max_score'] == 0] = None

Solution

1. Step 1: Use row-wise function to handle zero max_score

   Using df.apply with a lambda lets us check each row and apply the condition: if max_score is zero, return None; else calculate percentage.

2. Step 2: Assign the result to new column 'percentage'

   The lambda returns the correct value per row, and axis=1 ensures row-wise operation.

3. Final Answer:

   df['percentage'] = df.apply(lambda row: row['score'] / row['max_score'] * 100 if row['max_score'] != 0 else None, axis=1) -> Option C

4. Quick Check:

   Use apply with condition for safe division [OK]

Hint: Use apply with lambda for row-wise conditional columns [OK]

Common Mistakes:

• Replacing zero with None in Series causes errors
• Using chained indexing which raises warnings
• Not handling division by zero properly