Creating new columns in Pandas - Why You Should Know This

The Big Idea

What if you could add new data columns instantly without any manual typing or errors?

The Scenario

Imagine you have a spreadsheet with sales data, and you want to add a new column that shows the total price by multiplying quantity and price per item. Doing this by hand means opening each row and calculating manually.

The Problem

Manually adding new columns is slow and boring. It's easy to make mistakes, especially with many rows. Updating or changing calculations means redoing everything, which wastes time and causes frustration.

The Solution

Using pandas to create new columns lets you add or change data quickly and accurately. You write one simple line of code, and pandas applies the calculation to every row automatically.

Before vs After

✗ Before

for i in range(len(data)):
    data.loc[i, 'total'] = data.loc[i, 'quantity'] * data.loc[i, 'price']

✓ After

data['total'] = data['quantity'] * data['price']

What It Enables

This lets you explore and analyze data faster, making smarter decisions without tedious manual work.

Real Life Example

A store manager can quickly add a column for total sales per product to see which items earn the most money, helping decide what to stock more.

Key Takeaways

Manual column creation is slow and error-prone.

pandas lets you add new columns with simple, fast code.

This speeds up data analysis and reduces mistakes.

Practice

1. What is the correct way to add a new column named age + 10 to a DataFrame df that adds 10 to the existing age column?

easy

A. df['age + 10'] == df['age'] + 10

B. df['age + 10'] = df['age'] + 10

C. df.add_column('age + 10', df['age'] + 10)

D. df.age + 10 = df.age + 10

Solution

1. Step 1: Understand how to create a new column in pandas

   In pandas, new columns are created by assigning a value to df['new_column_name'].

2. Step 2: Apply the addition operation to the existing column

   Adding 10 to the age column is done by df['age'] + 10. Assigning this to df['age + 10'] creates the new column.

3. Final Answer:

   df['age + 10'] = df['age'] + 10 -> Option B

4. Quick Check:

   New column assignment = df['age + 10'] = df['age'] + 10 [OK]

Hint: Use df['new_col'] = expression to add columns [OK]

Common Mistakes:

• Using double equals (==) instead of single equals (=) for assignment
• Trying to use a method like add_column which doesn't exist
• Using dot notation for new columns with spaces or special characters

2. Which of the following is the correct syntax to create a new column full_name by combining first_name and last_name columns with a space in between in DataFrame df?

easy

A. df['full_name'] = df['first_name'] + ' ' + df['last_name']

B. df['full_name'] = df['first_name'].append(' ').append(df['last_name'])

C. df['full_name'] = df['first_name'].join(' ', df['last_name'])

D. df['full_name'] = df['first_name'] & ' ' & df['last_name']

Solution

1. Step 1: Combine string columns with + operator

   In pandas, string columns can be combined using the + operator. To add a space, include ' ' between them.

2. Step 2: Assign the combined string to a new column

   Assign the result to df['full_name'] to create the new column.

3. Final Answer:

   df['full_name'] = df['first_name'] + ' ' + df['last_name'] -> Option A

4. Quick Check:

   String concat with + = df['full_name'] = df['first_name'] + ' ' + df['last_name'] [OK]

Hint: Use + to join string columns with spaces [OK]

Common Mistakes:

• Using append or join methods which don't work on pandas Series
• Using & operator which is for bitwise operations
• Forgetting to add space between names

3. Given the DataFrame df below:

   price  quantity
0    100         2
1    150         3
2    200         1

What will be the output of this code?

df['total'] = df['price'] * df['quantity']
df['total']

medium

A. [200, 450, 200]

B. [100, 150, 200]

C. [2, 3, 1]

D. [300, 450, 600]

Solution

1. Step 1: Multiply price and quantity element-wise

   Multiply each row's price by quantity: 100*2=200, 150*3=450, 200*1=200.

2. Step 2: Assign result to new column and output it

   The new column 'total' will have values [200, 450, 200]. Printing df['total'] shows this list.

3. Final Answer:

   [200, 450, 200] -> Option A

4. Quick Check:

   Element-wise multiply = [200, 450, 200] [OK]

Hint: Multiply columns element-wise for new column [OK]

Common Mistakes:

• Adding instead of multiplying columns
• Confusing row-wise with column-wise operations
• Expecting a scalar instead of a Series output

4. What is wrong with this code that tries to create a new column discounted_price by subtracting 10 from price in DataFrame df?

df.discounted_price = df['price'] - 10

medium

A. It will fail if the column name has spaces or special characters.

B. It will cause a syntax error because of dot notation.

C. It will create a new variable, not a DataFrame column.

D. It will create the column correctly.

Solution

1. Step 1: Understand dot notation for DataFrame columns

   Dot notation df.col = value works to create new columns only if 'col' is a valid Python identifier (no spaces, no special characters).

2. Step 2: Check the column name 'discounted_price'

   'discounted_price' consists of letters and underscores, which is valid. Therefore, the code creates the column correctly.

3. Final Answer:

   It will create the column correctly. -> Option D

4. Quick Check:

   Dot notation works for valid identifiers [OK]

Hint: Use df['col'] if column name has spaces or special chars [OK]

Common Mistakes:

• Thinking dot notation fails for valid column names
• Confusing DataFrame column assignment with local variable creation
• Believing underscores invalidate column names for dot notation

5. You have a DataFrame df with columns score and max_score. You want to create a new column percentage that shows the score as a percentage of max_score, but if max_score is zero, set percentage to None. Which code correctly does this?

hard

A. df['percentage'] = df['score'] / df['max_score'].replace(0, None) * 100

B. df.loc[df['max_score'] == 0, 'percentage'] = None
df['percentage'] = df['score'] / df['max_score'] * 100

C. df['percentage'] = df.apply(lambda row: row['score'] / row['max_score'] * 100 if row['max_score'] != 0 else None, axis=1)

D. df['percentage'] = df['score'] / df['max_score'] * 100
df['percentage'][df['max_score'] == 0] = None

Solution

1. Step 1: Use row-wise function to handle zero max_score

   Using df.apply with a lambda lets us check each row and apply the condition: if max_score is zero, return None; else calculate percentage.

2. Step 2: Assign the result to new column 'percentage'

   The lambda returns the correct value per row, and axis=1 ensures row-wise operation.

3. Final Answer:

   df['percentage'] = df.apply(lambda row: row['score'] / row['max_score'] * 100 if row['max_score'] != 0 else None, axis=1) -> Option C

4. Quick Check:

   Use apply with condition for safe division [OK]

Hint: Use apply with lambda for row-wise conditional columns [OK]

Common Mistakes:

• Replacing zero with None in Series causes errors
• Using chained indexing which raises warnings
• Not handling division by zero properly