Creating new columns in Pandas

Introduction

We create new columns to add extra information or calculations to our data table. This helps us analyze data better.

You want to calculate a new value from existing columns, like total price from quantity and price per item.

You need to label data based on conditions, like marking students as pass or fail based on scores.

You want to add a constant value to all rows, like a new column showing the data source.

You want to combine text from two columns into one, like full names from first and last names.

Syntax

Pandas

df['new_column'] = expression_using_existing_columns_or_values

Use square brackets with the new column name as a string to create or replace a column.

The expression can be a calculation, a constant, or a function applied to existing columns.

Examples

Creates a new column 'total' by multiplying 'quantity' and 'price' columns.

Pandas

df['total'] = df['quantity'] * df['price']

Adds a 'status' column with 'pass' or 'fail' based on the 'score' column.

Pandas

df['status'] = ['pass' if x >= 50 else 'fail' for x in df['score']]

Adds a new column 'constant' with the same value 100 for all rows.

Pandas

df['constant'] = 100

Combines 'first_name' and 'last_name' columns into a new 'full_name' column.

Pandas

df['full_name'] = df['first_name'] + ' ' + df['last_name']

Sample Program

This code creates a DataFrame with some sample data. Then it adds four new columns:

• total: product of quantity and price
• status: pass/fail based on score
• constant: a fixed value 100
• full_name: combined first and last names

Finally, it prints the updated DataFrame.

Pandas

import pandas as pd

data = {'quantity': [2, 5, 3], 'price': [10, 7, 12], 'score': [45, 82, 67], 'first_name': ['Anna', 'Bob', 'Cara'], 'last_name': ['Smith', 'Jones', 'Lee']}
df = pd.DataFrame(data)

df['total'] = df['quantity'] * df['price']
df['status'] = ['pass' if x >= 50 else 'fail' for x in df['score']]
df['constant'] = 100
df['full_name'] = df['first_name'] + ' ' + df['last_name']

print(df)

Important Notes

When creating new columns with conditions, list comprehensions or apply() can be used.

Assigning a constant value adds the same value to all rows.

Make sure the new column name does not conflict with existing columns unless you want to replace them.

Summary

New columns add extra information or calculations to your data.

Use df['new_column'] = ... to create or update columns.

You can use calculations, conditions, constants, or combine text to make new columns.

Practice

1. What is the correct way to add a new column named age + 10 to a DataFrame df that adds 10 to the existing age column?

easy

A. df['age + 10'] == df['age'] + 10

B. df['age + 10'] = df['age'] + 10

C. df.add_column('age + 10', df['age'] + 10)

D. df.age + 10 = df.age + 10

Solution

1. Step 1: Understand how to create a new column in pandas

   In pandas, new columns are created by assigning a value to df['new_column_name'].

2. Step 2: Apply the addition operation to the existing column

   Adding 10 to the age column is done by df['age'] + 10. Assigning this to df['age + 10'] creates the new column.

3. Final Answer:

   df['age + 10'] = df['age'] + 10 -> Option B

4. Quick Check:

   New column assignment = df['age + 10'] = df['age'] + 10 [OK]

Hint: Use df['new_col'] = expression to add columns [OK]

Common Mistakes:

• Using double equals (==) instead of single equals (=) for assignment
• Trying to use a method like add_column which doesn't exist
• Using dot notation for new columns with spaces or special characters

2. Which of the following is the correct syntax to create a new column full_name by combining first_name and last_name columns with a space in between in DataFrame df?

easy

A. df['full_name'] = df['first_name'] + ' ' + df['last_name']

B. df['full_name'] = df['first_name'].append(' ').append(df['last_name'])

C. df['full_name'] = df['first_name'].join(' ', df['last_name'])

D. df['full_name'] = df['first_name'] & ' ' & df['last_name']

Solution

1. Step 1: Combine string columns with + operator

   In pandas, string columns can be combined using the + operator. To add a space, include ' ' between them.

2. Step 2: Assign the combined string to a new column

   Assign the result to df['full_name'] to create the new column.

3. Final Answer:

   df['full_name'] = df['first_name'] + ' ' + df['last_name'] -> Option A

4. Quick Check:

   String concat with + = df['full_name'] = df['first_name'] + ' ' + df['last_name'] [OK]

Hint: Use + to join string columns with spaces [OK]

Common Mistakes:

• Using append or join methods which don't work on pandas Series
• Using & operator which is for bitwise operations
• Forgetting to add space between names

3. Given the DataFrame df below:

   price  quantity
0    100         2
1    150         3
2    200         1

What will be the output of this code?

df['total'] = df['price'] * df['quantity']
df['total']

medium

A. [200, 450, 200]

B. [100, 150, 200]

C. [2, 3, 1]

D. [300, 450, 600]

Solution

1. Step 1: Multiply price and quantity element-wise

   Multiply each row's price by quantity: 100*2=200, 150*3=450, 200*1=200.

2. Step 2: Assign result to new column and output it

   The new column 'total' will have values [200, 450, 200]. Printing df['total'] shows this list.

3. Final Answer:

   [200, 450, 200] -> Option A

4. Quick Check:

   Element-wise multiply = [200, 450, 200] [OK]

Hint: Multiply columns element-wise for new column [OK]

Common Mistakes:

• Adding instead of multiplying columns
• Confusing row-wise with column-wise operations
• Expecting a scalar instead of a Series output

4. What is wrong with this code that tries to create a new column discounted_price by subtracting 10 from price in DataFrame df?

df.discounted_price = df['price'] - 10

medium

A. It will fail if the column name has spaces or special characters.

B. It will cause a syntax error because of dot notation.

C. It will create a new variable, not a DataFrame column.

D. It will create the column correctly.

Solution

1. Step 1: Understand dot notation for DataFrame columns

   Dot notation df.col = value works to create new columns only if 'col' is a valid Python identifier (no spaces, no special characters).

2. Step 2: Check the column name 'discounted_price'

   'discounted_price' consists of letters and underscores, which is valid. Therefore, the code creates the column correctly.

3. Final Answer:

   It will create the column correctly. -> Option D

4. Quick Check:

   Dot notation works for valid identifiers [OK]

Hint: Use df['col'] if column name has spaces or special chars [OK]

Common Mistakes:

• Thinking dot notation fails for valid column names
• Confusing DataFrame column assignment with local variable creation
• Believing underscores invalidate column names for dot notation

5. You have a DataFrame df with columns score and max_score. You want to create a new column percentage that shows the score as a percentage of max_score, but if max_score is zero, set percentage to None. Which code correctly does this?

hard

A. df['percentage'] = df['score'] / df['max_score'].replace(0, None) * 100

B. df.loc[df['max_score'] == 0, 'percentage'] = None
df['percentage'] = df['score'] / df['max_score'] * 100

C. df['percentage'] = df.apply(lambda row: row['score'] / row['max_score'] * 100 if row['max_score'] != 0 else None, axis=1)

D. df['percentage'] = df['score'] / df['max_score'] * 100
df['percentage'][df['max_score'] == 0] = None

Solution

1. Step 1: Use row-wise function to handle zero max_score

   Using df.apply with a lambda lets us check each row and apply the condition: if max_score is zero, return None; else calculate percentage.

2. Step 2: Assign the result to new column 'percentage'

   The lambda returns the correct value per row, and axis=1 ensures row-wise operation.

3. Final Answer:

   df['percentage'] = df.apply(lambda row: row['score'] / row['max_score'] * 100 if row['max_score'] != 0 else None, axis=1) -> Option C

4. Quick Check:

   Use apply with condition for safe division [OK]

Hint: Use apply with lambda for row-wise conditional columns [OK]

Common Mistakes:

• Replacing zero with None in Series causes errors
• Using chained indexing which raises warnings
• Not handling division by zero properly