Why reshaping data matters in Pandas - Challenge Your Understanding

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output shape after pivoting?

Given the DataFrame df below, what will be the shape of the DataFrame after applying pivot?

Pandas

import pandas as pd

df = pd.DataFrame({
    'Date': ['2024-01-01', '2024-01-01', '2024-01-02', '2024-01-02'],
    'City': ['NY', 'LA', 'NY', 'LA'],
    'Temperature': [30, 60, 28, 65]
})

pivoted = df.pivot(index='Date', columns='City', values='Temperature')
print(pivoted.shape)

A (4, 1)

B (2, 4)

C (1, 4)

D (2, 2)

💡 Hint

Think about how many unique dates and cities there are.

The pivot creates a table with unique dates as rows and unique cities as columns. There are 2 unique dates and 2 unique cities, so the shape is (2, 2).

❓ data_output

intermediate

Result of melting a DataFrame

What is the resulting DataFrame after melting the following data?

Pandas

import pandas as pd

df = pd.DataFrame({
    'ID': [1, 2],
    'Math': [90, 80],
    'Science': [85, 95]
})
melted = pd.melt(df, id_vars=['ID'], var_name='Subject', value_name='Score')
print(melted)

A [{'ID':1,'Subject':'Math','Score':90},{'ID':2,'Subject':'Science','Score':95}]

B [{'ID':1,'Subject':'Math','Score':90},{'ID':1,'Subject':'Science','Score':85},{'ID':2,'Subject':'Math','Score':80},{'ID':2,'Subject':'Science','Score':95}]

C [{'Subject':'Math','Score':[90,80]},{'Subject':'Science','Score':[85,95]}]

D [{'ID':1,'Math':90,'Science':85},{'ID':2,'Math':80,'Science':95}]

💡 Hint

Melt turns columns into rows keeping the id_vars fixed.

The melt function converts columns 'Math' and 'Science' into rows under 'Subject' and their values under 'Score', repeating the 'ID' for each.

❓ visualization

advanced

Visualizing reshaped data for comparison

You have sales data for two products across three months in wide format. Which plot best shows the monthly sales comparison after reshaping the data to long format?

Pandas

import pandas as pd
import matplotlib.pyplot as plt

df = pd.DataFrame({
    'Month': ['Jan', 'Feb', 'Mar'],
    'Product_A': [100, 120, 130],
    'Product_B': [90, 110, 115]
})

long_df = pd.melt(df, id_vars=['Month'], var_name='Product', value_name='Sales')

plt.figure(figsize=(6,4))
for product in long_df['Product'].unique():
    subset = long_df[long_df['Product'] == product]
    plt.plot(subset['Month'], subset['Sales'], marker='o', label=product)
plt.legend()
plt.title('Monthly Sales Comparison')
plt.xlabel('Month')
plt.ylabel('Sales')
plt.tight_layout()
plt.show()

A Line plot with months on x-axis and sales on y-axis, separate lines for each product

B Pie chart showing total sales per product

C Bar chart with months on x-axis and sales stacked by product

D Scatter plot with sales on x-axis and months on y-axis

💡 Hint

Think about how to compare trends over time for multiple products.

The line plot clearly shows sales trends over months for each product after reshaping data to long format.

🔧 Debug

advanced

Identify the error in reshaping code

What error will this code raise?

Pandas

import pandas as pd

df = pd.DataFrame({
    'Date': ['2024-01-01', '2024-01-02'],
    'City': ['NY', 'LA'],
    'Temperature': [30, 60]
})

pivoted = df.pivot(index='Date', columns='Temperature', values='City')
print(pivoted)

A No error, prints pivoted DataFrame

B KeyError: 'Temperature'

C ValueError: Index contains duplicate entries, cannot reshape

D TypeError: unhashable type: 'list'

💡 Hint

Check if the index and columns have unique values.

Since 'Date' and 'Temperature' columns have unique values, pivot works without error and prints the reshaped DataFrame.

🚀 Application

expert

Choosing reshaping method for time series analysis

You have a DataFrame with daily sales data for multiple stores in wide format, where each store is a column. You want to prepare the data for time series analysis that requires a single sales column and a store identifier column. Which reshaping method should you use?

A Use <code>groupby</code> to aggregate sales by store

B Use <code>pivot</code> to convert the data to long format

C Use <code>melt</code> to convert the data to long format

D Use <code>concat</code> to join all store columns vertically

💡 Hint

Think about turning columns into rows while keeping identifiers.

melt is used to convert wide data into long format by turning columns into rows, which is needed for time series analysis with a single sales column and store identifier.

Practice

1. Why is reshaping data important in pandas?

easy

A. It organizes data to make analysis and visualization easier.

B. It deletes unnecessary rows automatically.

C. It changes data types of columns.

D. It encrypts data for security.

Solution

1. Step 1: Understand the purpose of reshaping

   Reshaping changes the layout of data to make it easier to work with.

2. Step 2: Identify the benefit for analysis and visualization

   Organized data helps tools and people understand patterns and trends better.

3. Final Answer:

   It organizes data to make analysis and visualization easier. -> Option A

4. Quick Check:

   Reshaping = Organizing data [OK]

Hint: Think: reshaping makes data easier to read and use [OK]

Common Mistakes:

• Confusing reshaping with deleting data
• Thinking reshaping changes data types
• Believing reshaping encrypts data

2. Which pandas function would you use to make a DataFrame longer by turning columns into rows?

easy

A. groupby()

B. pivot()

C. melt()

D. concat()

Solution

1. Step 1: Recall the function to make data longer

   The melt() function converts columns into rows, making data longer.

2. Step 2: Differentiate from pivot()

   pivot() does the opposite by making data wider, so it is not correct here.

3. Final Answer:

   melt() -> Option C

4. Quick Check:

   Longer data = melt() [OK]

Hint: Remember: melt makes data longer, pivot makes it wider [OK]

Common Mistakes:

• Choosing pivot() instead of melt()
• Confusing groupby() with reshaping
• Using concat() which combines dataframes, not reshapes

3. What is the output of this code?

import pandas as pd
df = pd.DataFrame({
  'Name': ['Anna', 'Bob'],
  'Math': [90, 80],
  'Science': [85, 95]
})
melted = df.melt(id_vars='Name', var_name='Subject', value_name='Score')
print(melted)

medium

A. An error because melt() needs a list for id_vars.

B. A DataFrame with columns Name, Math, Science unchanged.

C. A DataFrame with columns Name, Subject, Score but only 2 rows.

D. A DataFrame with columns Name, Subject, Score and 4 rows showing each subject score per student.

Solution

1. Step 1: Understand melt() with id_vars

   The melt() keeps 'Name' fixed and turns 'Math' and 'Science' columns into rows.

2. Step 2: Count resulting rows and columns

   Each student has 2 subjects, so 2 students x 2 subjects = 4 rows with columns Name, Subject, Score.

3. Final Answer:

   A DataFrame with columns Name, Subject, Score and 4 rows showing each subject score per student. -> Option D

4. Quick Check:

   melt() expands columns to rows [OK]

Hint: melt with id_vars keeps those columns fixed [OK]

Common Mistakes:

• Thinking melt() keeps original shape
• Believing id_vars must be a list, not a string
• Expecting fewer rows than subjects x students

4. You try to reshape data using pivot() but get an error: "ValueError: Index contains duplicate entries, cannot reshape". What is the likely cause?

medium

A. pivot() requires numeric columns only.

B. The DataFrame has duplicate index values preventing pivot.

C. pivot() needs a list for columns parameter.

D. pivot() cannot be used on DataFrames with missing values.

Solution

1. Step 1: Understand pivot() requirements

   pivot() needs unique index/column pairs to reshape data without duplicates.

2. Step 2: Identify cause of duplicate entries error

   If duplicates exist in the index or columns used, pivot() cannot reshape and throws this error.

3. Final Answer:

   The DataFrame has duplicate index values preventing pivot. -> Option B

4. Quick Check:

   Duplicate entries block pivot() [OK]

Hint: Check for duplicates before pivoting [OK]

Common Mistakes:

• Assuming pivot() only works with numeric data
• Thinking pivot() needs list for columns always
• Believing missing values cause this error

5. You have sales data with columns: Store, Month, and Sales. You want a table showing each store as a row and each month as a column with sales values. Which steps correctly reshape the data?

hard

A. Use pivot(index='Store', columns='Month', values='Sales') to make data wider.

B. Use melt() to make data longer, then groupby() to sum sales.

C. Use concat() to join Store and Month columns.

D. Use drop_duplicates() to remove repeated sales before pivoting.

Solution

1. Step 1: Identify desired shape

   You want stores as rows and months as columns, which means making data wider.

2. Step 2: Choose correct function for widening

   pivot() reshapes data from long to wide format using specified index, columns, and values.

3. Final Answer:

   Use pivot(index='Store', columns='Month', values='Sales') to make data wider. -> Option A

4. Quick Check:

   Wide table = pivot() [OK]

Hint: pivot() makes wide tables from long data [OK]

Common Mistakes:

• Using melt() when wide format is needed
• Trying concat() to reshape data
• Dropping duplicates unnecessarily before pivot