Why transformation reshapes data for analysis in Data Analysis Python - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why reshape data before analysis?

Why do data scientists often reshape data before analyzing it?

A To organize data into a format that makes analysis easier and more meaningful

B To convert numerical data into text format

C To encrypt data for security reasons

D To reduce the size of the dataset by removing rows

💡 Hint

Think about how changing the shape of data can help reveal patterns.

Reshaping data helps organize it so that analysis tools can work better and insights become clearer.

❓ data_output

intermediate

Output of pivot operation on sales data

Given this sales data, what is the output after pivoting by 'Month' with 'Product' as columns and 'Sales' as values?

import pandas as pd
data = pd.DataFrame({
    'Month': ['Jan', 'Jan', 'Feb', 'Feb'],
    'Product': ['A', 'B', 'A', 'B'],
    'Sales': [100, 150, 200, 250]
})
pivoted = data.pivot(index='Month', columns='Product', values='Sales')
print(pivoted)

A

Month  Product  Sales
Jan    A        100
Jan    B        150
Feb    A        200
Feb    B        250

B

Month  Product  Sales
0      Jan        A    100
1      Jan        B    150
2      Feb        A    200
3      Feb        B    250

C

      A    B
Month             
Jan    100  150
Feb    200  250

D

Product  Month  Sales
A        Jan    100
B        Jan    150
A        Feb    200
B        Feb    250

💡 Hint

Pivot changes rows into columns based on unique values.

The pivot method rearranges data so each product becomes a column with sales values for each month as rows.

🔧 Debug

advanced

Identify error in melting data

What error does this code produce when trying to reshape data using melt?

import pandas as pd
data = pd.DataFrame({
    'ID': [1, 2],
    'Math': [90, 80],
    'Science': [85, 95]
})
melted = pd.melt(data, id_vars=['ID'], value_vars=['Math', 'History'])
print(melted)

A TypeError: id_vars must be a list

B KeyError: 'History'

C ValueError: value_vars cannot be empty

D No error, prints melted DataFrame

💡 Hint

Check if all columns in value_vars exist in the DataFrame.

The column 'History' does not exist in the DataFrame, causing a KeyError.

❓ visualization

advanced

Visualizing reshaped data with a heatmap

Which option shows the correct heatmap code to visualize sales data after pivoting by 'Month' and 'Product'?

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
data = pd.DataFrame({
    'Month': ['Jan', 'Jan', 'Feb', 'Feb'],
    'Product': ['A', 'B', 'A', 'B'],
    'Sales': [100, 150, 200, 250]
})
pivoted = data.pivot(index='Month', columns='Product', values='Sales')

A

sns.heatmap(pivoted, annot=True)
plt.show()

B

plt.plot(pivoted)
plt.show()

C

sns.barplot(data=pivoted)
plt.show()

D

sns.heatmap(data, annot=True)
plt.show()

💡 Hint

Heatmaps require a 2D matrix-like input.

The pivoted DataFrame is a matrix suitable for heatmap visualization showing sales per product per month.

🚀 Application

expert

Choosing the right reshape method for time series analysis

You have a dataset with daily temperature readings for multiple cities in wide format (each city is a column). You want to analyze temperature trends over time for each city. Which reshaping method is best to prepare the data?

A Use groupby to aggregate temperatures by city without reshaping

B Use pivot to convert the data into a wider format with cities as rows

C Use concat to join city columns into a single column without changing the format

D Use melt to convert the wide format into long format with columns for date, city, and temperature

💡 Hint

Long format is better for time series analysis with multiple groups.

Melt reshapes wide data into long format, making it easier to analyze trends per city over time.

Practice

1. Why do we transform data before analysis?

easy

A. To reshape data for easier comparison and visualization

B. To delete important information

C. To make data harder to understand

D. To increase the file size unnecessarily

Solution

1. Step 1: Understand the purpose of data transformation

   Data transformation changes the shape or format of data to make it easier to analyze.

2. Step 2: Identify the benefit of reshaping data

   Reshaping helps in comparing values side-by-side and creating clear visualizations.

3. Final Answer:

   To reshape data for easier comparison and visualization -> Option A

4. Quick Check:

   Transformation = Easier analysis [OK]

Hint: Think: transformation makes data easier to use [OK]

Common Mistakes:

• Confusing transformation with deleting data
• Believing transformation makes data more complex
• Thinking transformation increases file size

2. Which of the following is the correct way to use pandas melt to reshape data?

easy

A. pd.melt(df, id_vars=['score'], value_vars=['id'])

B. pd.melt(df, id_vars=['id'], value_vars=['score'])

C. pd.melt(df, id_vars='score', value_vars='id')

D. pd.melt(df, id_vars=['score', 'id'])

Solution

1. Step 1: Recall pandas melt syntax

   pandas melt requires id_vars as columns to keep and value_vars as columns to unpivot.

2. Step 2: Check pd.melt(df, id_vars=['id'], value_vars=['score'])

   pd.melt(df, id_vars=['id'], value_vars=['score']) correctly uses id_vars=['id'] and value_vars=['score'], matching the syntax.

3. Final Answer:

   pd.melt(df, id_vars=['id'], value_vars=['score']) -> Option B

4. Quick Check:

   Correct melt syntax = pd.melt(df, id_vars=['id'], value_vars=['score']) [OK]

Hint: id_vars are columns to keep, value_vars to unpivot [OK]

Common Mistakes:

• Swapping id_vars and value_vars
• Passing strings instead of lists
• Omitting value_vars parameter

3. Given this DataFrame:

import pandas as pd
df = pd.DataFrame({
  'id': [1, 2],
  'math': [90, 80],
  'english': [85, 88]
})
reshaped = pd.melt(df, id_vars=['id'], value_vars=['math', 'english'])
print(reshaped)

What is the output?

medium

A. A DataFrame with columns: id, math, english and 2 rows

B. A DataFrame with columns: variable, value and 2 rows

C. A DataFrame with columns: id, variable, value and 4 rows

D. An error because of wrong melt usage

Solution

1. Step 1: Understand the original DataFrame

   It has 2 rows and 3 columns: id, math, english.

2. Step 2: Apply pd.melt with id_vars=['id'] and value_vars=['math', 'english']

   This reshapes math and english columns into two rows per id, creating 4 rows total with columns id, variable, value.

3. Final Answer:

   A DataFrame with columns: id, variable, value and 4 rows -> Option C

4. Quick Check:

   melt reshapes wide to long with 4 rows [OK]

Hint: Melt stacks columns into rows, increasing row count [OK]

Common Mistakes:

• Expecting same number of rows after melt
• Confusing columns after melt
• Thinking melt causes error here

4. What is wrong with this code snippet?

import pandas as pd
df = pd.DataFrame({'id':[1,2],'score':[10,20]})
reshaped = pd.melt(df, id_vars='id', value_vars='score')
print(reshaped)

medium

A. print statement is missing parentheses

B. DataFrame creation syntax is incorrect

C. pd.melt does not exist in pandas

D. id_vars and value_vars should be lists, not strings

Solution

1. Step 1: Check pandas melt parameters

   id_vars and value_vars expect list of column names, not single strings.

2. Step 2: Identify error cause

   Passing 'id' and 'score' as strings causes pandas to treat each character as a column, leading to error or wrong output.

3. Final Answer:

   id_vars and value_vars should be lists, not strings -> Option D

4. Quick Check:

   Use lists for id_vars and value_vars [OK]

Hint: Always pass lists for id_vars and value_vars [OK]

Common Mistakes:

• Passing single strings instead of lists
• Misunderstanding melt parameters
• Assuming melt is not in pandas

5. You have sales data in wide format:

import pandas as pd
df = pd.DataFrame({
  'store': ['A', 'B'],
  'Jan': [100, 150],
  'Feb': [120, 160],
  'Mar': [130, 170]
})

You want to analyze monthly sales trends easily. Which transformation helps and why?

hard

A. Use pd.melt to convert months into one column for easier trend analysis

B. Use df.sort_values() to reorder rows by store

C. Use df.drop() to remove month columns

D. Use df.groupby('store') without reshaping

Solution

1. Step 1: Identify the data format and goal

   Data is wide with months as columns; goal is to analyze trends over months.

2. Step 2: Choose transformation to reshape data

   Using pd.melt converts month columns into a single 'month' column and a 'sales' column, making trend analysis easier.

3. Step 3: Explain why other options don't fit

   Sorting or dropping columns doesn't reshape data for trend analysis; grouping without reshaping keeps data wide.

4. Final Answer:

   Use pd.melt to convert months into one column for easier trend analysis -> Option A

5. Quick Check:

   Wide to long format helps trend analysis [OK]

Hint: Melt wide months into one column for trends [OK]

Common Mistakes:

• Trying to analyze trends without reshaping
• Dropping useful columns
• Confusing sorting with reshaping