Long to wide format conversion in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of converting data from long to wide format?

Converting data from long to wide format reshapes the data so that each unique value in a specified column becomes a new column. This makes it easier to compare values side by side, like turning a list of measurements into a table with separate columns for each measurement type.

beginner

Which pandas function is commonly used to convert data from long to wide format?

The pandas function pivot() or pivot_table() is used to convert data from long to wide format. They rearrange the data by setting one column as the new columns and another as the values.

intermediate

What is the difference between pivot() and pivot_table() in pandas?

pivot() requires unique index/column pairs and will fail if duplicates exist. pivot_table() can handle duplicates by aggregating values using a function like mean or sum.

beginner

Given a DataFrame with columns ['Date', 'City', 'Temperature'] in long format, how would you convert it to wide format with cities as columns?

Use df.pivot(index='Date', columns='City', values='Temperature'). This makes each city a column and rows indexed by date, showing temperatures side by side.

intermediate

Why might you need to reset the index after pivoting a DataFrame?

Pivoting often sets one or more columns as the index. Resetting the index with reset_index() turns the index back into regular columns, which can be easier to work with or export.

Which pandas function would you use to convert a DataFrame from long to wide format when there are duplicate entries for the same index and column?

A melt()

B pivot()

C pivot_table()

D groupby()

pivot_table() can handle duplicates by aggregating values, while pivot() cannot.

In a long format DataFrame, which column typically becomes the new columns in wide format after pivoting?

A Value column

B Column specified in <code>columns=</code> parameter

C Index column

D Any numeric column

The column specified in the columns= parameter becomes the new columns in the wide format.

What happens if you try to use pivot() on data with duplicate index-column pairs?

A It raises an error

B It converts duplicates to NaN

C It ignores duplicates

D It automatically aggregates duplicates

pivot() requires unique index-column pairs and will raise an error if duplicates exist.

After pivoting, why might you want to use reset_index()?

A To convert index back to columns

B To remove duplicate rows

C To sort the DataFrame

D To rename columns

reset_index() converts the index back into regular columns for easier access.

Which of these is NOT a typical use case for converting long to wide format?

A Comparing values side by side

B Preparing data for plotting

C Creating summary tables

D Reducing the number of columns

Converting long to wide format usually increases the number of columns, not reduces them.

Explain how to convert a DataFrame from long to wide format using pandas. Include the key function and parameters.

Think about which columns become rows, columns, and values.

Describe the difference between pivot() and pivot_table() in pandas and when to use each.

Consider how duplicates in data affect the choice.

Practice

1. What does converting data from long to wide format using pivot in pandas mainly achieve?

easy

A. It removes duplicate rows from the dataset.

B. It sorts the data alphabetically by column names.

C. It turns row values into columns to compare data side by side.

D. It aggregates data by summing all values.

Solution

1. Step 1: Understand long vs wide format

   Long format has repeated rows for categories; wide format spreads categories into columns.

2. Step 2: Role of pivot in pandas

   Pivot reshapes data by turning unique row values into columns for side-by-side comparison.

3. Final Answer:

   It turns row values into columns to compare data side by side. -> Option C

4. Quick Check:

   Long to wide = row values become columns [OK]

Hint: Pivot turns row values into columns for easy comparison [OK]

Common Mistakes:

• Thinking pivot sorts data alphabetically
• Confusing pivot with aggregation functions
• Assuming pivot removes duplicates automatically

2. Which of the following is the correct syntax to convert a DataFrame df from long to wide format using pivot with index='Date', columns='City', and values='Temperature'?

easy

A. df.pivot(columns='Date', index='City', values='Temperature')

B. df.pivot(index='Date', columns='City', values='Temperature')

C. df.pivot(index='Temperature', columns='Date', values='City')

D. df.pivot(values='Date', index='City', columns='Temperature')

Solution

1. Step 1: Identify correct parameter order

   Pivot requires index (rows), columns, and values parameters in any order but with correct names.

2. Step 2: Match parameters to data roles

   Index='Date' means rows by date, columns='City' means columns by city, values='Temperature' fills data.

3. Final Answer:

   df.pivot(index='Date', columns='City', values='Temperature') -> Option B

4. Quick Check:

   Correct pivot syntax = df.pivot(index='Date', columns='City', values='Temperature') [OK]

Hint: Use pivot(index=rows, columns=cols, values=data) [OK]

Common Mistakes:

• Swapping index and columns parameters
• Using wrong column names in parameters
• Passing parameters in wrong order causing errors

3. Given the DataFrame df below:

   Date    City  Temperature
0  2023-01-01  NY     30
1  2023-01-01  LA     60
2  2023-01-02  NY     28
3  2023-01-02  LA     62

What is the output of df.pivot(index='Date', columns='City', values='Temperature')?

medium

A. City LA NY
Date
2023-01-01 60 30
2023-01-02 62 28

B. City NY LA
Date
2023-01-01 30 60
2023-01-02 28 62

C. Date 2023-01-01 2023-01-02
City
LA 60 62
NY 30 28

D. City LA NY
Date
2023-01-01 30 60
2023-01-02 28 62

Solution

1. Step 1: Understand pivot parameters

   Index='Date' means rows are dates; columns='City' means columns are cities; values='Temperature' fills data.

2. Step 2: Arrange data accordingly

   For 2023-01-01, LA=60, NY=30; for 2023-01-02, LA=62, NY=28; columns sorted alphabetically by default.

3. Final Answer:

   City LA NY Date 2023-01-01 60 30 2023-01-02 62 28 -> Option A

4. Quick Check:

   Pivot output matches City LA NY Date 2023-01-01 60 30 2023-01-02 62 28 [OK]

Hint: Pivot columns sorted alphabetically by default [OK]

Common Mistakes:

• Mixing up row and column labels
• Assuming city order is NY then LA
• Confusing index and columns roles

4. You try to run df.pivot(index='Date', columns='City', values='Temperature') but get a ValueError: "Index contains duplicate entries, cannot reshape". What is the most likely cause?

medium

A. The columns parameter should be a list, not a string.

B. The DataFrame is missing the Temperature column.

C. The pivot function requires numeric index values only.

D. The DataFrame has multiple rows with the same Date and City combination.

Solution

1. Step 1: Understand pivot restrictions

   Pivot requires unique pairs of index and columns to create a single value cell.

2. Step 2: Identify cause of duplicate error

   If multiple rows share the same Date and City, pivot cannot decide which value to use, causing the error.

3. Final Answer:

   The DataFrame has multiple rows with the same Date and City combination. -> Option D

4. Quick Check:

   Duplicate index-column pairs cause ValueError [OK]

Hint: Check for duplicate index-column pairs before pivot [OK]

Common Mistakes:

• Assuming missing columns cause this error
• Thinking pivot needs numeric index only
• Believing columns param must be a list

5. You have a DataFrame df with columns ['Date', 'City', 'Measurement', 'Value'] where Measurement can be 'Temperature' or 'Humidity'. How do you convert this long format into a wide format with Date as index and separate columns for each City and Measurement combination showing Value?

hard

A. Use df.pivot(index='Date', columns=['City', 'Measurement'], values='Value')

B. Use df.pivot(index=['Date', 'City'], columns='Measurement', values='Value')

C. Use df.pivot(index='Measurement', columns=['Date', 'City'], values='Value')

D. Use df.pivot(index='Date', columns='Value', values=['City', 'Measurement'])

Solution

1. Step 1: Understand multi-level columns in pivot

   Pivot allows multiple columns by passing a list to the columns parameter to create hierarchical columns.

2. Step 2: Apply correct parameters

   Index='Date' sets rows, columns=['City', 'Measurement'] creates multi-level columns, values='Value' fills data.

3. Final Answer:

   Use df.pivot(index='Date', columns=['City', 'Measurement'], values='Value') -> Option A

4. Quick Check:

   Multi-index columns = Use df.pivot(index='Date', columns=['City', 'Measurement'], values='Value') [OK]

Hint: Pass list to columns for multi-level wide format [OK]

Common Mistakes:

• Passing multiple columns to index instead of columns
• Swapping values and columns parameters
• Using wrong columns for values parameter