pivot_longer (wide to long) in R Programming - Time & Space Complexity

Time Complexity: pivot_longer (wide to long)

O(n * k)

Understanding Time Complexity

When we change data from wide to long format using pivot_longer, it takes some time to rearrange the data.

We want to know how the time needed grows as the data gets bigger.

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

library(tidyr)
data <- data.frame(
  id = 1:1000,
  score_math = rnorm(1000),
  score_english = rnorm(1000),
  score_science = rnorm(1000)
)

long_data <- pivot_longer(data, cols = starts_with("score_"),
                          names_to = "subject", values_to = "score")

This code takes a data frame with many score columns and turns it into a longer format with one score column and a subject column.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Going through each row and each selected column to reshape data.
• How many times: For each of the n rows, it processes k columns (here 3 score columns).

How Execution Grows With Input

As the number of rows (n) or columns to pivot (k) grows, the work grows by multiplying these two.

Input Size (n rows)	Approx. Operations (n * k)
10	30 (10 rows * 3 columns)
100	300 (100 * 3)
1000	3000 (1000 * 3)

Pattern observation: The time grows linearly with the number of rows and columns involved.

Final Time Complexity

Time Complexity: O(n * k)

This means the time needed grows proportionally to the number of rows times the number of columns you want to pivot.

Common Mistake

[X] Wrong: "pivot_longer only depends on the number of rows, so time grows linearly with rows only."

[OK] Correct: The function must also process each selected column for every row, so columns matter too.

Interview Connect

Understanding how data reshaping scales helps you handle bigger datasets efficiently and shows you think about performance in real tasks.

Self-Check

"What if we increased the number of columns to pivot from 3 to 10? How would the time complexity change?"

Practice

1. What does the pivot_longer() function do in R's tidyverse?

easy

A. It filters rows based on a condition.

B. It creates new columns by splitting existing ones.

C. It transforms wide data into long data by stacking columns into rows.

D. It sorts data by a specific column.

Solution

1. Step 1: Understand the purpose of pivot_longer()

   The function is designed to reshape data from wide format (many columns) to long format (fewer columns, more rows).

2. Step 2: Identify the correct description

   Among the options, only It transforms wide data into long data by stacking columns into rows. correctly describes stacking columns into rows, which is what pivot_longer does.

3. Final Answer:

   It transforms wide data into long data by stacking columns into rows. -> Option C

4. Quick Check:

   pivot_longer = wide to long [OK]

Hint: pivot_longer stacks columns into rows to tidy data [OK]

Common Mistakes:

• Confusing pivot_longer with filtering or sorting functions
• Thinking it splits columns instead of stacking
• Assuming it creates new columns instead of reshaping

2. Which of the following is the correct syntax to pivot columns Jan and Feb into longer format with new columns named Month and Sales?

easy

A. pivot_longer(data, cols = c(Jan, Feb), names_to = "Month", values_to = "Sales")

B. pivot_longer(data, columns = c(Jan, Feb), names = "Month", values = "Sales")

C. pivot_longer(data, cols = c("Jan", "Feb"), names_to = Month, values_to = Sales)

D. pivot_longer(data, cols = Jan, Feb, names_to = "Month", values_to = "Sales")

Solution

1. Step 1: Check argument names and syntax

   The correct argument names are cols, names_to, and values_to. Columns should be passed as symbols or strings inside c().

2. Step 2: Validate each option

   pivot_longer(data, cols = c(Jan, Feb), names_to = "Month", values_to = "Sales") uses correct argument names and syntax with symbols inside c(). pivot_longer(data, columns = c(Jan, Feb), names = "Month", values = "Sales") uses wrong argument names. pivot_longer(data, cols = c("Jan", "Feb"), names_to = Month, values_to = Sales) misses quotes around new column names. pivot_longer(data, cols = Jan, Feb, names_to = "Month", values_to = "Sales") incorrectly separates columns without c().

3. Final Answer:

   pivot_longer(data, cols = c(Jan, Feb), names_to = "Month", values_to = "Sales") -> Option A

4. Quick Check:

   Correct args: cols, names_to, values_to [OK]

Hint: Use cols = c(), names_to and values_to with quotes [OK]

Common Mistakes:

• Using wrong argument names like columns or names
• Forgetting quotes around new column names
• Not wrapping columns in c()

3. Given this data frame in R:

df <- data.frame(ID = 1:2, Jan = c(10, 20), Feb = c(15, 25))

What is the output of:

pivot_longer(df, cols = c(Jan, Feb), names_to = "Month", values_to = "Value")

medium

A. A data frame with columns ID, Month, and Value, where Jan and Feb values are stacked under Value.

B. A data frame with columns ID, Jan, Feb unchanged.

C. An error because cols argument is missing quotes.

D. A data frame with columns ID, Month, Value but values are all NA.

Solution

1. Step 1: Understand pivot_longer behavior on df

   The function stacks the Jan and Feb columns into two rows per ID, creating Month and Value columns.

2. Step 2: Predict the output structure

   The output will have 4 rows (2 IDs x 2 months) and columns: ID, Month (Jan or Feb), and Value (corresponding numbers).

3. Final Answer:

   A data frame with columns ID, Month, and Value, where Jan and Feb values are stacked under Value. -> Option A

4. Quick Check:

   pivot_longer stacks columns into rows [OK]

Hint: pivot_longer stacks selected columns into new key-value columns [OK]

Common Mistakes:

• Expecting original wide format unchanged
• Thinking it causes errors without quotes on cols
• Assuming values become NA

4. What is wrong with this code?

pivot_longer(df, cols = Jan, Feb, names_to = "Month", values_to = "Value")

medium

A. The function pivot_longer does not exist.

B. The cols argument should be a vector using c(), not separate arguments.

C. The values_to argument is missing.

D. The names_to argument should not be in quotes.

Solution

1. Step 1: Check the cols argument format

   The cols argument must be a vector of columns, written as cols = c(Jan, Feb). Passing them separately is invalid syntax.

2. Step 2: Verify other arguments

   names_to and values_to are correctly used with quotes. The function exists in the tidyverse package.

3. Final Answer:

   The cols argument should be a vector using c(), not separate arguments. -> Option B

4. Quick Check:

   cols = c(...) required [OK]

Hint: Wrap multiple columns in c() inside cols argument [OK]

Common Mistakes:

• Passing multiple columns without c()
• Removing quotes from names_to or values_to
• Assuming pivot_longer is not available

5. You have a data frame:

df <- data.frame(ID = 1:2, Q1_2019 = c(5, 6), Q2_2019 = c(7, 8), Q1_2020 = c(9, 10))

You want to pivot it longer so that the quarter and year are separate columns named Quarter and Year, and the values go into Score. Which code achieves this?

hard

A. pivot_longer(df, cols = c(Q1_2019, Q2_2019, Q1_2020), names_to = c("Quarter", "Year"), values_to = "Score")

B. pivot_longer(df, cols = c(Q1_2019, Q2_2019, Q1_2020), names_to = "Quarter_Year", values_to = "Score")

C. pivot_longer(df, cols = c(Q1_2019, Q2_2019, Q1_2020), names_to = c("Year", "Quarter"), names_sep = "_", values_to = "Score")

D. pivot_longer(df, cols = starts_with("Q"), names_to = c("Quarter", "Year"), names_sep = "_", values_to = "Score")

Solution

1. Step 1: Identify columns to pivot

   All columns starting with "Q" represent quarters and years, so starts_with("Q") selects them.

2. Step 2: Separate names into Quarter and Year

   Using names_to = c("Quarter", "Year") with names_sep = "_" splits the column names at underscore into two new columns.

3. Step 3: Assign values to Score

   The values_to = "Score" argument names the new value column.

4. Final Answer:

   pivot_longer(df, cols = starts_with("Q"), names_to = c("Quarter", "Year"), names_sep = "_", values_to = "Score") -> Option D

5. Quick Check:

   Use names_sep to split names_to vector [OK]

Hint: Use names_to vector and names_sep to split column names [OK]

Common Mistakes:

• Not using names_sep to split names into multiple columns
• Swapping order of Quarter and Year in names_to
• Not selecting all relevant columns with starts_with