Selecting columns in Data Analysis Python - Time & Space Complexity

Time Complexity: Selecting columns

O(m)

Understanding Time Complexity

When we select columns from a dataset, we want to know how the time to do this changes as the dataset grows.

We ask: How does the work increase when the number of rows or columns grows?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

def select_columns(df, cols):
    return df[cols]

# Example usage:
# df is a DataFrame with many rows and columns
# cols is a list of column names to select

This code returns a new DataFrame with only the columns listed in cols.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Processing each selected column.
• How many times: Once for each column in cols.

How Execution Grows With Input

As the number of selected columns grows, the work grows linearly because each selected column must be incorporated into the new DataFrame.

Input Size (columns)	Approx. Operations
10	10 times the work per column
100	100 times the work per column
1000	1000 times the work per column

Pattern observation: The work grows directly with the number of columns.

Final Time Complexity

Time Complexity: O(m)

This means the time to select columns grows in a straight line as the number of columns increases.

Common Mistake

[X] Wrong: "Selecting columns is instant and does not depend on the number of columns."

[OK] Correct: Even though we only pick columns, the system still processes each selected column to build the new DataFrame, so time grows with columns.

Interview Connect

Understanding how data selection scales helps you explain your code choices clearly and shows you know what happens behind the scenes.

Self-Check

"What if we select only one column instead of many? How would the time complexity change?"

Practice

1. Which of the following is the correct way to select a single column named 'Age' from a DataFrame df?

easy

A. df[['Age']]

B. df.select('Age')

C. df.Age()

D. df['Age']

Solution

1. Step 1: Understand single column selection syntax

   To select one column from a DataFrame, use df['column_name'].

2. Step 2: Check each option

   df['Age'] uses df['Age'], which is correct. df[['Age']] returns a DataFrame, not a Series. df.Age() uses parentheses incorrectly. df.select('Age') is not a valid pandas method.

3. Final Answer:

   df['Age'] -> Option D

4. Quick Check:

   Single column selection = df['Age'] [OK]

Hint: Use single brackets for one column, double for multiple [OK]

Common Mistakes:

• Using double brackets for single column returns DataFrame, not Series
• Using dot notation with parentheses like a function
• Trying non-existent methods like select()

2. Which syntax correctly selects multiple columns 'Name' and 'Salary' from DataFrame df?

easy

A. df['Name', 'Salary']

B. df[['Name', 'Salary']]

C. df.Name, df.Salary

D. df.select(['Name', 'Salary'])

Solution

1. Step 1: Recall syntax for selecting multiple columns

   Use double brackets with a list of column names: df[['col1', 'col2']].

2. Step 2: Evaluate options

   df[['Name', 'Salary']] matches the correct syntax. df['Name', 'Salary'] is invalid syntax. df.Name, df.Salary selects columns separately, not together. df.select(['Name', 'Salary']) is not a pandas method.

3. Final Answer:

   df[['Name', 'Salary']] -> Option B

4. Quick Check:

   Multiple columns = df[['Name', 'Salary']] [OK]

Hint: Use double brackets with list for multiple columns [OK]

Common Mistakes:

• Using single brackets with multiple column names separated by commas
• Trying to select multiple columns with dot notation
• Using non-existent select() method

3. Given the DataFrame df below:

   Name  Age  Salary
0  John   28   50000
1  Anna   22   60000
2  Mike   32   55000

What is the output of df[['Name', 'Age']]?

medium

A. Name Age
0 John 28
1 Anna 22
2 Mike 32

B. Name Age Salary
0 John 28 50000
1 Anna 22 60000
2 Mike 32 55000

C. 0 John
1 Anna
2 Mike
Name: Name, dtype: object

D. 28
22
32

Solution

1. Step 1: Understand what df[['Name', 'Age']] does

   This selects only the columns 'Name' and 'Age' from the DataFrame, returning a smaller DataFrame with those columns.

2. Step 2: Match output to options

      Name  Age
   0  John   28
   1  Anna   22
   2  Mike   32

   shows only 'Name' and 'Age' columns with all rows, which matches the expected output.

      Name  Age  Salary
   0  John   28   50000
   1  Anna   22   60000
   2  Mike   32   55000

   includes 'Salary' which is not selected.

   0    John
   1    Anna
   2    Mike
   Name: Name, dtype: object

   shows only the 'Name' column as a Series.

   28
   22
   32

   shows only ages as a list, which is incorrect.

3. Final Answer:

   DataFrame with columns 'Name' and 'Age' only -> Option A

4. Quick Check:

   Selected columns = 'Name' and 'Age' only [OK]

Hint: Double brackets return DataFrame with selected columns [OK]

Common Mistakes:

• Expecting single brackets to return multiple columns
• Confusing Series output with DataFrame output
• Including unselected columns in output

4. What is wrong with this code to select the column 'Score' from DataFrame df?

df.Score()

medium

A. Parentheses cause an error; dot notation should be without ()

B. Column names cannot have uppercase letters

C. Column name must be in double brackets

D. Dot notation cannot be used to select columns

Solution

1. Step 1: Understand dot notation for column selection

   Dot notation selects columns as attributes without parentheses, like df.Score.

2. Step 2: Identify error in code

   Using parentheses () after Score tries to call it like a function, causing an error.

3. Final Answer:

   Parentheses cause an error; dot notation should be without () -> Option A

4. Quick Check:

   Dot notation = df.Score without parentheses [OK]

Hint: Dot notation is attribute access, no parentheses [OK]

Common Mistakes:

• Adding parentheses after column name with dot notation
• Thinking dot notation works for all column names
• Using double brackets with dot notation

5. You have a DataFrame df with columns 'A', 'B', 'C', and 'D'. You want to create a new DataFrame df_new that contains only columns 'B' and 'D' but in reverse order ('D' first, then 'B'). Which code correctly does this?

hard

A. df_new = df['D', 'B']

B. df_new = df[['B', 'D']]

C. df_new = df[['D', 'B']]

D. df_new = df[['B', 'D']].reindex(columns=['B', 'D'])

Solution

1. Step 1: Understand column selection order

   The order of columns in the list inside double brackets determines the order in the new DataFrame.

2. Step 2: Check each option

   df_new = df[['D', 'B']] selects columns 'D' then 'B', which matches the requirement. df_new = df[['B', 'D']] selects 'B' then 'D', wrong order. df_new = df['D', 'B'] is invalid tuple syntax inside single brackets. df_new = df[['B', 'D']].reindex(columns=['B', 'D']) reindexes to the same order 'B' then 'D', incorrect.

3. Final Answer:

   df_new = df[['D', 'B']] -> Option C

4. Quick Check:

   Order columns by list inside double brackets [OK]

Hint: Order columns in list to set output order [OK]

Common Mistakes:

• Selecting columns in wrong order
• Using reindex but specifying the original order
• Using tuple inside single brackets for multiple columns