Why data exploration matters in Pandas - Challenge Your Understanding

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this data summary code?

Given the DataFrame below, what will be the output of the df.describe() method?

Pandas

import pandas as pd

data = {'age': [25, 30, 22, 40, 28], 'income': [50000, 60000, 45000, 80000, 52000]}
df = pd.DataFrame(data)
print(df.describe())

A

       age        income
count   5.0      5.000000
mean   29.0  57400.000000
std     7.0  13928.388277
min    22.0  45000.000000
25%    25.0  50000.000000
50%    28.0  52000.000000
75%    30.0  60000.000000
max    40.0  80000.000000

B

       age        income
count   5.0      5.000000
mean   29.0  57400.000000
std     7.5  13928.388277
min    22.0  45000.000000
25%    24.0  48000.000000
50%    28.0  52000.000000
75%    30.0  60000.000000
max    40.0  80000.000000

C

       age        income
count   5.0      5.000000
mean   29.0  57400.000000
std     7.5  13928.388277
min    22.0  45000.000000
25%    25.0  50000.000000
50%    28.0  52000.000000
75%    30.0  60000.000000
max    40.0  80000.000000

D

       age        income
count   5.0      5.000000
mean   29.0  57400.000000
std     7.5  13928.388277
min    22.0  45000.000000
25%    25.0  51000.000000
50%    28.0  52000.000000
75%    30.0  60000.000000
max    40.0  80000.000000

💡 Hint

Look carefully at the 25% percentile values for both columns.

The describe() method calculates count, mean, std, min, 25%, 50%, 75%, and max for numeric columns. The 25% percentile for age is 25.0 and for income is 50000.0 based on the data.

❓ data_output

intermediate

How many missing values are in the DataFrame?

Given the DataFrame below, what is the output of df.isnull().sum()?

Pandas

import pandas as pd
import numpy as np

data = {'name': ['Alice', 'Bob', 'Charlie', 'David'], 'age': [25, np.nan, 30, 22], 'income': [50000, 60000, np.nan, 45000]}
df = pd.DataFrame(data)
print(df.isnull().sum())

A

name      0
age       1
income    0
dtype: int64

B

name      0
age       1
income    1
dtype: int64

C

name      0
age       2
income    1
dtype: int64

D

name      1
age       1
income    1
dtype: int64

💡 Hint

Check which columns have missing values and count them.

The 'age' column has one missing value (NaN), and 'income' also has one missing value. The 'name' column has no missing values.

❓ visualization

advanced

Which plot best shows the distribution of a numeric column?

You want to understand how the values in the 'age' column are spread out. Which plot below is best for this?

Pandas

import pandas as pd
import matplotlib.pyplot as plt

data = {'age': [22, 25, 25, 30, 35, 40, 40, 40, 45, 50]}
df = pd.DataFrame(data)

# Option A
plt.hist(df['age'], bins=5)
plt.title('Histogram of Age')
plt.show()

# Option B
plt.scatter(range(len(df)), df['age'])
plt.title('Scatter plot of Age')
plt.show()

# Option C
plt.bar(df['age'].value_counts().index, df['age'].value_counts())
plt.title('Bar chart of Age counts')
plt.show()

# Option D
plt.boxplot(df['age'])
plt.title('Boxplot of Age')
plt.show()

A Bar chart - shows counts of each unique age value

B Scatter plot - shows individual age values by index

C Boxplot - shows median, quartiles, and outliers of age

D Histogram - shows frequency distribution of age values

💡 Hint

Think about which plot shows how often each age range occurs.

A histogram groups data into bins and shows how many values fall into each range, which is ideal for understanding distribution.

🧠 Conceptual

advanced

Why is data exploration important before modeling?

Which reason below best explains why exploring data is a crucial step before building a predictive model?

A To identify patterns, detect errors, and understand data quality before modeling

B To avoid visualizing data and rely only on automated tools

C To skip cleaning and directly use raw data for faster results

D To immediately train the model without checking data

💡 Hint

Think about what problems might happen if you don't understand your data first.

Exploring data helps find mistakes, missing values, and patterns that affect model accuracy and reliability.

🔧 Debug

expert

What error does this code raise during data exploration?

Consider this code snippet. What error will it raise when run?

Pandas

import pandas as pd

data = {'age': [25, 30, 22], 'income': [50000, 60000]}
df = pd.DataFrame(data)
print(df.describe())

A ValueError: All arrays must be of the same length

B KeyError: 'income'

C TypeError: unsupported operand type(s) for +: 'int' and 'str'

D No error, prints summary statistics

💡 Hint

Check if all columns have the same number of values.

The 'income' list has only 2 values while 'age' has 3, causing a ValueError when creating the DataFrame.

Practice

1. Why is data exploration important before starting analysis with pandas?

easy

A. It automatically cleans the data for you.

B. It helps understand the data and find issues early.

C. It creates visualizations without any code.

D. It speeds up the computer's processing power.

Solution

1. Step 1: Understand the purpose of data exploration

   Data exploration is done to get to know the data, its structure, and any problems it may have.

2. Step 2: Identify what data exploration does not do

   It does not clean data automatically, create visualizations by itself, or affect computer speed.

3. Final Answer:

   It helps understand the data and find issues early. -> Option B

4. Quick Check:

   Data exploration = Understand data [OK]

Hint: Remember: Explore first to know your data well [OK]

Common Mistakes:

• Thinking exploration cleans data automatically
• Confusing exploration with visualization
• Believing it improves computer speed

2. Which pandas command shows the first 5 rows of a DataFrame named df?

easy

A. df.head()

B. df.info()

C. df.describe()

D. df.tail()

Solution

1. Step 1: Recall pandas commands for viewing data

   head() shows the first rows, tail() shows last rows, describe() summarizes statistics, info() shows data info.

2. Step 2: Match command to requirement

   To see first 5 rows, df.head() is correct.

3. Final Answer:

   df.head() -> Option A

4. Quick Check:

   First rows = head() [OK]

Hint: head() = first rows, tail() = last rows [OK]

Common Mistakes:

• Using tail() to see first rows
• Confusing describe() with head()
• Using info() to view data rows

3. Given the code:

import pandas as pd
data = {'age': [25, 30, 22, 40], 'score': [88, 92, 85, 95]}
df = pd.DataFrame(data)
print(df.describe())

What will be the output?

medium

A. Summary statistics including count, mean, std, min, max for each column

B. The first 5 rows of the DataFrame

C. Information about data types and memory usage

D. An error because describe() needs numeric data only

Solution

1. Step 1: Understand what df.describe() does

   It calculates summary statistics like count, mean, std, min, max for numeric columns.

2. Step 2: Check the data types in df

   Both 'age' and 'score' are numeric lists, so describe() works and outputs stats.

3. Final Answer:

   Summary statistics including count, mean, std, min, max for each column -> Option A

4. Quick Check:

   describe() = summary stats [OK]

Hint: describe() summarizes numeric columns quickly [OK]

Common Mistakes:

• Thinking describe() shows rows
• Confusing describe() with info()
• Assuming describe() errors on numeric data

4. What is wrong with this code snippet for exploring a DataFrame df?

print(df.head(5))
print(df.describe)
print(df.info())

medium

A. df.info() cannot be printed directly.

B. df.head(5) should be df.tail(5) to see first rows.

C. df.describe is missing parentheses to call the method.

D. There is no error; the code runs fine.

Solution

1. Step 1: Check each line for syntax correctness

   df.head(5) is correct to show first 5 rows. df.describe without parentheses is a method reference, not a call. df.info() can be printed.

2. Step 2: Identify the error

   Missing parentheses on df.describe means it won't execute and print stats.

3. Final Answer:

   df.describe is missing parentheses to call the method. -> Option C

4. Quick Check:

   Method calls need () [OK]

Hint: Always add () to call pandas methods [OK]

Common Mistakes:

• Forgetting parentheses on methods
• Mixing head() and tail() usage
• Thinking info() can't be printed

5. You have a DataFrame df with missing values and mixed data types. Which sequence of pandas commands helps you explore and prepare for cleaning?

hard

A. df.describe() to fix missing values, then df.info() to visualize data.

B. df.head() to see last rows, then df.tail() for summary stats.

C. df.tail() to check data types, then df.head() to drop missing values.

D. df.info() to check data types and missing values, then df.describe() for stats.

Solution

1. Step 1: Use df.info() to check data types and missing values

   This command shows column types and counts of non-null values, helping spot missing data.

2. Step 2: Use df.describe() to get summary statistics

   This helps understand numeric data distribution and spot anomalies before cleaning.

3. Final Answer:

   df.info() to check data types and missing values, then df.describe() for stats. -> Option D

4. Quick Check:

   info() then describe() = explore before cleaning [OK]

Hint: Start with info(), then describe() to explore data [OK]

Common Mistakes:

• Using tail() to see first rows
• Thinking describe() fixes missing values
• Confusing order of commands