Exploratory data analysis workflow in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the first step in an Exploratory Data Analysis (EDA) workflow?

The first step is to load and inspect the data. This means reading the data into a program and looking at its structure, like rows, columns, and data types.

beginner

Why do we check for missing values during EDA?

Checking for missing values helps us understand if some data points are incomplete. This is important because missing data can affect analysis and may need to be handled specially.

beginner

What does 'summary statistics' mean in EDA?

Summary statistics are simple numbers that describe data, like mean (average), median (middle value), minimum, maximum, and standard deviation (spread). They give a quick idea about the data.

beginner

How can visualizations help in EDA?

Visualizations like histograms, scatter plots, and box plots help us see patterns, trends, and outliers in data. They make it easier to understand data than just looking at numbers.

beginner

What is the purpose of checking data types in EDA?

Checking data types ensures that each column has the correct kind of data (numbers, text, dates). This helps avoid errors and guides how to analyze or transform the data.

What is the main goal of Exploratory Data Analysis?

A To build a final machine learning model

B To understand the main features and patterns in the data

C To clean the data by removing all rows

D To write documentation for the dataset

EDA helps us understand the data's main features and patterns before further analysis.

Which pandas function shows the first few rows of a DataFrame?

A df.head()

B df.tail()

C df.describe()

D df.info()

df.head() shows the first 5 rows by default, helping to inspect data quickly.

Which method helps find missing values in a pandas DataFrame?

A df.sum()

B df.mean()

C df.isnull()

D df.dropna()

df.isnull() returns True for missing values, helping identify them.

What does df.describe() provide?

A Summary statistics for numeric columns

B List of column names

C Data types of columns

D Number of missing values

df.describe() gives count, mean, std, min, max, and quartiles for numeric data.

Which plot is best to see the distribution of a single numeric variable?

A Scatter plot

B Bar chart

C Line plot

D Histogram

Histograms show how data values are spread across ranges.

Describe the main steps you would follow in an Exploratory Data Analysis workflow using pandas.

Think about how you get to know a new dataset step by step.

Explain why visualizations are important in exploratory data analysis.

Consider how pictures help us understand complex information.

Practice

1. Which step is usually done first in an exploratory data analysis workflow?

easy

A. Viewing the first few rows of the dataset

B. Creating complex machine learning models

C. Removing all missing values immediately

D. Generating final reports without checking data

Solution

1. Step 1: Understand the initial data

   Exploratory data analysis starts by looking at the first few rows to get a feel for the data structure and values.

2. Step 2: Avoid premature actions

   Jumping to modeling or removing missing data without understanding can cause mistakes.

3. Final Answer:

   Viewing the first few rows of the dataset -> Option A

4. Quick Check:

   First step = View data sample [OK]

Hint: Always start by looking at data samples [OK]

Common Mistakes:

• Starting with modeling before understanding data
• Removing missing values without checking their impact
• Ignoring data overview and jumping to conclusions

2. Which pandas command correctly shows summary statistics of numeric columns in a DataFrame df?

easy

A. df.head()

B. df.info()

C. df.describe()

D. df.sample()

Solution

1. Step 1: Identify summary statistics function

   df.describe() returns count, mean, std, min, max, and quartiles for numeric columns.

2. Step 2: Differentiate other commands

   df.head() shows first rows, df.info() shows data types, and df.sample() shows random rows.

3. Final Answer:

   df.describe() -> Option C

4. Quick Check:

   Summary stats = df.describe() [OK]

Hint: Use df.describe() for quick numeric summaries [OK]

Common Mistakes:

• Using df.head() to get statistics
• Confusing df.info() with statistical summary
• Expecting df.sample() to summarize data

3. Given the DataFrame df below, what is the output of df.isnull().sum()?

import pandas as pd
data = {'A': [1, 2, None], 'B': [None, 2, 3], 'C': [1, 2, 3]}
df = pd.DataFrame(data)

medium

A. {'A': 1, 'B': 1, 'C': 0}

B. {'A': 0, 'B': 0, 'C': 0}

C. {'A': 1, 'B': 2, 'C': 1}

D. {'A': 2, 'B': 1, 'C': 0}

Solution

1. Step 1: Check missing values per column

   Column 'A' has one None, 'B' has one None, 'C' has none.

2. Step 2: Use isnull().sum() to count missing

   This method counts missing values per column, so output is {'A': 1, 'B': 1, 'C': 0}.

3. Final Answer:

   {'A': 1, 'B': 1, 'C': 0} -> Option A

4. Quick Check:

   Missing counts = {'A':1,'B':1,'C':0} [OK]

Hint: Count missing with df.isnull().sum() [OK]

Common Mistakes:

• Counting total missing instead of per column
• Confusing None with zero or empty string
• Misreading which columns have missing values

4. You run df.info() but get an error: AttributeError: 'DataFrame' object has no attribute 'infoo'. What is the likely cause?

medium

A. The DataFrame is empty

B. You typed the method name incorrectly

C. The pandas library is not imported

D. The DataFrame has no columns

Solution

1. Step 1: Check the error message

   The error says 'infoo' attribute not found, which suggests a typo in method name.

2. Step 2: Confirm correct method name

   The correct method is df.info() with one 'o'.

3. Final Answer:

   You typed the method name incorrectly -> Option B

4. Quick Check:

   Typo in method name = error [OK]

Hint: Check spelling of method names carefully [OK]

Common Mistakes:

• Assuming empty DataFrame causes this error
• Forgetting to import pandas
• Thinking DataFrame must have columns to use info()

5. You have a DataFrame df with columns ['Age', 'Salary', 'Department']. You want to find the average salary per department but only for employees older than 30. Which pandas workflow is correct?

hard

A. df.groupby('Department')['Salary'].mean()

B. df['Salary'].mean().groupby('Department')[df['Age'] > 30]

C. df.groupby('Department').mean()['Salary'][df['Age'] > 30]

D. df[df['Age'] > 30].groupby('Department')['Salary'].mean()

Solution

1. Step 1: Filter rows where Age > 30

   Use df[df['Age'] > 30] to keep only employees older than 30.

2. Step 2: Group filtered data by Department and calculate mean Salary

   Apply .groupby('Department')['Salary'].mean() on filtered data to get average salary per department.

3. Final Answer:

   df[df['Age'] > 30].groupby('Department')['Salary'].mean() -> Option D

4. Quick Check:

   Filter then group and mean = df[df['Age'] > 30].groupby('Department')['Salary'].mean() [OK]

Hint: Filter first, then group and aggregate [OK]

Common Mistakes:

• Grouping before filtering data
• Trying to filter after aggregation
• Using mean() without grouping