Exploratory data analysis in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Exploratory Data Analysis (EDA)?

EDA is the process of examining data sets to summarize their main characteristics, often using visual methods. It helps understand the data before applying any modeling.

beginner

Name three common techniques used in EDA.

Common techniques include:
1. Summary statistics (mean, median, mode)
2. Data visualization (histograms, scatter plots)
3. Checking for missing values and outliers.

beginner

Why is it important to check for missing values during EDA?

Missing values can affect analysis and model accuracy. Identifying them early helps decide how to handle them, like filling in or removing those data points.

intermediate

What does a box plot show in EDA?

A box plot shows the distribution of data through its quartiles, highlights the median, and identifies outliers. It helps quickly see spread and skewness.

intermediate

How can correlation analysis help in EDA?

Correlation analysis shows how two variables move together. It helps find relationships or dependencies, which can guide feature selection for modeling.

Which of the following is NOT a typical step in Exploratory Data Analysis?

A Building a final predictive model

B Calculating summary statistics

C Checking for missing values

D Visualizing data distributions

Building a final predictive model is done after EDA, not during it.

What does a histogram help you understand in EDA?

A Relationship between two variables

B Exact values of data points

C Frequency distribution of a single variable

D Missing data locations

Histograms show how often values occur in intervals for one variable.

Why is it useful to identify outliers during EDA?

A They can distort analysis and may need special handling

B They always improve model accuracy

C They represent the average data points

D They are always errors and should be deleted

Outliers can affect results and may need to be treated carefully, not always deleted.

Which plot is best to check the relationship between two numeric variables?

A Box plot

B Bar chart

C Pie chart

D Scatter plot

Scatter plots show how two numeric variables relate to each other.

What does a correlation coefficient close to 0 indicate?

A Strong negative relationship

B No linear relationship

C Strong positive relationship

D Perfect linear relationship

A correlation near 0 means no linear relationship between variables.

Describe the main goals and steps involved in Exploratory Data Analysis.

Think about how you would get to know a new dataset before using it.

Explain how visualizations help in Exploratory Data Analysis and name two common types.

Imagine looking at pictures of your data to understand it better.

Practice

1. What is the main goal of exploratory data analysis (EDA) in machine learning?

easy

A. To train the machine learning model

B. To understand the main features and quality of the data

C. To deploy the model to production

D. To write the final report without looking at data

Solution

1. Step 1: Understand the purpose of EDA

   EDA is used to explore and understand the data before modeling.

2. Step 2: Differentiate EDA from other steps

   Training and deployment happen after EDA, so they are not the main goal of EDA.

3. Final Answer:

   To understand the main features and quality of the data -> Option B

4. Quick Check:

   EDA = Understand data [OK]

Hint: EDA means exploring data, not modeling or deploying [OK]

Common Mistakes:

• Confusing EDA with model training
• Thinking EDA is for deployment
• Skipping data understanding step

2. Which Python library is commonly used for basic exploratory data analysis tasks like viewing data samples and summary statistics?

easy

A. matplotlib

B. tensorflow

C. pandas

D. scikit-learn

Solution

1. Step 1: Identify libraries for EDA

   Pandas provides functions like head() and describe() for data samples and summary stats.

2. Step 2: Exclude other libraries

   Matplotlib is for plotting, scikit-learn for modeling, tensorflow for deep learning, not basic EDA.

3. Final Answer:

   pandas -> Option C

4. Quick Check:

   pandas = data samples and stats [OK]

Hint: Use pandas for quick data overview and stats [OK]

Common Mistakes:

• Using matplotlib for data samples
• Confusing modeling libraries with EDA
• Trying tensorflow for simple stats

3. Given the following Python code using pandas, what will print(df.describe()) output?

import pandas as pd
data = {'age': [25, 30, 22, 40, 28], 'score': [88, 92, 85, 95, 90]}
df = pd.DataFrame(data)
print(df.describe())

medium

A. Summary statistics including count, mean, std, min, max for 'age' and 'score'

B. The first 5 rows of the dataframe

C. A plot showing distribution of 'age' and 'score'

D. An error because describe() needs parameters

Solution

1. Step 1: Understand what describe() does

   It returns summary statistics like count, mean, std, min, max for numeric columns.

2. Step 2: Check the code output

   Since 'age' and 'score' are numeric, describe() will show stats for both columns.

3. Final Answer:

   Summary statistics including count, mean, std, min, max for 'age' and 'score' -> Option A

4. Quick Check:

   describe() = summary stats [OK]

Hint: describe() shows stats, head() shows rows [OK]

Common Mistakes:

• Thinking describe() shows data rows
• Expecting a plot from describe()
• Assuming describe() needs arguments

4. You run this code but get an error:

import pandas as pd
data = {'age': [25, 30, None, 40], 'score': [88, 92, 85, 95]}
df = pd.DataFrame(data)
print(df.mean())

What is the likely cause and fix?

medium

A. Error due to None value; fix by using df.mean(skipna=True)

B. Error due to missing import; fix by importing numpy

C. Error due to wrong data type; fix by converting all to strings

D. No error; code runs fine as is

Solution

1. Step 1: Check how pandas handles None in numeric columns

   Pandas treats None as NaN and by default mean() skips NaN values.

2. Step 2: Verify if code runs without error

   The code runs fine and calculates mean ignoring None.

3. Final Answer:

   No error; code runs fine as is -> Option D

4. Quick Check:

   pandas mean() skips None by default [OK]

Hint: pandas mean() ignores None/NaN by default [OK]

Common Mistakes:

• Assuming None causes error in mean()
• Thinking numpy import is needed here
• Trying to convert numbers to strings

5. You have a dataset with columns age, income, and purchased (Yes/No). Which EDA step helps you find if income affects purchased?

hard

A. Plotting boxplots of income grouped by purchased categories

B. Calculating mean age of all customers

C. Checking for missing values in age column

D. Sorting data by income in ascending order

Solution

1. Step 1: Identify relationship to explore

   We want to see if income differs between those who purchased and those who did not.

2. Step 2: Choose appropriate EDA method

   Boxplots grouped by purchased categories show income distribution differences clearly.

3. Final Answer:

   Plotting boxplots of income grouped by purchased categories -> Option A

4. Quick Check:

   Boxplots show group differences [OK]

Hint: Use grouped boxplots to compare numeric by category [OK]

Common Mistakes:

• Ignoring grouping by purchased
• Checking unrelated columns
• Just sorting without visualization