Why Exploratory data analysis in ML Python? - Purpose & Use Cases

The Big Idea

What if you could unlock hidden stories in your data in minutes instead of hours?

The Scenario

Imagine you have a huge spreadsheet full of sales data. You want to understand trends, spot mistakes, and find interesting patterns. Doing this by scrolling through rows and columns manually feels like searching for a needle in a haystack.

The Problem

Manually checking data is slow and tiring. You might miss important details or make mistakes. It's hard to see the big picture or spot hidden problems just by looking at numbers one by one.

The Solution

Exploratory data analysis (EDA) uses simple tools and visuals to quickly summarize and understand data. It helps you find patterns, spot errors, and get insights without endless scrolling or guesswork.

Before vs After

✗ Before

print(data[0:10])  # Just print first 10 rows
# Manually scan for patterns or errors

✓ After

print(data.describe())
data.plot(kind='hist')  # Quick summary and visualization

What It Enables

EDA lets you confidently explore data to discover stories and guide smart decisions fast.

Real Life Example

A marketing team uses EDA to find which customer groups buy most often and what products are popular, helping them plan better campaigns.

Key Takeaways

Manual data checks are slow and error-prone.

EDA uses simple summaries and visuals to reveal data insights quickly.

It helps make smarter decisions by understanding data clearly.

Practice

1. What is the main goal of exploratory data analysis (EDA) in machine learning?

easy

A. To train the machine learning model

B. To understand the main features and quality of the data

C. To deploy the model to production

D. To write the final report without looking at data

Solution

1. Step 1: Understand the purpose of EDA

   EDA is used to explore and understand the data before modeling.

2. Step 2: Differentiate EDA from other steps

   Training and deployment happen after EDA, so they are not the main goal of EDA.

3. Final Answer:

   To understand the main features and quality of the data -> Option B

4. Quick Check:

   EDA = Understand data [OK]

Hint: EDA means exploring data, not modeling or deploying [OK]

Common Mistakes:

• Confusing EDA with model training
• Thinking EDA is for deployment
• Skipping data understanding step

2. Which Python library is commonly used for basic exploratory data analysis tasks like viewing data samples and summary statistics?

easy

A. matplotlib

B. tensorflow

C. pandas

D. scikit-learn

Solution

1. Step 1: Identify libraries for EDA

   Pandas provides functions like head() and describe() for data samples and summary stats.

2. Step 2: Exclude other libraries

   Matplotlib is for plotting, scikit-learn for modeling, tensorflow for deep learning, not basic EDA.

3. Final Answer:

   pandas -> Option C

4. Quick Check:

   pandas = data samples and stats [OK]

Hint: Use pandas for quick data overview and stats [OK]

Common Mistakes:

• Using matplotlib for data samples
• Confusing modeling libraries with EDA
• Trying tensorflow for simple stats

3. Given the following Python code using pandas, what will print(df.describe()) output?

import pandas as pd
data = {'age': [25, 30, 22, 40, 28], 'score': [88, 92, 85, 95, 90]}
df = pd.DataFrame(data)
print(df.describe())

medium

A. Summary statistics including count, mean, std, min, max for 'age' and 'score'

B. The first 5 rows of the dataframe

C. A plot showing distribution of 'age' and 'score'

D. An error because describe() needs parameters

Solution

1. Step 1: Understand what describe() does

   It returns summary statistics like count, mean, std, min, max for numeric columns.

2. Step 2: Check the code output

   Since 'age' and 'score' are numeric, describe() will show stats for both columns.

3. Final Answer:

   Summary statistics including count, mean, std, min, max for 'age' and 'score' -> Option A

4. Quick Check:

   describe() = summary stats [OK]

Hint: describe() shows stats, head() shows rows [OK]

Common Mistakes:

• Thinking describe() shows data rows
• Expecting a plot from describe()
• Assuming describe() needs arguments

4. You run this code but get an error:

import pandas as pd
data = {'age': [25, 30, None, 40], 'score': [88, 92, 85, 95]}
df = pd.DataFrame(data)
print(df.mean())

What is the likely cause and fix?

medium

A. Error due to None value; fix by using df.mean(skipna=True)

B. Error due to missing import; fix by importing numpy

C. Error due to wrong data type; fix by converting all to strings

D. No error; code runs fine as is

Solution

1. Step 1: Check how pandas handles None in numeric columns

   Pandas treats None as NaN and by default mean() skips NaN values.

2. Step 2: Verify if code runs without error

   The code runs fine and calculates mean ignoring None.

3. Final Answer:

   No error; code runs fine as is -> Option D

4. Quick Check:

   pandas mean() skips None by default [OK]

Hint: pandas mean() ignores None/NaN by default [OK]

Common Mistakes:

• Assuming None causes error in mean()
• Thinking numpy import is needed here
• Trying to convert numbers to strings

5. You have a dataset with columns age, income, and purchased (Yes/No). Which EDA step helps you find if income affects purchased?

hard

A. Plotting boxplots of income grouped by purchased categories

B. Calculating mean age of all customers

C. Checking for missing values in age column

D. Sorting data by income in ascending order

Solution

1. Step 1: Identify relationship to explore

   We want to see if income differs between those who purchased and those who did not.

2. Step 2: Choose appropriate EDA method

   Boxplots grouped by purchased categories show income distribution differences clearly.

3. Final Answer:

   Plotting boxplots of income grouped by purchased categories -> Option A

4. Quick Check:

   Boxplots show group differences [OK]

Hint: Use grouped boxplots to compare numeric by category [OK]

Common Mistakes:

• Ignoring grouping by purchased
• Checking unrelated columns
• Just sorting without visualization