Data validation checks in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of data validation checks in pandas?

Data validation checks help ensure that the data is clean, accurate, and meets expected rules before analysis. They catch errors or inconsistencies early.

beginner

How can you check for missing values in a pandas DataFrame?

Use the isnull() or isna() methods to find missing values. For example, df.isnull().sum() shows the count of missing values per column.

intermediate

What pandas method helps to check if all values in a column meet a condition?

The all() method can be used after a condition. For example, (df['age'] > 0).all() checks if all ages are positive.

beginner

How do you find duplicate rows in a pandas DataFrame?

Use df.duplicated() to get a boolean Series marking duplicates. To see duplicates, use df[df.duplicated()].

intermediate

Explain how to validate data types of columns in pandas.

Use df.dtypes to see data types of each column. You can check if a column has the expected type, e.g., df['price'].dtype == 'float64'.

Which pandas method helps identify missing values in a DataFrame?

A isnull()

B drop_duplicates()

C astype()

D head()

The isnull() method returns a DataFrame of booleans indicating missing values.

How do you check if all values in a column 'age' are greater than zero?

A (df['age'] > 0).all()

B df['age'] > 0

C df['age'].sum() > 0

D df['age'].isnull()

Using (df['age'] > 0).all() returns True only if every value in 'age' is greater than zero.

What does df.duplicated() return?

A Rows with missing values

B Boolean Series marking duplicate rows

C Count of unique rows

D Data types of columns

df.duplicated() returns a boolean Series where True marks duplicate rows.

Which method shows the data types of each column in a DataFrame?

A df.info()

B df.describe()

C df.dtypes

D df.head()

df.dtypes returns a Series with the data type of each column.

To count missing values per column, which code is correct?

A df.duplicated()

B df.dropna()

C df.fillna(0)

D df.isnull().sum()

df.isnull().sum() counts missing values in each column.

Describe how you would perform basic data validation checks on a new dataset using pandas.

Think about common data issues like missing data, duplicates, and wrong types.

Explain why data validation checks are important before analyzing data.

Consider what happens if you analyze bad data.