dtypes and data type checking in Pandas

Introduction

We use dtypes to know what kind of data is stored in each column of a table. This helps us work with data correctly and avoid mistakes.

When you want to see if a column has numbers or words.

Before doing math on data, to make sure the data is numeric.

To check if dates are stored as dates or just text.

When cleaning data, to find columns with wrong data types.

To convert data from one type to another for analysis.

Syntax

Pandas

DataFrame.dtypes

# or for a single column
Series.dtype

dtypes shows the data type of each column in a DataFrame.

For a single column (Series), use dtype to get its type.

Examples

This shows the data type of each column in the DataFrame.

Pandas

import pandas as pd

data = {'age': [25, 30, 22], 'name': ['Alice', 'Bob', 'Carol']}
df = pd.DataFrame(data)
print(df.dtypes)

This shows the data type of the 'age' column only.

Pandas

print(df['age'].dtype)

This shows the data type of the 'name' column only.

Pandas

print(df['name'].dtype)

Sample Program

This program creates a table with numbers, text, booleans, and dates. It then prints the data type of each column and the data type of the 'date' column.

Pandas

import pandas as pd

# Create a DataFrame with different types of data
data = {
    'temperature': [22.5, 23.0, 21.8],
    'city': ['Paris', 'London', 'Berlin'],
    'rain': [False, True, False],
    'date': pd.to_datetime(['2023-06-01', '2023-06-02', '2023-06-03'])
}
df = pd.DataFrame(data)

# Check data types of all columns
print(df.dtypes)

# Check data type of one column
print(df['date'].dtype)

Important Notes

In pandas, object usually means text (strings).

Data types help pandas decide what operations can be done on the data.

Use pd.to_datetime() to convert text to date type.

Summary

dtypes tell you the kind of data in each column.

Check dtypes before analysis to avoid errors.

You can check the type of the whole table or just one column.

Practice

1. What does the dtypes attribute in a pandas DataFrame tell you?

easy

A. The type of data stored in each column

B. The number of rows in the DataFrame

C. The names of the columns

D. The total memory used by the DataFrame

Solution

1. Step 1: Understand what dtypes represents

   The dtypes attribute shows the data type of each column in a DataFrame, such as integer, float, or object (text).

2. Step 2: Compare with other DataFrame attributes

   Other attributes like shape give row and column counts, and columns give column names, but dtypes specifically shows data types.

3. Final Answer:

   The type of data stored in each column -> Option A

4. Quick Check:

   dtypes = data types per column [OK]

Hint: Remember: dtypes = data types of columns [OK]

Common Mistakes:

• Confusing dtypes with number of rows
• Thinking dtypes shows column names
• Assuming dtypes shows memory usage

2. Which of the following is the correct way to check the data type of a single column named 'age' in a pandas DataFrame df?

easy

A. df.dtypes('age')

B. df['age'].dtype

C. df.dtype['age']

D. df.get_dtype('age')

Solution

1. Step 1: Recall syntax for accessing column dtype

   To get the data type of a single column, use df['column_name'].dtype.

2. Step 2: Check other options for correctness

   Options B, C, and D use incorrect syntax or methods that do not exist in pandas.

3. Final Answer:

   df['age'].dtype -> Option B

4. Quick Check:

   Single column dtype = df['column'].dtype [OK]

Hint: Use df['column'].dtype to check one column's type [OK]

Common Mistakes:

• Using parentheses with dtypes like a function
• Trying to access dtype as a dictionary
• Using non-existent methods like get_dtype

3. Given the following code:

import pandas as pd
data = {'name': ['Anna', 'Bob'], 'score': [85, 92]}
df = pd.DataFrame(data)
print(df.dtypes)

What will be the output?

medium

A. name object
score object
dtype: object

B. name int64
score int64
dtype: object

C. name float64
score int64
dtype: object

D. name object
score int64
dtype: object

Solution

1. Step 1: Identify data types of each column

   The 'name' column contains text, so pandas assigns it the 'object' dtype. The 'score' column contains integers, so it gets 'int64'.

2. Step 2: Understand pandas output format

   Printing df.dtypes shows each column name with its dtype, followed by 'dtype: object' indicating the Series type.

3. Final Answer:

   name object score int64 dtype: object -> Option D

4. Quick Check:

   Text = object, integers = int64 [OK]

Hint: Text columns show as object, numbers as int64 or float64 [OK]

Common Mistakes:

• Assuming text columns show as string dtype
• Confusing int64 with float64 for integers
• Expecting score to be object dtype

4. You run this code:

import pandas as pd
data = {'A': [1, 2], 'B': ['x', 'y']}
df = pd.DataFrame(data)
print(df['A'].dtypes())

What error will you get?

medium

A. AttributeError: 'Series' object has no attribute 'dtypes'

B. TypeError: 'dtype' object is not callable

C. SyntaxError: invalid syntax

D. No error, prints int64

Solution

1. Step 1: Identify the issue with the code

   A pandas Series (df['A']) has a .dtype attribute (singular), not .dtypes (plural, which is for DataFrames).

2. Step 2: Understand the error caused

   Accessing .dtypes on a Series raises AttributeError: 'Series' object has no attribute 'dtypes'. The parentheses would cause further issues if the attribute existed.

3. Final Answer:

   AttributeError: 'Series' object has no attribute 'dtypes' -> Option A

4. Quick Check:

   Series: .dtype | DataFrame: .dtypes [OK]

Hint: Series: .dtype | DataFrame: .dtypes [OK]

Common Mistakes:

• Confusing Series .dtype with DataFrame .dtypes
• Adding parentheses after attributes
• Expecting no error from wrong attribute

5. You have a DataFrame df with columns ['id', 'value', 'date']. The value column is stored as object dtype but contains numbers as strings. Which pandas command will convert value to numeric dtype safely, turning invalid parsing into NaN?

hard

A. df['value'] = df['value'].astype('float64')

B. df['value'] = df['value'].astype(int)

C. df['value'] = pd.to_numeric(df['value'], errors='coerce')

D. df['value'] = df['value'].convert_dtypes()

Solution

1. Step 1: Understand the problem with object dtype

   The 'value' column has numbers stored as strings, so converting directly with astype(int) or astype(float64) may fail if there are invalid strings.

2. Step 2: Use pd.to_numeric with errors='coerce'

   This function converts strings to numbers and replaces invalid parsing with NaN, preventing errors.

3. Step 3: Check other options

   df['value'] = df['value'].convert_dtypes() uses convert_dtypes() which infers types but does not coerce invalid strings safely.

4. Final Answer:

   df['value'] = pd.to_numeric(df['value'], errors='coerce') -> Option C

5. Quick Check:

   Use pd.to_numeric with errors='coerce' for safe conversion [OK]

Hint: Use pd.to_numeric with errors='coerce' to convert safely [OK]

Common Mistakes:

• Using astype(int) on strings causing errors
• Assuming convert_dtypes fixes string numbers
• Not handling invalid strings causing crashes