dtypes and data type checking in Pandas - Step-by-Step Execution

Concept Flow - dtypes and data type checking

Create DataFrame

↓

Check dtypes attribute

↓

Get data types of each column

↓

Use dtype-checking functions

↓

Decide actions based on types

Start with a DataFrame, check its dtypes attribute to see each column's data type, then use functions to check or convert types.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({
    'A': [1, 2, 3],
    'B': [1.1, 2.2, 3.3],
    'C': ['x', 'y', 'z']
})
print(df.dtypes)

Create a DataFrame with integer, float, and string columns, then print their data types.

Execution Table

Step	Action	Code/Expression	Result/Output
1	Create DataFrame	df = pd.DataFrame({'A':[1,2,3],'B':[1.1,2.2,3.3],'C':['x','y','z']})	DataFrame with 3 columns A(int), B(float), C(object)
2	Check dtypes attribute	df.dtypes	A int64 B float64 C object dtype: object
3	Check if column A is integer type	pd.api.types.is_integer_dtype(df['A'])	True
4	Check if column B is float type	pd.api.types.is_float_dtype(df['B'])	True
5	Check if column C is string/object type	pd.api.types.is_object_dtype(df['C'])	True
6	Convert column C to category type	df['C'] = df['C'].astype('category')	Column C dtype changed to category
7	Check new dtypes	df.dtypes	A int64 B float64 C category dtype: object
8	Exit	No more steps	End of execution

💡 All columns checked and one converted; execution ends.

Variable Tracker

Variable	Start	After Step 1	After Step 6	Final
df	undefined	DataFrame with A:int64, B:float64, C:object	DataFrame with A:int64, B:float64, C:category	Same as after Step 6

Key Moments - 3 Insights

Why does column C show dtype 'object' initially?

Because pandas uses 'object' dtype for string columns by default, as shown in execution_table step 2.

How do we check if a column is of a specific type?

Use pandas functions like is_integer_dtype or is_float_dtype as in steps 3 and 4 of the execution_table.

What happens when we convert a column to 'category' dtype?

The column's dtype changes to 'category', which can save memory and speed up operations, as seen in step 6 and confirmed in step 7.

Visual Quiz - 3 Questions

Look at the execution_table at step 2, what is the dtype of column B?

A float64

B int64

C object

D category

💡 Hint

Check the 'Result/Output' column in step 2 of execution_table.

At which step does column C change its dtype to 'category'?

A Step 5

B Step 6

C Step 3

D Step 7

💡 Hint

Look for the action mentioning 'astype' conversion in execution_table.

If we check is_integer_dtype on column B at step 4, what is the result?

A True

B Error

C False

D None

💡 Hint

Refer to the 'Result/Output' in step 4 of execution_table.

Concept Snapshot

pandas DataFrame columns have dtypes showing data type.
Use df.dtypes to see all column types.
Check types with pd.api.types functions.
Convert types with astype() method.
Category dtype saves memory for repeated strings.

Full Transcript

We start by creating a pandas DataFrame with three columns: A with integers, B with floats, and C with strings. We then check the data types of each column using the df.dtypes attribute, which shows A as int64, B as float64, and C as object (pandas uses object for strings). Next, we use pandas type-checking functions like is_integer_dtype and is_float_dtype to confirm the types of columns A and B. We also check if column C is of object type. Then, we convert column C to the 'category' data type using astype('category'), which can improve performance and reduce memory. Finally, we verify the new data types with df.dtypes again, seeing that column C is now category. This process helps us understand and manage data types in pandas DataFrames effectively.

Practice

1. What does the dtypes attribute in a pandas DataFrame tell you?

easy

A. The type of data stored in each column

B. The number of rows in the DataFrame

C. The names of the columns

D. The total memory used by the DataFrame

Solution

1. Step 1: Understand what dtypes represents

   The dtypes attribute shows the data type of each column in a DataFrame, such as integer, float, or object (text).

2. Step 2: Compare with other DataFrame attributes

   Other attributes like shape give row and column counts, and columns give column names, but dtypes specifically shows data types.

3. Final Answer:

   The type of data stored in each column -> Option A

4. Quick Check:

   dtypes = data types per column [OK]

Hint: Remember: dtypes = data types of columns [OK]

Common Mistakes:

• Confusing dtypes with number of rows
• Thinking dtypes shows column names
• Assuming dtypes shows memory usage

2. Which of the following is the correct way to check the data type of a single column named 'age' in a pandas DataFrame df?

easy

A. df.dtypes('age')

B. df['age'].dtype

C. df.dtype['age']

D. df.get_dtype('age')

Solution

1. Step 1: Recall syntax for accessing column dtype

   To get the data type of a single column, use df['column_name'].dtype.

2. Step 2: Check other options for correctness

   Options B, C, and D use incorrect syntax or methods that do not exist in pandas.

3. Final Answer:

   df['age'].dtype -> Option B

4. Quick Check:

   Single column dtype = df['column'].dtype [OK]

Hint: Use df['column'].dtype to check one column's type [OK]

Common Mistakes:

• Using parentheses with dtypes like a function
• Trying to access dtype as a dictionary
• Using non-existent methods like get_dtype

3. Given the following code:

import pandas as pd
data = {'name': ['Anna', 'Bob'], 'score': [85, 92]}
df = pd.DataFrame(data)
print(df.dtypes)

What will be the output?

medium

A. name object
score object
dtype: object

B. name int64
score int64
dtype: object

C. name float64
score int64
dtype: object

D. name object
score int64
dtype: object

Solution

1. Step 1: Identify data types of each column

   The 'name' column contains text, so pandas assigns it the 'object' dtype. The 'score' column contains integers, so it gets 'int64'.

2. Step 2: Understand pandas output format

   Printing df.dtypes shows each column name with its dtype, followed by 'dtype: object' indicating the Series type.

3. Final Answer:

   name object score int64 dtype: object -> Option D

4. Quick Check:

   Text = object, integers = int64 [OK]

Hint: Text columns show as object, numbers as int64 or float64 [OK]

Common Mistakes:

• Assuming text columns show as string dtype
• Confusing int64 with float64 for integers
• Expecting score to be object dtype

4. You run this code:

import pandas as pd
data = {'A': [1, 2], 'B': ['x', 'y']}
df = pd.DataFrame(data)
print(df['A'].dtypes())

What error will you get?

medium

A. AttributeError: 'Series' object has no attribute 'dtypes'

B. TypeError: 'dtype' object is not callable

C. SyntaxError: invalid syntax

D. No error, prints int64

Solution

1. Step 1: Identify the issue with the code

   A pandas Series (df['A']) has a .dtype attribute (singular), not .dtypes (plural, which is for DataFrames).

2. Step 2: Understand the error caused

   Accessing .dtypes on a Series raises AttributeError: 'Series' object has no attribute 'dtypes'. The parentheses would cause further issues if the attribute existed.

3. Final Answer:

   AttributeError: 'Series' object has no attribute 'dtypes' -> Option A

4. Quick Check:

   Series: .dtype | DataFrame: .dtypes [OK]

Hint: Series: .dtype | DataFrame: .dtypes [OK]

Common Mistakes:

• Confusing Series .dtype with DataFrame .dtypes
• Adding parentheses after attributes
• Expecting no error from wrong attribute

5. You have a DataFrame df with columns ['id', 'value', 'date']. The value column is stored as object dtype but contains numbers as strings. Which pandas command will convert value to numeric dtype safely, turning invalid parsing into NaN?

hard

A. df['value'] = df['value'].astype('float64')

B. df['value'] = df['value'].astype(int)

C. df['value'] = pd.to_numeric(df['value'], errors='coerce')

D. df['value'] = df['value'].convert_dtypes()

Solution

1. Step 1: Understand the problem with object dtype

   The 'value' column has numbers stored as strings, so converting directly with astype(int) or astype(float64) may fail if there are invalid strings.

2. Step 2: Use pd.to_numeric with errors='coerce'

   This function converts strings to numbers and replaces invalid parsing with NaN, preventing errors.

3. Step 3: Check other options

   df['value'] = df['value'].convert_dtypes() uses convert_dtypes() which infers types but does not coerce invalid strings safely.

4. Final Answer:

   df['value'] = pd.to_numeric(df['value'], errors='coerce') -> Option C

5. Quick Check:

   Use pd.to_numeric with errors='coerce' for safe conversion [OK]

Hint: Use pd.to_numeric with errors='coerce' to convert safely [OK]

Common Mistakes:

• Using astype(int) on strings causing errors
• Assuming convert_dtypes fixes string numbers
• Not handling invalid strings causing crashes