Datetime type in Pandas - Mini Project: Build & Apply

Working with Datetime Type in pandas

📖 Scenario: You work in a small company that tracks employee attendance. You have a list of dates when employees checked in. You want to analyze these dates using pandas to understand attendance patterns.

🎯 Goal: You will create a pandas DataFrame with date strings, convert these strings to datetime type, filter dates after a certain day, and finally print the filtered dates.

📋 What You'll Learn

Create a pandas DataFrame with a column of date strings

Create a variable with a date threshold as a datetime object

Filter the DataFrame to keep only dates after the threshold

Print the filtered dates

💡 Why This Matters

🌍 Real World

Datetime data is everywhere: tracking attendance, sales dates, sensor logs, and more. Being able to convert and filter dates helps analyze trends over time.

💼 Career

Data scientists and analysts often work with time series data. Knowing how to handle datetime types in pandas is a key skill for cleaning and analyzing such data.

1

Create a pandas DataFrame with date strings

Import pandas as pd. Create a DataFrame called df with one column named 'check_in' containing these exact date strings: '2024-01-10', '2024-02-15', '2024-03-20', '2024-04-25', '2024-05-30'.

Pandas

# Import pandas as pd
# Create DataFrame df with column 'check_in' and the given date strings
# Your code here

Hint

Use pd.DataFrame and pass a dictionary with key 'check_in' and the list of date strings as value.

2

Create a datetime threshold variable

Create a variable called threshold_date by converting the string '2024-03-01' to a pandas datetime object using pd.to_datetime().

Pandas

import pandas as pd

df = pd.DataFrame({
    'check_in': ['2024-01-10', '2024-02-15', '2024-03-20', '2024-04-25', '2024-05-30']
})

# Create threshold_date by converting '2024-03-01' to datetime
# Your code here

Hint

Use pd.to_datetime('2024-03-01') to convert the string to datetime.

3

Convert 'check_in' column to datetime and filter dates

Convert the 'check_in' column in df to datetime type using pd.to_datetime(). Then create a new DataFrame called filtered_df that contains only rows where 'check_in' is after threshold_date.

Pandas

import pandas as pd

df = pd.DataFrame({
    'check_in': ['2024-01-10', '2024-02-15', '2024-03-20', '2024-04-25', '2024-05-30']
})

threshold_date = pd.to_datetime('2024-03-01')

# Convert 'check_in' column to datetime
# Create filtered_df with rows where 'check_in' > threshold_date
# Your code here

Hint

Use df['check_in'] = pd.to_datetime(df['check_in']) to convert the column. Then filter with df[df['check_in'] > threshold_date].

4

Print the filtered dates

Print the filtered_df DataFrame to show only the dates after threshold_date.

Pandas

import pandas as pd

df = pd.DataFrame({
    'check_in': ['2024-01-10', '2024-02-15', '2024-03-20', '2024-04-25', '2024-05-30']
})

threshold_date = pd.to_datetime('2024-03-01')

df['check_in'] = pd.to_datetime(df['check_in'])
filtered_df = df[df['check_in'] > threshold_date]

# Print filtered_df
# Your code here

Hint

Use print(filtered_df) to display the filtered DataFrame.

Practice

1. What is the main purpose of the Datetime type in pandas?

easy

A. To store images and multimedia files

B. To store only text data

C. To store dates and times in a format pandas can understand and work with

D. To store numerical data like integers and floats

Solution

1. Step 1: Understand what Datetime type stores

   Datetime type is designed to hold date and time information in a structured way.

2. Step 2: Compare with other data types

   Unlike text or numbers, Datetime allows easy date calculations and filtering.

3. Final Answer:

   To store dates and times in a format pandas can understand and work with -> Option C

4. Quick Check:

   Datetime type = stores dates/times [OK]

Hint: Datetime type is for dates and times only [OK]

Common Mistakes:

• Thinking Datetime stores text or numbers
• Confusing Datetime with string type
• Assuming Datetime stores images

2. Which of the following is the correct way to convert a string column df['date_str'] to datetime type in pandas?

easy

A. df['date_str'].astype('datetime')

B. pd.to_datetime(df['date_str'])

C. df['date_str'].to_datetime()

D. pd.datetime(df['date_str'])

Solution

1. Step 1: Recall the pandas function for datetime conversion

   The correct function is pd.to_datetime() to convert strings to datetime.

2. Step 2: Check the options for correct syntax

   Only pd.to_datetime(df['date_str']) uses pd.to_datetime() correctly with the column.

3. Final Answer:

   pd.to_datetime(df['date_str']) -> Option B

4. Quick Check:

   Use pd.to_datetime() for conversion [OK]

Hint: Use pd.to_datetime() to convert strings to datetime [OK]

Common Mistakes:

• Using astype('datetime') which is invalid
• Calling to_datetime() as a method on Series
• Using pd.datetime which does not exist

3. What will be the output of this code?

import pandas as pd
s = pd.Series(['2023-01-01', '2023-02-15', 'not a date'])
s_dt = pd.to_datetime(s, errors='coerce')
s_dt.isna().sum()

medium

A. 0

B. Raises an error

C. 2

D. 1

Solution

1. Step 1: Convert strings to datetime with errors='coerce'

   The invalid string 'not a date' becomes NaT (missing datetime).

2. Step 2: Count missing datetime values

   Only one value is invalid, so isna().sum() returns 1.

3. Final Answer:

   1 -> Option D

4. Quick Check:

   Invalid dates become NaT, count is 1 [OK]

Hint: errors='coerce' makes invalid dates NaT [OK]

Common Mistakes:

• Expecting error instead of NaT
• Counting all as valid dates
• Confusing NaT with zero

4. The following code throws an error. What is the mistake?

import pandas as pd
s = pd.Series(['2023-01-01', '2023-02-15'])
s_dt = s.to_datetime()

medium

A. Series object has no method to_datetime()

B. pd.to_datetime() requires a list, not Series

C. Missing import for datetime module

D. Strings are not in correct date format

Solution

1. Step 1: Check method usage on Series

   Series does not have a method called to_datetime().

2. Step 2: Correct method to convert Series to datetime

   Use pd.to_datetime(s) instead of s.to_datetime().

3. Final Answer:

   Series object has no method to_datetime() -> Option A

4. Quick Check:

   Use pd.to_datetime(), not s.to_datetime() [OK]

Hint: Use pd.to_datetime(), not Series.to_datetime() [OK]

Common Mistakes:

• Trying to call to_datetime() on Series
• Assuming pd.to_datetime() needs list input
• Thinking import datetime is required here

5. You have a DataFrame df with a column 'date_str' containing dates as strings in mixed formats like '2023-01-01', '01/02/2023', and 'March 3, 2023'. How do you convert this column to datetime type reliably?

hard

A. Use pd.to_datetime(df['date_str'], errors='coerce')

B. Use pd.to_datetime(df['date_str'], errors='coerce', dayfirst=true)

C. Use df['date_str'].astype('datetime64')

D. Use pd.to_datetime(df['date_str'], errors='ignore')

Solution

1. Step 1: Understand mixed date formats and errors

   Mixed formats can cause parsing errors; using errors='coerce' converts unparseable dates to NaT safely.

2. Step 2: Evaluate options for handling mixed formats

   Use pd.to_datetime(df['date_str'], errors='coerce') uses pd.to_datetime() with errors='coerce', which is reliable for mixed formats without forcing dayfirst.

3. Final Answer:

   Use pd.to_datetime(df['date_str'], errors='coerce') -> Option A

4. Quick Check:

   errors='coerce' handles mixed formats safely [OK]

Hint: Use errors='coerce' to handle mixed date formats safely [OK]

Common Mistakes:

• Using errors='ignore' which keeps strings unconverted
• Using astype('datetime64') which fails on mixed formats
• Setting dayfirst=true incorrectly for all dates