Date feature extraction in Data Analysis Python

Introduction

Date feature extraction helps us get useful parts from dates, like year or month, to understand patterns in data better.

You want to find sales trends by month or day.

You need to group data by weekdays or weekends.

You want to see if events happen more in certain years.

You want to add new columns like hour or quarter from timestamps.

Syntax

Data Analysis Python

df['new_feature'] = df['date_column'].dt.feature_name

Use .dt to access date parts from a pandas datetime column.

Common features: year, month, day, weekday, hour, quarter.

Examples

Extracts the year from the date column.

Data Analysis Python

df['year'] = df['date'].dt.year

Extracts the month number (1-12) from the date column.

Data Analysis Python

df['month'] = df['date'].dt.month

Extracts the weekday as a number (Monday=0, Sunday=6).

Data Analysis Python

df['weekday'] = df['date'].dt.weekday

Extracts the quarter of the year (1 to 4).

Data Analysis Python

df['quarter'] = df['date'].dt.quarter

Sample Program

This code creates a table with dates, converts them to date type, then adds new columns for year, month, weekday, and quarter.

Data Analysis Python

import pandas as pd

# Create sample data with dates
data = {'date': ['2023-01-15', '2023-04-20', '2023-07-10', '2023-10-05']}
df = pd.DataFrame(data)

# Convert 'date' column to datetime type
df['date'] = pd.to_datetime(df['date'])

# Extract year, month, weekday, and quarter
 df['year'] = df['date'].dt.year
 df['month'] = df['date'].dt.month
 df['weekday'] = df['date'].dt.weekday
 df['quarter'] = df['date'].dt.quarter

print(df)

Important Notes

Make sure your date column is in datetime format before extracting features.

Weekday numbers start at 0 for Monday, which helps in grouping by weekdays.

Quarter divides the year into four parts: Jan-Mar (1), Apr-Jun (2), Jul-Sep (3), Oct-Dec (4).

Summary

Date feature extraction breaks down dates into parts like year, month, and day.

It helps find patterns and group data by time periods.

Use pandas .dt accessor to get these features easily.

Practice

1. What does the dt.year attribute do when applied to a pandas datetime column?

easy

A. Extracts the year part from each date in the column

B. Extracts the day of the week as a number

C. Converts the date to a string format

D. Calculates the difference between dates

Solution

1. Step 1: Understand pandas datetime accessor

   The dt accessor allows extraction of date parts from datetime columns.

2. Step 2: Identify what dt.year extracts

   dt.year extracts the year component from each datetime value.

3. Final Answer:

   Extracts the year part from each date in the column -> Option A

4. Quick Check:

   dt.year = year extraction [OK]

Hint: Remember dt.year gives the year number from dates [OK]

Common Mistakes:

• Confusing dt.year with dt.day or dt.month
• Thinking dt.year converts dates to strings
• Assuming dt.year calculates date differences

2. Which of the following is the correct syntax to extract the month from a pandas datetime column named df['date']?

easy

A. df['date'].month

B. df['date'].dt.month

C. df['date'].get_month()

D. df['date'].extract('month')

Solution

1. Step 1: Recall pandas datetime accessor usage

   To extract date parts, pandas requires the dt accessor before the attribute.

2. Step 2: Identify correct syntax for month extraction

   The correct syntax is df['date'].dt.month. Other options are invalid methods or missing dt.

3. Final Answer:

   df['date'].dt.month -> Option B

4. Quick Check:

   Use dt.month to get month [OK]

Hint: Use .dt before date parts like month or day [OK]

Common Mistakes:

• Omitting .dt accessor
• Using non-existent methods like get_month()
• Trying to extract with extract('month')

3. Given the following code, what will be the output?

import pandas as pd
dates = pd.to_datetime(['2023-01-15', '2023-07-04', '2023-12-31'])
df = pd.DataFrame({'date': dates})
df['day_of_week'] = df['date'].dt.dayofweek
print(df['day_of_week'].tolist())

medium

A. [6, 1, 0]

B. [0, 1, 6]

C. [6, 2, 6]

D. [6, 1, 6]

Solution

1. Step 1: Convert dates to day of week numbers

   Using dt.dayofweek, Monday=0 ... Sunday=6. Dates: 2023-01-15 is Sunday (6), 2023-07-04 is Tuesday (1), 2023-12-31 is Sunday (6).

2. Step 2: List the dayofweek values

   The list is [6, 1, 6].

3. Final Answer:

   [6, 1, 6] -> Option D

4. Quick Check:

   Sunday=6, Tuesday=1, Sunday=6 [OK]

Hint: Remember Monday=0, Sunday=6 for dayofweek [OK]

Common Mistakes:

• Confusing dayofweek with weekday names
• Assuming Sunday=0 instead of 6
• Mixing up dayofweek and day_name()

4. What is wrong with this code snippet to extract the hour from a datetime column df['timestamp']?

df['hour'] = df['timestamp'].hour

medium

A. Missing .dt accessor before .hour

B. The column must be converted to string first

C. The .hour attribute does not exist in pandas

D. The code should use .time.hour instead

Solution

1. Step 1: Check pandas datetime attribute access

   Datetime properties require the dt accessor to access parts like hour.

2. Step 2: Identify missing .dt in code

   The code uses df['timestamp'].hour which is invalid; it should be df['timestamp'].dt.hour.

3. Final Answer:

   Missing .dt accessor before .hour -> Option A

4. Quick Check:

   Use .dt.hour to extract hour [OK]

Hint: Always use .dt before date parts like hour [OK]

Common Mistakes:

• Trying to access .hour directly on Series
• Converting datetime to string unnecessarily
• Using .time.hour which is invalid in pandas

5. You have a pandas DataFrame df with a datetime column 'order_date'. You want to create a new column 'is_weekend' that is True if the order was placed on Saturday or Sunday, and False otherwise. Which code snippet correctly achieves this?

hard

A. df['is_weekend'] = df['order_date'].dt.dayofweek == 6 or 5

B. df['is_weekend'] = df['order_date'].dt.weekday in [5,6]

C. df['is_weekend'] = df['order_date'].dt.dayofweek >= 5

D. df['is_weekend'] = df['order_date'].dt.day_name() in ['Saturday', 'Sunday']

Solution

1. Step 1: Understand dayofweek values for weekend

   In pandas, dt.dayofweek returns 0 for Monday through 6 for Sunday. Weekend days are 5 (Saturday) and 6 (Sunday).

2. Step 2: Check which code correctly marks weekend

   df['is_weekend'] = df['order_date'].dt.dayofweek >= 5 uses df['order_date'].dt.dayofweek >= 5, which is True for 5 and 6, correctly marking weekends. Other options use invalid syntax or incorrect logic.

3. Final Answer:

   df['is_weekend'] = df['order_date'].dt.dayofweek >= 5 -> Option C

4. Quick Check:

   Weekend days are dayofweek >= 5 [OK]

Hint: Weekend days have dayofweek 5 or 6, use >= 5 [OK]

Common Mistakes:

• Using 'in' without vectorized methods
• Using 'or' instead of bitwise operators
• Confusing dayofweek numbers or names