Resampling time series data in Pandas

Introduction

Resampling helps change the frequency of time series data to see trends better or prepare data for analysis.

You have daily sales data but want to see monthly totals.

You want to convert minute-level temperature readings to hourly averages.

You need to fill missing dates by resampling and filling gaps.

You want to downsample data to reduce size for faster processing.

You want to upsample data to a finer time scale and fill missing values.

Syntax

Pandas

DataFrame.resample(rule, axis=0, closed=None, label=None, convention='start', kind=None, loffset=None, base=0, on=None, level=None)

rule is a string like 'D' for day, 'M' for month, 'H' for hour.

Use aggregation functions like mean(), sum() after resampling.

Examples

Resample data to monthly frequency and sum values in each month.

Pandas

df.resample('M').sum()

Resample data to hourly frequency and calculate average for each hour.

Pandas

df.resample('H').mean()

Upsample data to daily frequency and fill missing values forward.

Pandas

df.resample('D').ffill()

Sample Program

This code creates hourly data for 6 hours, then resamples it to daily sums and 2-hour averages.

Pandas

import pandas as pd
import numpy as np

# Create sample time series data with hourly frequency
rng = pd.date_range('2024-01-01', periods=6, freq='H')
data = pd.DataFrame({'value': [10, 15, 14, 20, 18, 22]}, index=rng)

# Resample to daily frequency and sum values
daily_sum = data.resample('D').sum()

# Resample to 2-hour frequency and mean values
two_hour_mean = data.resample('2H').mean()

print('Original data:')
print(data)
print('\nDaily sum:')
print(daily_sum)
print('\n2-hour mean:')
print(two_hour_mean)

Important Notes

Resampling works only if the DataFrame index is a datetime type.

Downsampling reduces data points; upsampling increases them and may need filling.

Common rules: 'D' = day, 'M' = month end, 'H' = hour, 'T' or 'min' = minute.

Summary

Resampling changes the time frequency of data for easier analysis.

Use aggregation functions like sum or mean after resampling.

Upsampling needs filling missing values; downsampling aggregates data.