Timezone handling basics in Pandas

Introduction

Timezones help us understand when events happen in different parts of the world. Handling timezones correctly avoids confusion in time data.

You have timestamps from users in different countries and want to compare them.

You want to convert event times from one timezone to another.

You need to store times in a standard timezone like UTC for consistency.

You want to display times in the local timezone of your audience.

You are analyzing data collected across multiple timezones.

Syntax

Pandas

import pandas as pd

dt = pd.Timestamp('2024-06-01 12:00')
# Create timezone-aware datetime
pd.Timestamp('2024-06-01 12:00').tz_localize('America/New_York')

# Convert timezone
dt.tz_convert('Europe/London')

tz_localize sets the timezone for naive timestamps (no timezone info).

tz_convert changes the timezone of an aware timestamp.

Examples

Make a naive timestamp timezone-aware by setting it to UTC.

Pandas

import pandas as pd

dt = pd.Timestamp('2024-06-01 12:00')
dt_aware = dt.tz_localize('UTC')
print(dt_aware)

Convert the UTC time to New York time.

Pandas

dt_ny = dt_aware.tz_convert('America/New_York')
print(dt_ny)

Create a New York time and convert it to UTC.

Pandas

dt_ny = pd.Timestamp('2024-06-01 08:00').tz_localize('America/New_York')
dt_utc = dt_ny.tz_convert('UTC')
print(dt_utc)

Sample Program

This program shows how to start with a time without timezone, add UTC timezone, then convert it to Tokyo and New York times.

Pandas

import pandas as pd

# Create a naive timestamp
naive_time = pd.Timestamp('2024-06-01 15:00')
print(f"Naive time: {naive_time}")

# Localize to UTC timezone
utc_time = naive_time.tz_localize('UTC')
print(f"UTC time: {utc_time}")

# Convert UTC time to Tokyo timezone
tokyo_time = utc_time.tz_convert('Asia/Tokyo')
print(f"Tokyo time: {tokyo_time}")

# Convert UTC time to New York timezone
ny_time = utc_time.tz_convert('America/New_York')
print(f"New York time: {ny_time}")

Important Notes

Always localize naive timestamps before converting timezones.

Timezone names like 'America/New_York' follow the IANA timezone database.

UTC is a good standard timezone to store times before converting to local zones.

Summary

Timezone handling avoids confusion when working with times from different places.

Use tz_localize to add timezone info to naive timestamps.

Use tz_convert to change the timezone of aware timestamps.

Practice

1. What does the tz_localize method do in pandas when working with timestamps?

easy

A. Converts timestamps from one timezone to another.

B. Adds timezone information to timestamps without any timezone (naive timestamps).

C. Removes timezone information from timestamps.

D. Changes the timestamp format to string.

Solution

1. Step 1: Understand naive timestamps

   Naive timestamps have no timezone information attached.

2. Step 2: Role of tz_localize

   This method adds timezone info to naive timestamps without changing the actual time.

3. Final Answer:

   Adds timezone information to timestamps without any timezone (naive timestamps). -> Option B

4. Quick Check:

   tz_localize adds timezone [OK]

Hint: Remember: localize adds timezone, convert changes it [OK]

Common Mistakes:

• Confusing tz_localize with tz_convert
• Thinking tz_localize changes the time value
• Assuming tz_localize removes timezone info

2. Which of the following is the correct way to convert a timezone-aware pandas datetime Series dt from UTC to US/Eastern timezone?

easy

A. dt.tz_convert('US/Eastern')

B. dt.tz_localize('UTC').tz_convert('US/Eastern')

C. dt.tz_localize('US/Eastern')

D. dt.tz_convert('UTC')

Solution

1. Step 1: Check if dt is timezone-aware

   Since dt is timezone-aware, we use tz_convert to change its timezone.

2. Step 2: Apply correct method

   tz_convert('US/Eastern') converts from current timezone (UTC) to US/Eastern.

3. Final Answer:

   dt.tz_convert('US/Eastern') -> Option A

4. Quick Check:

   Convert aware timestamps with tz_convert [OK]

Hint: Use tz_convert only on timezone-aware timestamps [OK]

Common Mistakes:

• Using tz_localize on already aware timestamps
• Not converting timezone properly
• Confusing order of localization and conversion

3. Given the following code, what will be the output?

import pandas as pd
rng = pd.date_range('2024-01-01 12:00', periods=2, freq='H')
rng_utc = rng.tz_localize('UTC')
rng_est = rng_utc.tz_convert('US/Eastern')
print(rng_est)

medium

A. DatetimeIndex with times 17:00 and 18:00 in US/Eastern timezone

B. DatetimeIndex with times 12:00 and 13:00 in US/Eastern timezone

C. DatetimeIndex with times 07:00 and 08:00 in US/Eastern timezone

D. Error because tz_convert cannot be used after tz_localize

Solution

1. Step 1: Localize naive timestamps to UTC

   The original times are 12:00 and 13:00 UTC after tz_localize('UTC').

2. Step 2: Convert UTC times to US/Eastern timezone

   US/Eastern is UTC-5 in winter, so times become 07:00 and 08:00 with timezone info.

3. Final Answer:

   DatetimeIndex with times 07:00 and 08:00 in US/Eastern timezone -> Option C

4. Quick Check:

   UTC 12:00 -> EST 07:00 [OK]

Hint: Convert UTC to EST subtracts 5 hours [OK]

Common Mistakes:

• Assuming times stay the same after conversion
• Mixing up tz_localize and tz_convert order
• Expecting error instead of valid output

4. What is wrong with this code snippet?

import pandas as pd
dt = pd.to_datetime(['2024-01-01 10:00', '2024-01-01 11:00'])
dt = dt.tz_convert('Europe/London')

medium

A. Cannot use tz_convert on naive timestamps; must localize first.

B. The timezone string 'Europe/London' is invalid.

C. The to_datetime function needs a timezone argument.

D. The code is correct and runs without error.

Solution

1. Step 1: Check timestamp awareness

   The timestamps created by to_datetime are naive (no timezone).

2. Step 2: Understand tz_convert usage

   tz_convert requires timestamps to be timezone-aware; naive timestamps must use tz_localize first.

3. Final Answer:

   Cannot use tz_convert on naive timestamps; must localize first. -> Option A

4. Quick Check:

   tz_convert needs aware timestamps [OK]

Hint: Localize before convert on naive timestamps [OK]

Common Mistakes:

• Trying to convert timezone without localizing
• Assuming to_datetime adds timezone automatically
• Confusing error cause with invalid timezone string

5. You have a pandas Series of timestamps without timezone info representing local times in New York. You want to convert them to UTC timestamps correctly considering daylight saving time. Which sequence of operations should you perform?

hard

A. Convert timestamps to string first, then parse with UTC timezone.

B. Use tz_convert('UTC') directly on naive timestamps.

C. Use tz_localize('UTC') then tz_convert('America/New_York').

D. Use tz_localize('America/New_York') then tz_convert('UTC').

Solution

1. Step 1: Localize naive timestamps to New York timezone

   Since timestamps are local New York times without timezone, use tz_localize('America/New_York') to mark them as such, correctly handling daylight saving.

2. Step 2: Convert localized timestamps to UTC

   Use tz_convert('UTC') to get equivalent UTC times.

3. Final Answer:

   Use tz_localize('America/New_York') then tz_convert('UTC'). -> Option D

4. Quick Check:

   Localize then convert for correct timezone shifts [OK]

Hint: Localize first, then convert to UTC [OK]

Common Mistakes:

• Converting naive timestamps directly
• Localizing to wrong timezone first
• Ignoring daylight saving time effects