Timezone handling basics in Pandas - Step-by-Step Execution

Concept Flow - Timezone handling basics

Create naive datetime

↓

Localize to timezone

↓

Convert to another timezone

↓

Use timezone-aware datetime for analysis

Start with a datetime without timezone, add timezone info, then convert to another timezone for correct time representation.

Execution Sample

Pandas

import pandas as pd
naive = pd.Timestamp('2024-06-01 12:00:00')
localized = naive.tz_localize('US/Eastern')
converted = localized.tz_convert('Europe/London')
print(converted)

Create a naive datetime, localize it to US/Eastern timezone, then convert it to Europe/London timezone.

Execution Table

Step	Variable	Value/Action	Result/Output
1	naive	pd.Timestamp('2024-06-01 12:00:00')	2024-06-01 12:00:00 (naive)
2	localized	naive.tz_localize('US/Eastern')	2024-06-01 12:00:00-04:00 (US/Eastern)
3	converted	localized.tz_convert('Europe/London')	2024-06-01 17:00:00+01:00 (Europe/London)
4	print(converted)	Output the converted datetime	2024-06-01 17:00:00+01:00

💡 All steps complete, datetime converted from naive to localized to another timezone.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
naive	None	2024-06-01 12:00:00 (naive)	2024-06-01 12:00:00 (naive)	2024-06-01 12:00:00 (naive)	2024-06-01 12:00:00 (naive)
localized	None	None	2024-06-01 12:00:00-04:00 (US/Eastern)	2024-06-01 12:00:00-04:00 (US/Eastern)	2024-06-01 12:00:00-04:00 (US/Eastern)
converted	None	None	None	2024-06-01 17:00:00+01:00 (Europe/London)	2024-06-01 17:00:00+01:00 (Europe/London)

Key Moments - 3 Insights

Why do we need to localize a naive datetime before converting timezones?

Because a naive datetime has no timezone info, tz_localize assigns the correct timezone so pandas knows how to convert it properly, as shown in step 2 of the execution_table.

What happens if we try to convert a naive datetime directly without localizing?

Pandas will raise an error because it cannot convert timezones without knowing the original timezone. This is why step 2 (localize) must happen before step 3 (convert).

Why does the time change from 12:00 to 17:00 when converting from US/Eastern to Europe/London?

Because Europe/London is 5 hours ahead of US/Eastern at that date, so the time shifts accordingly, as shown in step 3 of the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, what is the timezone of the 'localized' variable after step 2?

A US/Eastern

B Europe/London

C UTC

D No timezone (naive)

💡 Hint

Check the 'Value/Action' and 'Result/Output' columns for step 2 in the execution_table.

At which step does the datetime become timezone-aware?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Refer to the 'Result/Output' column in the execution_table where timezone info first appears.

If we skip tz_localize and try to convert a naive datetime, what will happen?

A It will convert correctly

B It will raise an error

C It will ignore timezone and keep time same

D It will localize automatically

💡 Hint

Recall the key moment about converting naive datetime without localization.

Concept Snapshot

Timezone handling in pandas:
- Start with naive datetime (no timezone).
- Use tz_localize() to assign a timezone.
- Use tz_convert() to change to another timezone.
- Time shifts according to timezone differences.
- Always localize before converting.

Full Transcript

This example shows how to handle timezones in pandas. We start with a naive datetime without timezone info. Then we localize it to US/Eastern timezone using tz_localize. After that, we convert it to Europe/London timezone using tz_convert. The time changes from 12:00 to 17:00 because London is 5 hours ahead of Eastern time. This process ensures correct timezone-aware datetime handling for analysis.

Practice

1. What does the tz_localize method do in pandas when working with timestamps?

easy

A. Converts timestamps from one timezone to another.

B. Adds timezone information to timestamps without any timezone (naive timestamps).

C. Removes timezone information from timestamps.

D. Changes the timestamp format to string.

Solution

1. Step 1: Understand naive timestamps

   Naive timestamps have no timezone information attached.

2. Step 2: Role of tz_localize

   This method adds timezone info to naive timestamps without changing the actual time.

3. Final Answer:

   Adds timezone information to timestamps without any timezone (naive timestamps). -> Option B

4. Quick Check:

   tz_localize adds timezone [OK]

Hint: Remember: localize adds timezone, convert changes it [OK]

Common Mistakes:

• Confusing tz_localize with tz_convert
• Thinking tz_localize changes the time value
• Assuming tz_localize removes timezone info

2. Which of the following is the correct way to convert a timezone-aware pandas datetime Series dt from UTC to US/Eastern timezone?

easy

A. dt.tz_convert('US/Eastern')

B. dt.tz_localize('UTC').tz_convert('US/Eastern')

C. dt.tz_localize('US/Eastern')

D. dt.tz_convert('UTC')

Solution

1. Step 1: Check if dt is timezone-aware

   Since dt is timezone-aware, we use tz_convert to change its timezone.

2. Step 2: Apply correct method

   tz_convert('US/Eastern') converts from current timezone (UTC) to US/Eastern.

3. Final Answer:

   dt.tz_convert('US/Eastern') -> Option A

4. Quick Check:

   Convert aware timestamps with tz_convert [OK]

Hint: Use tz_convert only on timezone-aware timestamps [OK]

Common Mistakes:

• Using tz_localize on already aware timestamps
• Not converting timezone properly
• Confusing order of localization and conversion

3. Given the following code, what will be the output?

import pandas as pd
rng = pd.date_range('2024-01-01 12:00', periods=2, freq='H')
rng_utc = rng.tz_localize('UTC')
rng_est = rng_utc.tz_convert('US/Eastern')
print(rng_est)

medium

A. DatetimeIndex with times 17:00 and 18:00 in US/Eastern timezone

B. DatetimeIndex with times 12:00 and 13:00 in US/Eastern timezone

C. DatetimeIndex with times 07:00 and 08:00 in US/Eastern timezone

D. Error because tz_convert cannot be used after tz_localize

Solution

1. Step 1: Localize naive timestamps to UTC

   The original times are 12:00 and 13:00 UTC after tz_localize('UTC').

2. Step 2: Convert UTC times to US/Eastern timezone

   US/Eastern is UTC-5 in winter, so times become 07:00 and 08:00 with timezone info.

3. Final Answer:

   DatetimeIndex with times 07:00 and 08:00 in US/Eastern timezone -> Option C

4. Quick Check:

   UTC 12:00 -> EST 07:00 [OK]

Hint: Convert UTC to EST subtracts 5 hours [OK]

Common Mistakes:

• Assuming times stay the same after conversion
• Mixing up tz_localize and tz_convert order
• Expecting error instead of valid output

4. What is wrong with this code snippet?

import pandas as pd
dt = pd.to_datetime(['2024-01-01 10:00', '2024-01-01 11:00'])
dt = dt.tz_convert('Europe/London')

medium

A. Cannot use tz_convert on naive timestamps; must localize first.

B. The timezone string 'Europe/London' is invalid.

C. The to_datetime function needs a timezone argument.

D. The code is correct and runs without error.

Solution

1. Step 1: Check timestamp awareness

   The timestamps created by to_datetime are naive (no timezone).

2. Step 2: Understand tz_convert usage

   tz_convert requires timestamps to be timezone-aware; naive timestamps must use tz_localize first.

3. Final Answer:

   Cannot use tz_convert on naive timestamps; must localize first. -> Option A

4. Quick Check:

   tz_convert needs aware timestamps [OK]

Hint: Localize before convert on naive timestamps [OK]

Common Mistakes:

• Trying to convert timezone without localizing
• Assuming to_datetime adds timezone automatically
• Confusing error cause with invalid timezone string

5. You have a pandas Series of timestamps without timezone info representing local times in New York. You want to convert them to UTC timestamps correctly considering daylight saving time. Which sequence of operations should you perform?

hard

A. Convert timestamps to string first, then parse with UTC timezone.

B. Use tz_convert('UTC') directly on naive timestamps.

C. Use tz_localize('UTC') then tz_convert('America/New_York').

D. Use tz_localize('America/New_York') then tz_convert('UTC').

Solution

1. Step 1: Localize naive timestamps to New York timezone

   Since timestamps are local New York times without timezone, use tz_localize('America/New_York') to mark them as such, correctly handling daylight saving.

2. Step 2: Convert localized timestamps to UTC

   Use tz_convert('UTC') to get equivalent UTC times.

3. Final Answer:

   Use tz_localize('America/New_York') then tz_convert('UTC'). -> Option D

4. Quick Check:

   Localize then convert for correct timezone shifts [OK]

Hint: Localize first, then convert to UTC [OK]

Common Mistakes:

• Converting naive timestamps directly
• Localizing to wrong timezone first
• Ignoring daylight saving time effects