Filtering Pandas time series by specific EST time of day

Question

I am trying to match rows in a pandas dataframe where the DatetimeIndex is in US/Eastern timezone at exactly 15:30:00 each day US/Eastern time by doing the following:

check_time = pd.to_datetime("15:30:00").time()
last_30m_mask = df.index.time == check_time
up_df = df[last_30m_mask]

However the rows I get back are as follows:

                               w1      w2
timestamp                                                              
2021-08-04 15:30:00-04:00  382.37  388.27  
2021-08-05 15:30:00-04:00  395.65  400.78  
2021-08-09 15:30:00-04:00  434.79  437.04
...

Am I correct in thinking that this is instead giving me 15:30 UTC which is 11:30 EST (or 10:30 EST for most of the year)?

If so, how would I re-write the check_time variable to give me 15:30 EST (US/Eastern) at all times?

Answer 1

As FObersteiner correctly pointed out the timestamp is indeed local and the offset gives you the delta to UTC.

The error I was committing was on the other end. When converting my data from the source I wasn't giving it the proper context.

I had: time_col = pd.to_datetime(source_data["time"]).tz_localize("US/Eastern")

Whereas I needed to have: time_col = pd.to_datetime(source_data["time"]).tz_localize("UTC").tz_convert("US/Eastern")

This way I can now correctly compare my local times with pd.to_datetime("XX:XX:XX").time() as desired.