Converting UNIX timestamps into pandas datetime taking timezone into account [closed]

Question

I would like to convert the following pandas series containing UNIX timestamps into a pandas datetime using either to_datetime() or arrow library in Python. I want to set the timezone to UTC and currently it is Europe/Paris

For Pandas I am using the following function, but not sure how to take the Europe\Paris timezone into account

pd.to_datetime(df['dates'], unit='s')

Answer 1

If you are passing datetime objects that are using Europe\Paris timezone:

Pass utc=True when converting, like this:

pd.to_datetime(df['dates'], unit='s', utc=True)

Quoting the pd.to_datetime() documentation:

utc : boolean, default None
    Return UTC DatetimeIndex if True (converting any tz-aware
    datetime.datetime objects as well).

Or, if like more control, use pd.Series.dt.tz_convert().

If the timestamps are generated using Europe\Paris timezone:

You should convert the timestamp to a native datetime object, localize it using the original timezone (make it timezone aware datetime object), and then convert it to the required timezone.

Like this:

# convert timestamp to native datetime
ps = pd.to_datetime(df['dates'], unit='s')  # dtype: datetime64[ns]
# localize it to Europe/Paris
ps = ps.dt.tz_localize('Europe/Paris')  # dtype: datetime64[ns, Europe/Paris]
# Finaly, convert to UTC
ps = ps.dt.tz_convert('UTC')  # dtype: datetime64[ns, UTC]

The difference between tz_localize and tz_convert is that the first doesn't move the time to another time zone, while the second does.

Answer 2

Suppose That You have Dataset like this,

2019-02-02 11:32:46.484236    4
2019-02-03 11:32:46.484236    1
2019-02-04 11:32:46.484236    8
2019-02-05 11:32:46.484236    2
2019-02-06 11:32:46.484236    4
2019-02-07 11:32:46.484236    2
2019-02-08 11:32:46.484236    5
2019-02-09 11:32:46.484236    5
2019-02-10 11:32:46.484236    6
2019-02-11 11:32:46.484236    1

Here it's on 'Asia/Kolkata' timezone.

As per your Question, You need UTC timezone from this.

So, For that First you need to localize that, this Dataset belongs to which timezone?.

So, For this.....

df = df.tz_localize(tz = 'Asia/Kolkata')

it will make localize on the name of 'Asia/Kolkata'.

Output:

2019-02-02 11:32:46.484236+05:30    4
2019-02-03 11:32:46.484236+05:30    1
2019-02-04 11:32:46.484236+05:30    8
2019-02-05 11:32:46.484236+05:30    2
2019-02-06 11:32:46.484236+05:30    4
2019-02-07 11:32:46.484236+05:30    2
2019-02-08 11:32:46.484236+05:30    5
2019-02-09 11:32:46.484236+05:30    5
2019-02-10 11:32:46.484236+05:30    6
2019-02-11 11:32:46.484236+05:30    1

Now You can convert your 'Asia/Kolkata' timezone to UTC. Like this...

df = df.tz_convert(tz = 'UTC')

Output:

2019-02-02 06:02:46.484236+00:00    4
2019-02-03 06:02:46.484236+00:00    1
2019-02-04 06:02:46.484236+00:00    8
2019-02-05 06:02:46.484236+00:00    2
2019-02-06 06:02:46.484236+00:00    4
2019-02-07 06:02:46.484236+00:00    2
2019-02-08 06:02:46.484236+00:00    5
2019-02-09 06:02:46.484236+00:00    5
2019-02-10 06:02:46.484236+00:00    6
2019-02-11 06:02:46.484236+00:00    1

You can do same for your 'Europe\Paris' timezone.