'PySpark: How to Convert UTC Timestamp Field to CST (US/Central) Keeping Timestamp Datatype
I have the following sample data set below. The column's datatype is timestamp
datetime_utc
2017-03-29T23:20:00Z
2017-04-17T19:00:00Z
I want to convert from UTC (coordinated universal time) to Central Standard Time (CST). I know how to do this in Python Pandas but don't know how in Pyspark.
Solution 1:[1]
In spark dataframe, you can use from_utc_timestamp() function to convert UTC to other timezones.
import pyspark.sql.functions as F
df.withColumn('datetime_cst', F.from_utc_timestamp('datetime_utc', 'CST')).show()
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Manoj Singh |