Matplotlib memory error with pandas dataframe having a datetime64[ns, UTC] index

Question

I have following pandas data frame with "time" field of datatype datetime64[ns, UTC] as a INDEX and "intensity" of type float64.

The data frame just consist of 10 rows. Still when I am calling the function df.plot or plt.show() then it gives the one of the following errors:

1. MemoryError: Unable to allocate 447. GiB for an array with shape (59996093697,) and data type int64
2. Or driver crashes and system restarts.

Library version and Environment:

• Environment: Jupyter Notebook
• Library versions pandas==1.4.1 and matplotlib==3.5.1.

Please let me know if someone else has faced this problem and how did you resolve it.

You can reproduce this problem with following steps:

df = pd.read_csv('output.csv')
print(df.dtypes)
df["time"] = pd.to_datetime(df["time"])
print(df.dtypes)
df = df.set_index("time")
df.plot()

output.csv contains following lines:

time                                 intensity
2014-08-14 05:49:55+00:00           -17624170.29
2014-08-14 05:49:55.003906250+00:00 -19298497.11
2014-08-14 05:49:55.007812500+00:00 -17602197.3
2014-08-14 05:49:55.011718750+00:00 -16704962.77

Answer 1

Try to floor your index:

df.index = df.index.floor('S')