'Pandas DataFrame - Replace NULL value of Datetime column with some old date

Question: How can we replace the NULL values of a datetime column in Pandas DataFrame to something like 1900-01-01 00:00:00.000?

I am using Pandas data frame to import a large data file into a SQL Server 2019 table. My following code correctly replaces NULL values of numeric columns to 0, and NULL values of object (string) columns to empty string. But it does not change the NULL values of the datetime columns to 1900-01-01 00:00:00.000:

import sqlalchemy as sq
import datetime
import pandas as pd
import numpy as np

............
............
c = df.select_dtypes(np.datetime64).columns
df[c] = df[c].fillna('1900-01-01 00:00:00.000')
df.fillna('', inplace=True)

Ref: pandas.DataFrame.select_dtypes and this SO post



Solution 1:[1]

I guess the issue is that 'fillna' doesn't pick up 'NULL' as this is a sql keyword for NA values - but not for pandas. Maybe try replacing directly that string in your code?

df[c].replace('NULL','1900-01-01 00:00:00.000')

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Daniel Weigel