'question when using interpolate for missing value

I had a dataset looks like below:

import pandas as pd
test = pd.DataFrame({
    'date': ['2018-01-03 00:00:00', '2018-01-04 00:00:00', '2018-01-05 00:00:00'],
    'coal': [2669.0, np.nan ,2731.0],
     'hydro': [222.0, np.nan ,230.0],
    'unit': ['Gwh', 'Gwh', 'Gwh'],
})

test['date'] = pd.to_datetime(test['date'])

and when i was trying to fill the null by using interpolate method:

for x in test.columns.to_list():
    test[x] = test[x].interpolate())

it got error :

Invalid fill method. Expecting pad (ffill) or backfill (bfill). Got linear

and when i remove the

test['date'] = pd.to_datetime(test['date'])

it will work just fine, so i dont know why this happened:(

and also, it works fine when i using the sample df, but when i try it on my own dataset, it has no error, but the null value still exist, like the fillna() never used:(

and when i was using below code, then use the fillna code:

test['date'] = test['date'].astype(object)

it works on my sample, but not my own dataset:( my own dataset still like i never used the fillna method

im so confused by now:( was wondering if someone could explain why this happened? I try to google it, but no result:(

or maybe i dont know how to google it:(

P.S. it works fine if i fill it one column at a time when i try it on my own dataset, like:

test['coal'] = test['coal'].interpolate())
test['hydro'] = test['hydro'].interpolate())

but not the for loops:(

Its working when i first change the dtype to float, so maybe something wrong when it was object(not all object)?

for x in total_list:
    df_all[x] = df_all[x].astype(float)
    df_all[x] = df_all[x].interpolate()

anyway, thank you all:) this took me 2hours already:(



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source