Count all NaNs in a pandas DataFrame

Question

I'm trying to count NaN element (data type class 'numpy.float64')in pandas series to know how many are there which data type is class 'pandas.core.series.Series'

This is for count null value in pandas series

import pandas as pd
oc=pd.read_csv(csv_file)
oc.count("NaN")

my expected output of oc,count("NaN") to be 7 but it show 'Level NaN must be same as name (None)'

Answer 1

The argument to count isn't what you want counted (it's actually the axis name or index).

You're looking for df.isna().values.sum() (to count NaNs across the entire DataFrame), or len(df) - df['column'].count() (to count NaNs in a specific column).

Answer 2

You can use either of the following if your Series.dtype is float64:

oc.isin([np.nan]).sum()
oc.isna().sum()

If your Series is of mixed data-type you can use the following:

oc.isin([np.nan, 'NaN']).sum()

Answer 3

oc.size : returns total element counts of dataframe including NaN
oc.count().sum(): return total element counts of dataframe excluding NaN

Therefore, another way to count number of NaN in dataframe is doing subtraction on them:

NaN_count = oc.size - oc.count().sum()

Answer 4

Just for fun, you can do either

df.isnull().sum().sum()

or

len(df)*len(df.columns) - len(df.stack())

Answer 5

If your dataframe looks like this ;

aa = pd.DataFrame(np.array([[1,2,np.nan],[3,np.nan,5],[8,7,6],
                 [np.nan,np.nan,0]]), columns=['a','b','c'])
    a    b    c
0  1.0  2.0  NaN
1  3.0  NaN  5.0
2  8.0  7.0  6.0
3  NaN  NaN  0.0

To count 'nan' by cols, you can try this

aa.isnull().sum()
a    1
b    2
c    1

For total count of nan

aa.isnull().values.sum()
4