Unifiy data cells on pandas according to certain conditions

Question

I have a problem with unifying data cells in pandas where I have a data frame named catalogue that looks like this:

I want to unify the data cells according to the last row but I should leave the data cells with the value "NaN" unchanged.

Here is my method:

def unify_table_cells(catalogue, matches):
    for match in matches:
        column = match[0][1]
        unified_val = match[1][1] 
        catalogue.where((catalogue[[column]] == unified_val) | (catalogue[[column]] == np.nan), unified_val, inplace =True)
        
    return catalogue

But it gives me a wrong output:

The data cell in the catalogue["table_2","Document Type"] should be NaN but it is also unified in the process, what I am doing wrong here ?

Also, I would like to unify the other columns for example (Year, Access Type, City) through which each column should has the same value as its column. For example in the column "Year", its data cells should be changed to "Year".

Is this possible to apply my conditions with my first question in one line in pandas?

Thanks a lot in advance

Answer 1

At last, I noticed the problem! the NaN in some of the data cells are referred to as an empty data cell which should be solved using this line instead:-

catalogue.where(catalogue[[column]].isnull(), unified_val, inplace =True)

This solved my issue. This link here helped me notice the problem https://medium.com/analytics-vidhya/dealing-with-missing-values-nan-and-none-in-python-6fc9b8fb4f31#:~:text=Unlike%20other%20popular%20programming%20languages,Python%20uses%20NaN%20and%20None%20.&text=nan%20is%20IEEE%20754%20floating,NoneType%20and%20is%20an%20object.