'Inserting Pandas DataFrame with NaN values into Google Sheet

I have a Pandas DataFrame and I am trying to insert it into a Google Sheet Worksheet using gspread library, but I am getting this exception:

Traceback (most recent call last):
  File "C:\Users\hugo.villalobos\Documents\code\runner.py", line 108, in <module>
    raise e
  File "C:\Users\hugo.villalobos\Documents\code\runner.py", line 72, in <module>
    main(
  File "C:\Users\hugo.villalobos\Documents\code\runner.py", line 44, in main
    outputs.execute()
  File "C:\Users\hugo.villalobos\Documents\code\outputs.py", line 17, in execute
    gsheet = self.generate_gsheet()
  File "C:\Users\hugo.villalobos\Documents\code\outputs.py", line 30, in generate_gsheet
    self.append_final_mile_detail_tab()
  File "C:\Users\hugo.villalobos\Documents\code\outputs.py", line 90, in append_final_mile_detail_tab
    raise Exception(e)
Exception: {'code': 400, 'message': 'Invalid JSON payload received. Unexpected token.\nUND", "94002", "2", NaN, false, 11, "GRO\n                    ^', 'status': 'INVALID_ARGUMENT'}

I have to do the same with several DataFrames using work_sheet_tab.update(data_frame.columns.values.tolist()] + data_frame.values.tolist()), and they all succedd handsomely, but this one, and the only difference I can notice is that the DataFrame that fails has NaN values, which are scattered randomly all around it.

I tried inserting the DataFrame records one by one by using:

        for record in records_to_add:
            try:
                work_sheet_tab.append_row(record)
            except Exception as e:
                print(e)
                print(str(i) + " " + str(record))
                raise Exception(e)

But I get the same exception, and the only row that succeeds is the first one, which contains only strings with the names of the columns.

I tried to change the values of NaN columns for None using:

for data in record:
    print(str(data) + ": " + str(type(data)))
    if data == np.NaN:
       data = None

But the Nan values don't change, which means that the comparison if data == np.NaN can't identify NaN values, even when they are there as you can see here:

ep: <class 'str'>
Friday: <class 'str'>
axle_ca_local_zone: <class 'str'>
CA_AXLE-G-1: <class 'str'>
AXLE: <class 'str'>
GROUND: <class 'str'>
94002: <class 'str'>
2: <class 'str'>
nan: <class 'float'>
False: <class 'bool'>
11: <class 'int'>
GROUND: <class 'str'>
Flat: <class 'str'>
: <class 'str'>
5.96: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
5.96: <class 'float'>
0.0: <class 'float'>
65.56: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>
nan: <class 'float'>

In this list, I printed the value and type of each column of the failing record.

How can I insert these values into a Google Sheet?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source