'When writing to google sheet, I get Object of type Decimal is not JSON serializable

I'm trying to write a dateframe to a google sheet. I followed the instructions in the sheets Python API page, but I'm getting a strange error.

The dataframe as decimal values, like this

USD        ORDERS_AVG     DATE
316.60     123.0          2022-05-05 12:30:47.624000-07:00

With the following code, I'm trying to write on a google sheet:

try:
    SCOPES = ['https://www.googleapis.com/auth/spreadsheets']
    SERVICE_ACCOUNT_FILE = 'key.json'

    creds = None
    creds = service_account.Credentials.from_service_account_file(
        SERVICE_ACCOUNT_FILE, scopes=SCOPES)

    SAMPLE_SPREADSHEET_ID = 'xxxxxxxxxxxxxxxxxxxxxxxxxx'
    service = build('sheets', 'v4', credentials=creds)
    sheet = service.spreadsheets()
except HttpError as err:
    print(err)

if df is not None and sheet:
    lol = df.values.tolist()
    try:
        clear_request = service.spreadsheets().values().clear(
            spreadsheetId=SAMPLE_SPREADSHEET_ID, range="Data!A2:S")
        clear_response = clear_request.execute()
        print(clear_response)
        request = sheet.values().update(spreadsheetId=SAMPLE_SPREADSHEET_ID,
                                        range="Data!A2", valueInputOption="USER_ENTERED", body={"values": lol}).execute()
        print(request)
    except HttpError as err:
        print(err)

But unless I convert the whole dataframe to string using .astype(str), I get the following error:

Traceback (most recent call last):
  File "dev2.py", line 107, in <module>
    request = sheet.values().update(spreadsheetId=SAMPLE_SPREADSHEET_ID,
  File "/home/alainmore/anaconda3/envs/slot_alerts_env/lib/python3.8/site-packages/googleapiclient/discovery.py", line 1094, in method
    headers, params, query, body = model.request(
  File "/home/alainmore/anaconda3/envs/slot_alerts_env/lib/python3.8/site-packages/googleapiclient/model.py", line 160, in request
    body_value = self.serialize(body_value)
  File "/home/alainmore/anaconda3/envs/slot_alerts_env/lib/python3.8/site-packages/googleapiclient/model.py", line 273, in serialize
    return json.dumps(body_value)
  File "/home/alainmore/anaconda3/envs/slot_alerts_env/lib/python3.8/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/home/alainmore/anaconda3/envs/slot_alerts_env/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/home/alainmore/anaconda3/envs/slot_alerts_env/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/home/alainmore/anaconda3/envs/slot_alerts_env/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Decimal is not JSON serializable

The thing is that I don't want to convert decimal values to strings. I need to keep numbers as numbers, and dates as dates, in the Google Sheet.

What am I doing wrong?

------EDIT 1

if I print df.values.lolist(), I get this:

[Decimal('91.21'), Decimal('4.1'), Timestamp('2022-05-05 14:30:11.601000-0700', tz='America/Los_Angeles')], 
[Decimal('122.04'), Decimal('2.23'), Timestamp('2022-05-05 14:30:11.601000-0700', tz='America/Los_Angeles')], 
[Decimal('118.83'), Decimal('5.00'), Timestamp('2022-05-05 14:30:11.601000-0700', tz='America/Los_Angeles')], 
[Decimal('1591.61'), Decimal('28.00'), Timestamp('2022-05-05 14:30:11.601000-0700', tz='America/Los_Angeles')], 
[Decimal('97.43'), Decimal('3.02'), Timestamp('2022-05-05 14:30:11.601000-0700', tz='America/Los_Angeles')]]

Sample dataframe:

x = datetime.datetime.now()  

data = {'USD':[2219.48, 392.01, 16211.63, None],
        'ORDERS':[167.00, 10.00, 572.00, None],
        'DATE':[x,x,x,x]}
df = pd.DataFrame(data)


Solution 1:[1]

Well, this was a really REALLY dumb solution. It seems the google sheet I was writing to was set with a locale for Colombia, where the use a decimal comma, instead of a decimal point... so values like 53.00 were considered as a string... after changing the locale, I was able to save the dataframe as strings and google sheets automatically recognized numeric values with decimals.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Alain