'airflow TypeError: cannot pickle 'pyodbc.Cursor' object

 {python.py:175} INFO - Done. Returned value was: <pyodbc.Cursor object at 0x7f55f4a02a30>
[2022-03-18, 17:56:17 UTC] {taskinstance.py:1700} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
    self._execute_task_with_callbacks(context)
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
    result = self._execute_task(context, self.task)
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
    self.xcom_push(key=XCOM_RETURN_KEY, value=result)
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 2135, in xcom_push
    XCom.set(
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/utils/session.py", line 67, in wrapper
    return func(*args, **kwargs)
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/xcom.py", line 100, in set
    value = XCom.serialize_value(value)
  File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/xcom.py", line 329, in serialize_value
    return pickle.dumps(value)
TypeError: cannot pickle 'pyodbc.Cursor' object

I am using pickle in my ML model loading, Not sure how to troubleshoot this error, how the pickle loading is connected to X_com, I stopped using Xcomm in code as well, even though I am getting the pyodbc.Cursor error. I am using pyodbc for my database connectivity check



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source