'airflow TypeError: cannot pickle 'pyodbc.Cursor' object
{python.py:175} INFO - Done. Returned value was: <pyodbc.Cursor object at 0x7f55f4a02a30>
[2022-03-18, 17:56:17 UTC] {taskinstance.py:1700} ERROR - Task failed with exception
Traceback (most recent call last):
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
self.xcom_push(key=XCOM_RETURN_KEY, value=result)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 2135, in xcom_push
XCom.set(
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/utils/session.py", line 67, in wrapper
return func(*args, **kwargs)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/xcom.py", line 100, in set
value = XCom.serialize_value(value)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/xcom.py", line 329, in serialize_value
return pickle.dumps(value)
TypeError: cannot pickle 'pyodbc.Cursor' object
I am using pickle in my ML model loading, Not sure how to troubleshoot this error, how the pickle loading is connected to X_com, I stopped using Xcomm in code as well, even though I am getting the pyodbc.Cursor error. I am using pyodbc for my database connectivity check
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
