rodigu/airflow-pickling.md

Created February 17, 2025 12:12

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/rodigu/7fe697d94849e50e0f8fdbef6b8aa548.js"></script>
Save rodigu/7fe697d94849e50e0f8fdbef6b8aa548 to your computer and use it in GitHub Desktop.

pickling classes in airflow

Raw

one of our classes, which made use of the pbipy library, could not be passed through XCom. we were getting the following error: airflow cannot pickle [CLASS] object

the fix involved changing a cfg variable to false:

# Whether to enable pickling for xcom (note that this is insecure and allows for
# RCE exploits).
# Variable: AIRFLOW__CORE__ENABLE_XCOM_PICKLING
enable_xcom_pickling = False