“As soon as we moved to the CeleryExecutor, despite solving one of our problems, new things started to go wrong. After a few days (or even hours) running, some of our Celery workers started dying from OOM (Out of Memory) problems.”

https://medium.com/apache-airflow/what-we-learned-after-running-airflow-on-kubernetes-for-2-years-0537b157acfd

Mastodon Source 🐘