Skip to content

Commit

Permalink
Clarify lack of sync between workers and scheduler (#37913)
Browse files Browse the repository at this point in the history
* Clarify lack of sync between workers and scheduler

* Update docs/apache-airflow/core-concepts/overview.rst

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
  • Loading branch information
eladkal and potiuk committed Mar 5, 2024
1 parent c29d728 commit 2867951
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions docs/apache-airflow/core-concepts/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,14 @@ executed in the context of the scheduler.

.. image:: ../img/diagram_dag_processor_airflow_architecture.png

.. note::

When DAG file is changed there can be cases where the scheduler and the worker will see different
versions of the DAG until both components catch up. You can avoid the issue by making sure dag is
deactivated during deployment and reactivate once finished. If needed, the cadence of sync and scan
of DAG folder can be configured. Please make sure you really know what you are doing if you change
the configurations.

.. _overview:workloads:

Workloads
Expand Down

0 comments on commit 2867951

Please sign in to comment.