'How to get queued time duration of Airflow task?

I'm looking for a way to get the queued time duration of an Airflow task.

I'm running on all tasks of a DAG, but if its state is Queued - there is no start_date and therefore I can't tell how long it is in a Queued state.

Does anyone know a way ?



Solution 1:[1]

I have an idea for a rough queued time, but it's only possible after the task has actually started.

You can find the queued time by substracting the end_date of the "Parent" task from the start_date of your desired task once it started

If there's no "parent" task, you can might use the DAG start_date

Notes:

  • Took into consideration the downstream task to start after ALL upstream tasks done, if your rule is different, please adapt the logic
  • If there's more than one upstream task, use the latest end_date

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Saar Levy