Written by Hariom Sharma & Thanigaivel Shanmugam
Apache Airflow is an open-source workflow management platform. Airflow allows you to programmatically author and schedule workflows and monitor them via the built-in Airflow user interface. Airflow is designed under the principle of “configuration as code”. Airflow uses directed acyclic graphs (DAGs) to manage workflow orchestration.
The standard UI of Airflow provides options to view the pipeline job status and filters. As the number of pipelines increases, the status and filters provided in the standard UI make it difficult to filter the information.
More number of pipelines = More difficult to monitor