Airflow

New Features

  • AIR-95: Logs for Airflow $AIRFLOW_HOME/logs have been moved to /media/ephemeral.

  • AIR-97: Allows you to select an Airflow data store other than the default local store.

  • AIR-98: A new Delete button is available in the Airflow UI to delete a DAG. The CLI support for deleting a DAG has been removed.

  • AIR-99: QDS now allows you to enable an email notification to alert you if an Airflow cluster that had active DAG runs in the past has been idle for over a week. If there are no active runs in the database, you’ll receive the notification daily.

Bug Fixes

  • AIR-101: To receive email alerts related to Airflow processes, enable _alert_via_email_ in the airflow.cfg file. Email alerts are sent when the Airflow’s web server, scheduler, rabbitmq, or worker process stops responding.

Enhancements

  • AIR-69: The UI errors are fixed (missing toggle button and DAG run info when the number of DAGs in the account is more than 25).