Airflow

New Features

  • GIT is now integrated with Airflow clusters through the DAG explorer.

  • AIR-523: The Jupyter Notebook Command is now available in QuboleOperator as jupytercmd; users can schedule their Jupyter Notebooks in Airflow (Cluster Restart Required).

  • AIR-500: Provides support for Apache Airflow version 1.10.9QDS. This version is supported only with Python version 3.7.

    The previous supported version was 1.10.2QDS, and there have been many new features and improvements since. This new version includes improvements such as the newly added GcsToGDriveOperator, support for optional configurations while triggering DAGs, configurable tags on DAGs, persistent serialised DAGs for webserver scalability, and so on. For more information on the new features and improvements, see the Apache Airflow Changelog.