What is NewΒΆ

The new features and enhancements are listed in the corresponding tabs below.


The label (in blue) that is against the description indicates the launch state, availability, and the default state of the feature/enhancement. For more information, click the label.

Unless stated otherwise, features are generally available, available as self-service and enabled by default.

  • QDS has introduced a new public API which you can use to reset your own authentication token. Account administrators can use this to reset the authentication token of all other users in the account (including system-administrators).
  • Accounts are now sorted alphabetically in the Account drop-down list.
  • The Account Settings tab now displays concurrent command limits at the account and user levels.
  • QDS now displays cluster tags for deleted clusters.

Learn more.

  • The Master/Worker terminology naming convention is now used in the QDS UI instead of the Master/Slave terminology that was used before.
  • QDS now supports c5n, m5a, and r5a instances. Cluster Restart Required

Learn more about the complete list of changes.

  • To monitor the running DAGs on Airflow, Prometheus is integrated for real-time monitoring and dashboard using Grafana (Cluster Restart Required). Via Support, Disabled

    Learn more

  • You can now monitor the metrics with your own setup of Prometheus.

  • Any changes to the DAG Explorer will now immediately reflect on the Airflow cluster after the changes are saved. Cluster Restart Required

  • Qubole Airflow clusters now support the Airflow REST API.

  • Apache Airflow v1.10.0 is now supported on QDS. While creating an Airflow cluster, you can run the new Airflow version using the Airflow Version drop-down on the cluster UI. Learn more

  • The Sqoop version has been upgraded to 1.4.7 (Cluster Restart Required). Via Support, Disabled

Learn more about the complete list of changes.

Learn more about the complete list of changes.

Learn more about the complete list of changes.

  • Amazon S3 Select is now integrated with Spark on Qubole, and works for CSV and JSON data sources and tables. This integration improves TPCDS query performance. Via Support.

    Learn more.

  • Python UDF pushdown is optimized to improve join performance by pushing down UDF when the joined output is larger than individual tables. Learn more.

  • Multitenant Qubole Job History Server that serves log and history of Spark jobs run on terminated cluster is now upgraded to Spark 2.3. Learn more.
  • Spark on Qubole now supports Hive Admin commands to allow users to grant privileges such as (SELECT, UPDATE, INSERT and DELETE.) to other users or roles. Learn more.