Explore

New Feature

  • EAM-2348: For Airflow version 1.10.9, Qubole exports metrics via statsd. Prometheus scrapes metrics from the statsd and Qubole displays it on Grafana from there.

    Due to the change in way of exporting the metrics and the names of the metrics, a new Dashboard is introduced on Grafana for the Airflow version 1.10.9. For more information on the list of metrics provided by Airflow, see Metrics. Qubole includes some of the important metrics (for monitoring Airflow) to Grafana. For more information, see Monitoring an Airflow Cluster.

  • EAM-2212: Now, Qubole allows the users to customize the port where the Hive Metastore Server (HMS) runs on clusters. Previously, it used to run on port 10000 by default. Contact Qubole Support to customize the HMS port (Cluster Restart Required).

Enhancements

  • EAM-2299: Now, the users have an option on the Scheduler page to specify a time interval to run the scheduler. To enable this, the following two new fields are added:

    • Interval: The user needs to enter the numerical value for the time interval corresponding to the file path (as entered by the user in the File Location field).

    • Increment: The user needs to select a time unit for the value specified in the Interval field. Both these are mandatory fields if the user selects Cron expression as the Frequency.

    For more information, see Configuring S3/Azure Blob Storage Files Data Dependency.

Bug Fixes

There are no bug fixes in this version.