Cluster Management

Datadog

ACM-5566: Datadog is now available in Beta for Azure clusters. On the Clusters page in the QDS UI, choose the Advanced Configuration tab of an existing or new cluster, and enter your Datadog API and Application keys. You can also enable Datadog at the QDS account level; use the Account Settings page in the Control Panel of the QDS UI. Settings for an individual cluster override the account settings. See also:

• Presto Metrics on the Default Datadog Dashboard
• Understanding the Presto Metrics for Monitoring
• Spark Metrics on the Default Datadog Dashboard
• Understanding the HiveServer2 Metrics
• Zeppelin Metrics on the Datadog UI

Integration with Azure Pre-Provisioned VM

QDS is integrated with the Azure Pre-Provisioned VM (virtual machine) service. This new Azure service pre-provisions a pool of VMs per region on the basis of usage. QDS integration with this service improves consistency in cluster start and autoscaling times.

Account-level Node Bootstrap

ACM-6680: QDS now supports an account-level node bootstrap script, which is executed for all clusters in an account. Currently, it is supported only through a REST API. Cluster Restart Required

Handling Cluster Terminations

ACM-6255: Improves handling of cluster terminations, including terminating unhealthy or idle clusters and terminating a cluster when its start command times out. Gradual Rollout | Cluster Restart Required

Enhancements

• ACM-5560: Improves error messaging to display the email address of a user who terminates the cluster manually.
• ACM-6297: New HiveServer2 clusters can now use the associated Hive cluster’s label.
• ACM-6301: Reduces the number of API calls made to Azure, reducing throttling and ensuring proper QDS autoscaling under heavy load.
• ACM-6319: Improves the API call that lists cluster states.

Bug Fix

• ACM-6291: Fixes a problem that caused terminated clusters to be incorrectly marked as up. This was due to a race condition between cluster termination and background healing jobs.