What’s New
Important new features and improvements are as follows.
Note
A link in blue text next to a description in these Release Notes indicates the launch state, availability, and default state of the item (for example, Beta). The link provides more information. Unless otherwise stated, features are generally available, available as self-service (without intervention by Qubole support), and enabled by default.
You can now enable and disable features at the account level through a self-service platform.
Qubole has updated its API throttling policy. Gradual Rollout
QDS now supports an account-level node bootstrap. Learn more. | Cluster Restart Required
QDS has made major improvements in API throttling policies.
QDS has improved its handling of cluster terminations. Learn more.| Gradual Rollout | Cluster Restart Required
Hive 1.2 is deprecated.
Any clusters still running Hive 0.13, which reached end-of-life in February 2019, will be automatically upgraded to 1.2 in QDS R59. You should consider upgrading to a later supported version as soon as possible.
Jupyter V2 notebooks provide Qviz with the Spark driver. Qviz allows you to visualize dataframes with improved charting options and Python plots. Gradual Rollout.
The Package Management UI has been redesigned with improvements. Users of existing accounts should contact Qubole Support to enable the new UI. Learn more.
Package Management now supports private Conda channels.
Dynamic Partition Pruning: Using Dynamic Filtering values, Dynamic Partition Pruning selects the specific partitions within the table that need to be read at runtime. This improves job performance for queries in which the join condition is on the partitioned column, by significantly reducing the amount of data read and processed. Dynamic Partition Pruning is available in Spark 2.4.3 and later versions. Gradual Rollout.