Changelog for api.qubole.com¶
|Date and time of release||Version||Change type||Change|
|22nd Mar, 2020 (11:45 PM PST)||58.0.71||Enhancement||
PRES-3360: Qubole has added a Datadog alert to detect runaway splits occupying execution slots for more than 10 minutes.
PRES-3404: Qubole has improved utilization of dynamic filters on worker nodes and reduced load on coordinator when dynamic filtering is enabled.
PRES-3429: QDS Presto version 317 is generally available now.
PRES-3469: Qubole has backported OS fixes to improve performance of inequality JOINs that involves
PRES-2481: Qubole has increased the default value of
PRES-3108: Presto queries failed with denied authorization to Hive metastore. To fix this issue, Qubole has added
a new configuration property called
PRES-3403: The issue where a Presto query with the
|19th Mar, 2020 (04:27 AM PST)||58.0.70||Enhancement||AN-1430: When the number of columns in the Results pane is greater than 30, only the first 30 columns are rendered. You can use the Column drop-down list to select any (other) 30 columns to view. Additionally, the entire result set is available for download. Beta|
|Bug fix||AN-2510: While pinning custom buckets in Workbench, you can now pick a different region to list bucket contents. Beta|
|17th Mar, 2020 (03:18 AM PST)||58.0.68||Bug fix||JUPY-672: The Spark application status was not displayed in the Jupyter notebook panel menu. This issue is fixed.|
|12th Mar, 2020 (11:22 AM PST)||58.0.64||Enhancement||ZEP-4327: Precaching for Table Explorer (Datasets) is now available in Zeppelin notebooks. Via Support.|
ZEP-4336: Paragraphs failed with the
ZEP-4321:Table Explorer was consuming high amount of memory when loading a large number of tables. As a result, notebooks page was unresponsive. This issue is fixed.
ZEP-4290: Notebook command failures with the
ZEP-4401: The paragraph run time in the Notebooks UI was displayed incorrectly. This issue is fixed.
ZEP-4378: HTML tags were not rendered under markdown paragraph. This issue is fixed.
ZEP-4310: Jobs were failing due to high logging late in the data-driven log filtering tool. This issue is fixed by adding async processing queue for the data-driven log filtering tool.
|10th Mar, 2020 (07:56 AM PST)||58.0.61||Enhancement||
ACM-6564: Added the m5a, m5ad, r5a and r5ad instance types for all the supported regions.
ACM-6585: Added the c5.12xlarge and c5.24xlarge instance types for all the supported regions.
JUPY-667: In case of isolated mode, the livy session name is unique to facilitate multiple executions of a notebook in parallel.
JUPY-654: Jupyter notebook commands were being marked as failed although the notebooks ran successfully. This issue is fixed.
JUPY-659: The create, update, and rename APIs of Jupyter notebooks were not handling few special characters in name validation. This issue is fixed.
JUPY-665: Jupyter notebook execution failed when more than 10 notebooks were run from the scheduler in parallel. This issue is fixed.
|4th Mar, 2020 (01:12 PM PST)||58.0.58||Bug fix||
ACM-6397: The issue in which fetching engine configuration caused command failures is resolved now.
ACM-6515: Fixed a bug where a local node bootstrap file was deleted and hence it did not get executed.
ACM-6531: The issue where clusters with version R56 got terminated due to health check failure is resolved now.
QHIVE-5118: You can use the Beeline client to escape control characters in results with a HiveServer2 (HS2) cluster. For more information on the Beeline command shell, see the Apache documentation, HiveServer 2 Clients: Beeline - Command Line Shell.
When this feature is enabled, SELECT queries with the LIMIT clause less than or equal to 999 do not launch a Hadoop job on the cluster. You can get this feature enabled by contacting Qubole Support. Via Support
QHIVE-5194: Fixed the issue which caused an error where query results were displayed in the ASCII format when the Hive table data was in the Parquet format.
|25th Feb, 2020 (07:19 AM PST)||58.0.54||Bug fix||HADTWO-2367: Qubole has backported HDFS-11499 to fix the issue in HDFS where DFS clients could not close a file as its last block did not have sufficient number of replicas. For more details, see HDFS-11499 and HDFS-11486.|
|24th Feb, 2020 (06:20 AM PST)||58.0.52||Bug fix||
JUPY-621: The Spark progress widget in Jupyter Notebooks displayed a harmless exception sometimes. This issue is fixed.
JUPY-636: Python kernels were not running with the environment attached to the cluster. This issue is fixed.
|18th Feb, 2020 (01:07 AM PST)||58.0.49||Bug fix||
PRES-3200: Presto was not supporting Hash in Ranger with Presto 0.208. Now it supports Hash in Ranger with Presto 0.208.
PRES-3242: Parquet binary statistics generated before PARQUET-251 were corrupted. To resolve this, Presto ignores Parquet binary statistics generated before PARQUET-251 was fixed.
PRES-3371: The issue where the Presto QueryInfo for a query was not retrievable is resolved now.
PRES-3421: Presto has improved CBO logic to prefer broadcast join in some queries for better performance.
|13th Feb, 2020 (07:09 AM PST)||58.0.46||Bug fix||ZEP-4330: Clusters start-up time had increased for the accounts with proxy settings as packages were taking a long time to load. This issue is fixed.|
|11th Feb, 2020 (01:50 AM PST)||58.0.45||Bug fix||ZEP-4322: The changes in notebook paragraphs were not saved and the paragraphs were reverting to the previous state. This issue is fixed.|
|9th Feb, 2020 (07:54 AM PST)||58.0.43||Bug fix||ACM-6456: To avoid unavailability of spot nodes or any interruptions in spot nodes’ availability, set required IAM permissions
as defined in Additional Permissions. This ensures better fulfillment and resiliency in spot requests.
Qubole has restored the fallback to use
|7th Feb, 2020 (12:33 PM PST)||58.0.42||Bug fix||HADTWO-2413: Inline credentials were missing from output’s results when a user queried an s3 path (with inline credentials) through s3a filesystem. This issue has been fixed now.|
|6th Feb, 2020 (12:37 PM PST)||58.0.41||Bug fix||PRES-3419: Updating and pushing configuration operations failed for non-Presto clusters that were on R57 and earlier versions due to validation of the Presto version. This issue is resolved as non-Presto clusters do not validate the Presto versions now.|
|5th Feb, 2020 (5:41 PM PST)||58.0.40||Bug fix||PRES-3412: Fixed a bug in the Ranger plugin that occurred while using Ranger user groups with Presto 317 (beta).|
|3rd Feb, 2020 (11:13 PM PST)||58.0.38||R58 Upgrade (Phase 1 frontend)|
|30th Jan, 2020 (03:53 AM PST)||57.0.111||Bug fix||
AD-3111: Access to the get_creds API is now restricted to only the credentials of the default bucket.
PRES-3395: Fixed a bug that caused some users to not see logs for their Presto queries that failed fast (for example, due to syntax errors).
|20th Jan, 2020 (06:28 AM PST)||57.0.106||Enhancement||AN-2201: Workbench displays columns in the same order as that of the describe <table>. Clicking on Name sorts them in the ascending order. Beta|
|Bug fix||ZEP-4275: Downloading dependencies in notebooks failed because maven stopped support for http. This issue is fixed and now all the references to maven use https.|
|16th Jan, 2020 (03:17 AM PST)||57.0.103||Enhancement||
ACM-5876: Qubole now supports
ACM-6032: Qubole now supports
ACM-6165: Qubole now supports
QHIVE-5051: Fixed the issue when reading columns statistics fails for a column of date type in a partitioned table. Qubole has backported open-source fix, HIVE-20098 to all Hive versions.
QTEZ-497: Fixed the issue which caused very large Tez DAG submission to fail with
|10th Jan, 2020 (03:46 AM PST)||57.0.100||Enhancement||EAM-1801: Bastion Support for the snowflake data store is introduced so that the users can use bastion nodes to whitelist IP and ensure more security (Disabled, Via Support, Cluster Restart Required).|
|19th Dec, 2019 (02:59 AM PST)||57.0.98||Enhancement||
AD-471: While whitelisting an IP address, you can now add a description as required. The description can contain a maximum of 255 characters. It is supported through the UI and API as well. For more information, see Whitelisting IP Addresses and Add a Whitelisted IP Address.
AD-645: Qubole has introduced a new monthly Qubole Compute Unit (QCU) API and it has deprecated the old QCUH and Monthly Usage API. For more information, see View your Qubole Compute Unit (QCU) Usage.
PRES-3234: Presto 317 (beta) now supports AWS Glue metastore.
PRES-3157: Qubole has fixed the
PRES-3240: The optimization to finish join tasks early if their probe side is empty, has been removed as it deadlocks the query execution if new nodes join the cluster while the query is executing.
PRES-3282: Presto has added support for Lambdas in
|18th Dec, 2019 (11:15 PM PST)||57.0.94||Enhancement||
AN-2078: While selecting a cluster for a Hive, Presto, or Spark command, you can view its memory and CPU usage and its Hive metastore connectivity. Reviewing this information helps you make an informed decision on which cluster to choose for submitting commands. Beta
AN-1584: Workbench displays clusters sorted by Up, Pending, Terminating, and Down. Within each set, cluster labels are sorted alphabetically. Beta
AN-2348: Workbench now supports the read-only view for Spark notebook commands. You can use these to view logs, results, and resource links for the selected command. Beta
AN-2275: The command composer can now be resized for Spark and Shell commands. Beta
|Bug fix||AN-2261: Fixed an issue that sometimes caused the command preview text to not display properly if a keyword search was performed. Beta|
|2nd Dec, 2019 (12:35 AM PST)||57.0.81||Bug fix||ACM-6109: The issue in which the cluster start failed due to the cluster settings’ file upload failure to SSE-KMS enabled buckets has been resolved.|
|26th Nov, 2019 (11:40 PM PST)||57.0.80||Bug fix||ACM-6044: The issue in which using
|22nd Nov, 2019 (07:44 AM PST)||57.0.79||Bug fix||ACM-5991: The issue in which a cluster containing spot nodes in its composition returned
|21st Nov, 2019 (05:04 PM PST)||57.0.77||Enhancement||
AD-2805: You can now validate an account using AWS simulation. This helps in identifying any missing permissions.
ZEP-493: Bitbucket is integrated with notebooks. You can use Bitbucket to manage versions of your notebooks. Learn more.
|20th Nov, 2019 (02:21 AM PST)||57.0.72||Enhancement||
AIR-404: The users can now provide custom on_failure_callback for QuboleCheckOperator just like QuboleOperator. It allows users to define a custom callback if the QuboleCheckOperator fails (Cluster Restart Required).
QUEST-385: Users must create Spark Streaming clusters to run Spark structured streaming pipelines at scale with the Quest features.
QUEST-443: Users can use the new Filter option in the Pipelines view to filter the Pipelines based on their state.
QUEST-435, QUEST-378, QUEST-377, QUEST-403: Users can now edit, clone, archive, and delete pipelines depending on the State of the pipelines.
QUEST-406: Users can now modify name of the pipeline, when the pipeline is in the Draft state.
QUEST-405: Users can add Amazon S3 as a source when creating streaming pipelines.
QUEST-413, QUEST-454: Users can view the health of the streaming pipelines in the left pane of the Pipelines view.
QUEST-334: Users can fetch schemas from the Kafka cluster. For more information, see https://github.com/confluentinc/schema-registry.
|New feature||PRES-3070: Presto version 317 (beta) is now available on the Presto cluster.|
PRES-2990: Improved efficiency of dynamic partition pruning by preventing listing and creation of Hive splits from partitions, which are pruned at runtime.
PRES-3112: Qubole has introduced a feature to enable dynamic partition pruning on Hive tables at account level. This feature is part of Gradual Rollout.
PRES-3051: It fixes the
PRES-3113: Improved accounting of queued work for calculation of optimal size in autoscaling.
PRES-3131: Qubole has added
PRES-3163: It fixes a bug which could cause some additional nodes to be added to the cluster for a short duration during spot loss.
PRES-3187: It fixes a bug that caused the
|14th Nov, 2019 (11:40 AM PST)||57.0.64||Enhancement||ACM-5958: Qubole has done improvements in cluster UI page’s loading to resolve the slow-loading cluster UI page.|
ZEP-4081: Notebooks were failing due to jar conflicts with Spark. Thrift version used by Zeppelin is upgraded to 0.9.3 to fix this issue.
ZEP-4119: Python environment for the Notebooks was not getting configured properly. This issue is fixed now.
|24th Oct, 2019 (4:45 AM PST)||57.0.50||Bug fix||ACM-5798: This is a bug fix for warning notifications that had timestamp as timeout value. To resolve this, Qubole provides a configuration to control the warning notifications for a given cluster. To know more, see Configuring Query Runtime Settings.|
|21st Oct, 2019 (8:42 PM PST)||57.0.48||Bug fix||
ACM-5937: It is a bug fix for spot rebalancing that was being triggered incorrectly, which may result in addition of 10% more nodes than the maximum cluster size configuration.
HADTWO-2185: The issue where jobs were stuck after switching to the s3a filesystem has been resolved.
|10th Oct, 2019 (03:46 PM PST)||57.0.40||Bug fix||ACM:5866: The issue in which a cluster could not be started due to lack of disk space is resolved now. Qubole has increased the root disk size from 72GB to 90GB for all cluster types. This increases the cost by 1.8 USD per cluster node per month.|
|6th Oct, 2019 (11:13 PM PST)||57.0.35||R57 Upgrade (Phase 1 frontend)|
|30th Sep, 2019 (06:29 AM PST)||56.0.159||Enhancement||INFRA-2441: The default Account Level Concurrent Command Limit on the Account Settings tab has increased from 20 to 100.|
|27th Sep, 2019 (01:13 AM PST)||56.0.158||Enhancement||PRES-2791: Qubole has ported open-source changes that are related to improvements in S3 reads to Qubole Presto 0.208 version. For more information, see Faster S3 reads.|
PRES-2961: In an IAM-Role-based account, the issue where incorrect IAM-Role info had been sent to Presto has been resolved now.
PRES-2999: The NullPointerException when local memory limits are exceeded and a leak in operator peak memory computations in Presto version 0.208 queries have been resolved now.
PRES-3009: The issue where the Presto coordinator disk was filling up due to presence of RubiX logs in the autoscaling
log file has been resolved.
|24th Sep, 2019 (08:43 AM PST)||56.0.155||Bug fix||ACM-5664: Fixed the caching issue of the cluster list in the Analyze (Workbench (beta) UI page.|
|23rd Sep, 2019 (11:43 PM PST)||56.0.154||Bug fix||ACM-5616: Status checking of Spot requests has been made more robust by doing retries for DescribeSpotInstanceRequests in case of the RequestResourceCountExceeded error.|
|Bug fix||QHIVE-4798: Fixed an issue which can lead to a memory leak in HiveServer2 JVM when a large number of concurrent applications are running on a given cluster.|
|10th Sep, 2019 (12:59 AM PST)||56.6.11||Bug fix||ACM-5689: Fixed an issue where the default Hadoop cluster did not start during the AWS test drive signup.|
|03rd Sep, 2019 (06:08 AM PST)||56.1||Enhancement||
ACM-5266: Qubole supports
ACM-5413: Qubole supports
ACM-5558: During provisioning of nodes on a running heterogeneous cluster, Qubole tries providing instance types set in the heterogeneous configuration before falling back to On-Demand instances. For more information, see Additional Permissions.
ACM-5532: During the scrub run that is removing dead nodes for a cluster, there were nodes that were unreachable. This resulted in performance issues. To resolve this issue, Qubole terminates such nodes that cannot be connected. Via Support.
HADTWO-2098: In some cases, the scrub run did not remove dead nodes from a cluster. It happened as the scrub run connected to dead nodes as well through SSH to check on the node bootstrap completion. To resolve this issue, Qubole has improved the scrub run process, which does not connect to a dead node registered in the ResourceManager. This helps removing dead nodes that remained in the cluster and prevents such dead nodes from remaining in the cluster unlike before.
|22nd Aug, 2019 (10:41 PM PST)||56.0.120||Enhancement||JUPY-1: Qubole provides JupyterLab interface, which is the next generation user interface for Jupyter, to create and manage Jupyter notebooks. Jupyter notebooks are supported on Spark 2.2 and later versions. Beta, Via Support. Learn more.|
|21st Aug, 2019 (11:55 AM PST)||56.5.1||Enhancement||
AN-2168: New Analyze is now Workbench. Beta.
AN-1814: You can resize the command query composer in Workbench for Hive, Presto, Quantum, and DB Query commands. Beta.
AN-1327: To make debugging easier, Qubole now displays the Cluster Instance ID under the Processing tab of the Status pane. This enables you to collect logs of the particular command by cluster instance. Beta.
AN-2210: You can now tag commands on the History tab in Workbench. You can later use these tags to filter out commands using the Tags field (in the history filter). Beta.
|Bug fix||AN-2219: Resource links are now clickable in the Logs pane in Workbench. Clicking the link redirects the user to the corresponding cluster dashboard.|
|09 Aug, 2019 (6:41 AM PST)||56.0.112||Bug fix||AN-2240: The cluster selection drop-down list in Workbench now displays Hadoop2 clusters.|
|06 Aug, 2019 (03:36 AM PST)||56.0.108||Enhancement||AIR-390: Now Airflow is supported on New Package Management. New Package Management brings in features like Python 3.7, a new version of Conda, and a lot of fixes on package installation such as support for no-arch packages, and so on (Cluster Restart Required, Disabled, Via Support).|
|20th Aug, 2019 (4:43 AM PST)||56.0.117||Enhancement||ZEP-3373: Users can now edit the notebooks even when the clusters are offline. Via Support|
|Bug fix||ACM-5344: Qubole supports configuring proxy setting of Internet proxy server and no_proxy settings for cluster nodes. To override them, contact Qubole Support. Via Support|
|06 Aug, 2019 (3:36 AM PST)||56.0.108||Enhancement||
ZEP-2717 and ZEP-3602: The Environments UI is now available in the Control Panel by default for the new users.
Limitation: The packages that are installed by default cannot be uninstalled in new version of Package Management.
|31st July, 2019 (06:09 AM PST)||56.0.102||Enhancement||AD-2476: To mitigate risk while rolling out a new feature, Qubole groups users, and accounts into different pods. When a change is rolled out to a pod, Qubole monitors the feature’s performance before rolling it out to subsequent pods. You can view the pod you belong to in the Account Details section of the Account Settings tab.|
|30 July, 2019 (09:50 AM PST)||56.0.100||Enhancement||SCHED-376: SQL Command type has been renamed as Quantum in the Scheduler UI’s command type drop-down list.|
|29th July, 2019 (2:29 PM PST)||56.0.99||Bug fix||
QHIVE-4662: Fixed an issue which caused recursive listing while dropping partition(s) on a managed Hive table. Related OSS Jira: HIVE-22054.
QTEZ-443: Fixed an issue in which Tez UI was unable to download LevelDB file (that contains timeline data/log) for application when the IP address of a cluster node in a private subnet got repeated.
QTEZ-450: Fixed the issue in which Reducer was not visible in Tez UI when the configured Tez version was 0.8.4.
|19th July, 2019 (9:37 AM PST)||56.0.92||Enhancement||PRES-2918: A new experimental configuration property called
PRES-2746: S3 buckets were inaccessible with IAM Roles configured on the Qubole account. To resolve this issue,
Presto has added
PRES-2797: Fixed the issue in the generated Presto Query Tracker when Presto version was changed on an active Presto cluster.
PRES-2856: Fixed the issue in which command results were displayed without column headers for Presto queries when Qubole drivers executed such queries in Presto FastPath.
PRES-2915: Fixed the issue in which a Presto cluster with idle cluster timeout configuration did not automatically terminate even when it was idle for a longer time.
|9th August, 2019 (6:41 AM PST)||56.0.112||Bug fix||ACM-5171: Fixed the issue where a multi-instance HS2-enabled Hadoop (Hive) cluster, the multi-instance HS2 intermittently failed to start at the first attempt while waiting for the Hadoop (Hive) cluster.|
|4th July, 2019 (1:25 AM PST)||56.2.3||Enhancement||Hive version 2.3 is generally available.|
QHIVE-4385: To resolve FileNotFoundException while calculating FileSplits for the ORC file format, Qubole has added retries
in the Tez AM configuration. You can configure number of retries by using
|20th June, 2019 (4:51 AM PST)||56.2.1||Enhancement||ACM-4221 and ACM-5016: Qubole supports Hive 3.1.1 (beta) on a Hive cluster. Starting cluster API v2.1, Hadoop 2 (Hive) clusters are renamed as Hive clusters. You can set Hive 3.1.1 (beta) version while creating/editing a cluster. Qubole supports creating Hive clusters only from cluster API v2.1 onwards. Via Support|
|13th June, 2019 (11:48 AM PST)||56.0.76||Enhancement||
AN-1639: The Status pane on the new Analyze page is now available for Quantum commands.
AN-1708: Permalinks for Hive tables on the new Analyze page now contain account IDs. When you navigate to a Hive table from a different account, a confirmation dialog box appears.
QUEST-321: Users can now use custom code in Python language for creating a streaming pipeline.
QUEST-332: Users can select AVRO as one of the input formats when creating a streaming pipeline in assisted mode with Kafka as source.
QUEST-341: Users can add additional configurations such as user defined metadata for data written to S3 sink.
SPAR-3514: The Test Run option in the QuEST UI, processes a limited number of records and maintains separate temporary checkpoint location to prevent any corruption to the runtime production checkpoint.
SPAR-3591: Users can pass
AN-1974: Column headers now appear on the Analyze page for large Hive queries (approximately 65KB and more).
QUEST-340: When the users performed a test run with Kinesis as source and IAM role enabled in the account, the test run failed. This issue is now fixed.
QUEST-324: If the checkpoint location is not unique, the QuEST UI displayed an inaccurate notification when running the streaming pipeline in assisted mode. Now, the accurate notification is displayed in this case.
|10th June, 2019 (8:55 AM PST)||56.0.67||Enhancement||
AD-1629: Qubole now supports IAM role-based account creation via API for AWS. Learn more.
QHIVE-4527: Qubole will deprecate Hive notebooks in the near future.
AD-2554: This bug fix resolves the intermittent provided token has expired error for AWS.
AD-2441: The Usage Status Dashboard has been revamped.
QTEZ-440: It is the fix for the issue in which a Tez DAG got hung when exceptions were uncaught during a DAG transition.
|4th June, 2019 (11:07 PM PST)||56.0.63||Enhancement||AN-2100: All file downloads on the Explore page are now handled through AWS Signature Version 4.|
|3rd June, 2019 (1:39 AM PST)||56.0.61||Enhancement||
JDBC-124: Qubole now supports concurrency of multiple statements in Presto FastPath.
PRES-2254: In a Presto notebook, you can now set
PRES-2600: These are the new enhancements in the Presto notebooks:
PRES-2515: The issue which caused ArrayIndexOutOfBoundsException for some queries when using Dynamic Filtering is resolved now.
PRES-2727: This fixes the issue where queries with a table containing sub column names of a
PRES-2775: This fixes a bug which could prevent an upscaled Presto cluster from downscaling if short running queries are regularly scheduled on the cluster.
|24th May, 2019 (12:42 PM PST)||56.0.57||Bug fix||ZEP-3659: The newly added paragraphs were displayed only after refreshing the page. This issue is fixed.|
|20th May, 2019 (7:59 AM PST)||56.0.48||Enhancement||AN-1382: Click the preview icon to preview a query. You can also dock the query preview while you work on another query.|
|15th May, 2019 (11:44 PM PST)||56.0.45||New Feature||SQOOP-183: On Failure of DbImport/DbExport Commands running on a pixie cluster, the user can see failed MR (Map Reduce) job logs in the command logs section.|
|10th May, 2019 (10:14 AM PST)||56.0.42||Enhancement||QTEZ-441: Qubole has added an enhancement which when enabled adds an application tag containing the Qubole command ID to Tez jobs submitted through Hive-on-master and QDS servers. This tag cleans/kills applications for which corresponding queries are killed. Create a ticket with the Qubole Support to enable it on the QDS account.|
|9th May, 2019 (10:04 AM PST)||56.0.40||Bug fix||ACM-5035: This resolves a bug where accounts using different AWS Keys (Access and Secret) for storage and compute credentials
were not able to start the cluster due to the
|8th May, 2019 (5:59 PM PST)||56.0.38||Enhancement||PRES-2769:
|7th May, 2019 (2:28 AM PST)||56.0.31||R56 Upgrade (Phase 1 frontend)|
|20th May, 2019 (7:59 AM PST)||55.0.64||Enhancement||AN-1382: Click the preview icon to preview a query. You can also dock the query preview while you work on another query.|
|16th May, 2019 (9:37 AM PST)||55.0.62||Bug fix||HADTWO-1714: It fixes an issue in which YARN autoscaling leaks nodes if there are intermittent failures in terminating nodes from the cloud provider. After issuing a terminate node request, YARN autoscaling tracks nodes which could not be terminated and it tries terminating them again.|
|25th April, 2019 (9:14 AM PST)||55.0.50||Bug fix||AN-1974: The issue of the Analyze page not displaying results for large Hive queries (approximately 65KB and more) was resolved by falling back to object storage to retrieve query metadata.|
|23rd April, 2019 (8:45 AM PST)||55.0.49||New feature||ACM-4765: Qubole now supports m5ad and r5ad family instance types in US East (N. Virginia), US West (Oregon), US East (Ohio), and Asia Pacific (Singapore) AWS regions.|
|22nd April, 2019 (7:25 AM PST)||55.0.48||Bug fix||AN-2056: Uploading objects or files through the Qubole UI is now supported for all AWS regions. KMS-enabled buckets are also supported.|
|02nd April, 2019 (3:57 PM PST)||55.0.37||Enhancement||
PRES-1350: Qubole supports configuring the required number of worker nodes during autoscaling. It is a cluster configuration
PRES-2397: Qubole supports escaping newline
PRES-2417: Presto clusters do not terminate while actively running Presto notebook paragraphs. The enhancement is not available by default. Create a ticket with Qubole Support to use it.
PRES-2474: The optimization to speed up queries on
PRES-2521: Qubole has added a cluster configuration property,
PRES-1373: This is a fix for an issue where the retry operation was not working on the failed Presto queries. Retries work on the failed Presto queries now. As part of the fix, retries are configurable for Presto queries that run on the QDS platform.
Configuring retries will just do a blind retry of a Presto query. This may lead to data corruption for non-Insert Overwrite Directory (IOD) queries.
PRES-2306: Fixed a bug where the Usage page on the QDS UI did not show the Presto query’s bytes read statistics.
PRES-2493: Fixed the data loss issue in the Hive connector when writing bucketed sorted table in Qubole Presto 0.208 queries.
PRES-2567: Fixed a bug in the user-overridden IAM Role feature where stale credentials were being used in a long-running cluster.
PRES-2571: Fixed a concurrency related issue in autoscaling on Qubole Presto 0.193 and 0.208 versions due to which the cluster’s size permanently exceeds its maximum size.
|01st April, 2019 (3:20 AM PST)||55.0.36||Bug fix||
QHIVE-3740: The Hive metastore API call has been replaced with a less expensive API call to resolve the issue of false alerts that were triggered on Datadog.
QHIVE-4040: It is a fix for an issue in
QHIVE-4053: It is a fix for an issue where the
QHIVE-4280: Hive jobs were failing with a Metaexception that included a timeout message. To resolve this issue, the SSH configuration parameters’ maximum value is increased on QDS servers.
QTEZ-335: It resolves an issue where the Tez Offline UI was inaccessible when
|26th March, 2019 (8:25 AM PST)||55.0.31||Bug fix||SPAR-3391: Some paragraphs ran successfully but with a warning message during the first run. However, those paragraphs ran successfully without the warning on a re-run. This issue is fixed.|
|22nd March, 2019 (6:27 AM PST)||55.0.29||Enhancement||
ACM-4112: The Spot Request Timeout option in the Cluster Composition UI tab and
ACM-4259: Qubole supports the
ACM-4426: Qubole supports
|22nd March, 2019 (6:27 AM PST)||55.0.29||Bug fix||TOOLS-1139: The previous docker version had security vulnerabilities, which could allow malicious containers to gain root-level
privileges on the host. To resolve security vulnerabilities of the previous docker version, the latest patch version of
|14th March, 2019 (9:59 PM PST)||55.0.23||Bug fix||PRES-2635: The earlier fix for improving planning time of queries which use CONCAT function in the WHERE clause on a partitioned
column introduced a degradation in execution time of such queries in some cases. This bug fix introduces the optimization which
is disabled by default. Enable
|13th March, 2019 (8:37 AM PST)||55.0.22||Enhancement||SPAR-3005: For the offline Spark clusters, only the event log files that are less than 400 MB are processed in the offline Spark History Server (SHS). This prevents high CPU utilization on the internal servers due to SHS.|
|Bug fix||SPAR-3371: Dynamic filtering now works for join keys of type Int, String, Long, and Short.|
|13th March, 2019 (8:37 AM PST)||55.0.22||Bug fix||
ZEP-3247: Zeppelin failed to start due to the
ZEP-3252: On a cluster restart, the latest dashboards are displayed after the scheduled runs.
ZEP-3230: Zeppelin server in Java 7 clusters was failing due to some ciphers that were disabled for security. Now, the Zeppelin server starts successfully as the required ciphers are added when starting the Zeppelin server.
ZEP-3243: A notebook with the ERROR status on a cluster failed to associate back to the cluster even after switching to another cluster. This issue is fixed.
ZEP-3234: Notebooks failed to load in case of higher latencies when fetching notebook permissions. This issue is fixed.
|11th March, 2019 (8:55 AM PST)||55.0.17||New feature||EAM-1632: A new API is introduced to retrieve the Hive table partitions and locations. For more information, see View Table Partitions and Location.|
|18th February, 2019 (12:12 AM PST)||55.0.11||Enhancement||QHIVE-4065: Qubole tmp tables can be dropped faster if you enable the
|5th February, 2019 (4:35 AM PST)||55.0.5||R55 Upgrade (Phase 1 frontend)|
|4th February, 2019 (1:25 PM PST)||54.0.64||Bug fix||AD-2051: Fixed an issue where sample data for Hive tables on the Explore page took a long time to appear due to a misconfiguration.|
|28th January, 2019 (9:58 AM PST)||54.0.59||Bug fix||QHIVE-3878: Fixed the issue when concurrently running the
|15th January, 2019 (9:04 AM PST)||54.0.53||Bug fix||SCHED-321: In the Cron expression, the scheduler calculates the Next Materialized Time (NMT)/Start time considering the current time as the base time and Cron expression passed. Start time is not honored in the Cron expression. It resolves the issue where the first instance of the scheduler did not honor the scheduled time.|
|11th January, 2019 (1:00 PM PST)||54.0.52||Bug fix||AN-1407: While filtering commands on the Analyze page, you must now provide the date range (start/end date) when searching for cluster labels.|
|7th January, 2019 (1:30 AM PST)||54.0.46||Enhancement||AD-1834: The list of email IDs configured in Account Settings will now receive notifications related to releases, account configuration, and feature changes.|
|19th December, 2018 (12:57 PM PST)||54.0.44||Bug fix||SPAR-3207: The Spark shuffle cleanup feature is not supported in Spark 2.2.1. However, the Spark shuffle cleanup feature continues to be supported in Spark 2.3.1 and later versions.|
|19th December, 2018 (12:47 PM PST)||54.0.44||Enhancement||
ACM-3833: Qubole now supports saving the node bootstrap on SSE-KMS encrypted buckets.
ACM-3891: As part of a specific cluster’s governance, the master node is not counted in the node time chart, that is node chart only shows worker nodes.
|Bug fix||ACM-3847: The issue where the cluster did not start from the Notebooks UI when global permissions were restrictive while the object-level ACL permissions for the same cluster were allowed, is now resolved.|
|17th December, 2018 (9:22 AM PST)||54.0.41||Bug fix||ZEP-3109: Fixed the issue that caused notebooks to freeze frequently due to the huge content, such as large number of clusters, users, and schemas.|
|13th December, 2018 (1:39 PM PST)||54.0.40||Bug fix||RUB-102: It is a fix to avoid unnecessary HTTP GET calls to Ganglia for fetching the cluster metrics due to which the master node was overloaded.|
|11th December, 2018 (12:57 AM PST)||54.0.39||Enhancement||
ACM-3436: QDS now supports c5n, m5a, and r5a instances.
ACM-3545: Qubole now provides a feature to avoid the cluster from starting whenever the master node’s Elastic IP address fails. This feature is not enabled by default. Contact Qubole Support to enable this feature.
|6th December, 2018 (5:29 AM PST)||54.0.38||Bug fix||EAM-1502: Fixed an issue in the Automatic Statistics Collection framework due to which the fresh statistics were not collected for a few tables.|
|28th November, 2018 (6:19 PM PST)||54.0.33||Enhancement||PRES-2380: Presto notebooks are GA and Presto 0.208 (beta) version is available for use.|
PRES-2338: Fixed a bug in Presto 0.180 and 0.193 to adhere to fallback to the On-Demand configuration in case of Spot loss.
PRES-2268: Fixed a bug in Presto 0.193 which caused SELECT queries with THE ROW type output to fail.
PRES-2269: Fixed a bug in Presto 0.193 due to which queries on Hive views from Presto were failing.
|26th November, 2018 (1:20 AM PST)||54.0.30||Enhancement||
SPAR-3151: Qubole now supports the latest Apache Spark 2.4.0 version. It is displayed as 2.4 latest (2.4.0) in the Spark Version field of the Create New Cluster page on QDS UI.
|26th November, 2018 (1:20 AM PST)||54.0.30||Bug fix||ACM-3893: The command failure due to an s3exception is resolved by setting up correct credentials in environment variables.|
|21st November, 2018 (1:29 AM PST)||54.0.26||Bug fix||
AD-1752: Bug fix to restrict the Service user from logging in via Gmail or SAML.
AD-1763: Bug fix to enable updating the feature status in an internal table.
|19th November, 2018 (7:23 AM PST)||54.0.23||Bug fix||ACM-3857: Fixed the incorrect volatile node count calculation in clusters with heterogeneous instance configuration enabled.|
|15th November, 2018 (12:19 PM PST)||54.0.22||Enhancement||TOOLS-906: Given the recent addition of new packages and package upgrades to clusters, increased the node root volume capacity from 45GB to 60GB as a defensive measure.|
|15th November, 2018 (3:05 AM PST)||54.0.21||Bug fix||EAM-1476: Fixed the error for the hive table_properties API to return the correct response code.|
|14th November, 2018 (7:04 PM PST)||54.0.20||Bug fix||SPAR-3160: Fixed the issue where role-based Spark clusters using the Native Amazon S3 FS did not get the Node Bootstrap correctly while bringing up the cluster.|
|14th November, 2018 (6:56 PM PST)||54.0.17||Bug fix||
AD-1739: Fixed the configuration error that caused our systems to incorrectly determine the status of customer payment plans.
AD-1742: Fixed the error that caused
|13th November, 2018 (1:35 AM PST)||54.0.15||Major release|