QDS Components: Supported Versions and Cloud Platforms¶
The following table shows the currently supported open source versions of QDS Components, and the Cloud platforms on which they run.
|QDS Component||Currently Supported Versions||Supported on (Cloud Platforms)|
|Airflow||1.7.0, 1.8.2, 1.10.0 (beta)||AWS, Azure, Oracle OCI, GCP|
|Cascading||All||AWS, Azure, Oracle OCI|
|Hadoop 1||0.20.1 (deprecated)||AWS|
|Hadoop 2||2.6.0||AWS, Azure, Oracle OCI, GCP|
|Hive||1.2, 2.1.1, 2.3 (beta)||AWS|
|1.2, 2.1.1||Azure, Oracle OCI|
||AWS, Azure, Oracle OCI|
|2.6.0||AWS, Azure, Oracle OCI, GCP|
|0.11||Azure, Oracle OCI|
|Presto||0.157, 0.180, 0.193, and 0.208 (beta)||AWS, Azure|
2.6, 2.7, and 3.5
Airflow supports only 2.7 and later
See Can I use Python 2.7 for Hadoop tasks? for more information
|2.7 and 3.5||Azure, Oracle OCI|
|3.6.x and 3.7||GCP|
|R||3.3.3||AWS, Oracle OCI|
|Scala||2.10 for Spark versions older than 2.0.0||AWS|
|2.11 for Spark 2.0.0 and later||AWS, Azure, Oracle OCI, GCP|
|1.6.2, 2.0.2, 2.1.1, 2.2.0, 2.2.1, 2.3.1, 2.4.0||AWS|
|2.0.2, 2.2.1, 2.3.1||Azure, Oracle OCI|
In the Spark Version drop-down list on the Clusters page of the
|1.4.6||AWS, Azure, Oracle OCI|
|Tez||0.7||AWS, Azure, Oracle OCI|
|Zeppelin (notebooks)||0.6.2||AWS, Azure, Oracle OCI, GCP|
The following table shows the deprecated versions of the engines and the corresponding timelines.
- Deprecation Timeline: Deprecated version is marked as (deprecated) on the UI. Existing clusters with this version function normally, however, new features and bug fixes are not available. It is not recommended for production use, therefore, you should upgrade the clusters to the later versions of engines. For assistance, contact Qubole Support.
- Removal Timeline: Version in the removal timeline is completely removed from the cluster image. Existing clusters with this version function when they are in running state. After the next restart, these clusters come up with the default version on the available version list.
|Engines on QDS||Deprecated Versions||Deprecation Timeline||Removal Timeline|
|Spark||1.3.1, 1.4.0, 1.4.1, 1.5.0||2017||Yet to be removed. Estimated release: R56|
|1.5.1, 1.6.0, 1.6.1, 2.0.0, 2.1.0||R54 (November 2018)||Yet to be removed. Estimated release: R56|
|Hive||0.13||Not supported for new customers. For legacy customers, support ended in R55.||Removed in R55|
|Presto||0.142||R52 (January 2018)||Yet to be removed. Estimated release: R56|
|0.157||R56||Yet to be removed. Estimated release: R57|
Spark Version Support¶
QDS supports two major versions of Spark, 1.x and 2.x. As of April 2017, Qubole began phasing out
support for older versions. Those versions are marked
deprecated in the drop-down list of available versions on the
Clusters page of the QDS UI. You can still launch clusters running a version marked
- No new features or bug fixes will be applied to this version.
- The version will no longer be eligible for Qubole Customer Support; tickets will not be addressed.
Hadoop 1 is Deprecated¶
Qubole has deprecated Hadoop 1 as-a-service. Qubole will support Hadoop 1 on the existing clusters until 31 December 2018. Creating a new Hadoop 1 cluster or cloning an existing Hadoop 1 cluster is not supported through the API and the UI.
If you are using Hadoop 1 clusters, then you must plan to migrate the workloads to Hadoop 2 clusters. Creating a new Hadoop 2 (Hive) cluster is one of the easiest ways to migrate the workload. As an alternative option, you can migrate the workloads to a Presto or a Spark cluster. For more information, contact the Qubole Support or the Customer Success Manager associated with the account.