QDS Components: Supported Versions and Cloud Platforms

The following table shows the currently supported open source versions of QDS Components, and the Cloud platforms on which they run.

QDS Component Currently Supported Versions Supported on (Cloud Platforms)
Airflow 1.7.0, 1.8.2 AWS
1.8.2 Azure, Oracle
Cascading All AWS, Azure, Oracle OCI
Hadoop 1 0.20.1 AWS
Hadoop 2 2.6.0 AWS, Azure, Oracle OCI
Hive 0.13.1, 1.2, 2.1.1 AWS
1.2, 2.1.1 Azure, Oracle OCI
Java
  • Hadoop 1 supports Java 1.7
AWS
  • Hadoop 2 and Spark support Java 1.7 as the default version, but 1.8 can be enabled through the node bootstrap.
AWS, Azure, Oracle OCI
  • Presto supports only 1.8
AWS, Azure
MapReduce 0.20.1 AWS
2.6.0 AWS, Azure, Oracle OCI
Pig 0.11, 0.15 AWS
0.11 Azure, Oracle OCI
Presto 0.142 (deprecated; only works on existing clusters without any configuration change), 0.157, 0.180, 0.193 (Beta) AWS, Azure
Python

2.6, 2.7

Airflow supports only 2.7 and later

AWS

See Can I use Python 2.7 for Hadoop tasks? for more information

2.7 Azure, Oracle OCI
R 3.3.3 AWS, Oracle OCI
3.3.2 Azure
RubiX 0.2.11 AWS
Scala 2.10 for Spark versions older than 2.0.0 AWS
2.11 for Spark 2.0.0 and later AWS, Azure, Oracle OCI
Spark: see Spark Version Support

Deprecated: 1.3.1, 1.4.0, 1.4.1, 1.5.0

Current Versions: 1.5.1, 1.6.0, 1.6.1, 1.6.2, 2.0.0, 2.0.2, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.3.1

AWS
2.1.0, 2.0.2, 2.2.1, 2.3.1 Azure, Oracle OCI

Note

In the Spark Version drop-down list on the Clusters page of the QDS UI, latest means the latest open-source maintenance version. If you choose latest, Qubole Spark is automatically upgraded when a new maintenance version is released. For example, if 2.x latest currently points to Spark 2.x.y, then when 2.x.(y+1) is released, QDS clusters running 2.x latest automatically start using 2.x.(y+1) on a cluster restart.

Sqoop 1.4.2 AWS
1.4.6 AWS, Azure, Oracle OCI
Tez 0.7 AWS, Azure, Oracle OCI
Zeppelin (notebooks) 0.6.2 AWS, Azure, Oracle OCI

Spark Version Support

QDS supports two major versions of Spark, 1.x and 2.x. As of April 2017, Qubole began phasing out support for older versions. Those versions are marked deprecated in the drop-down list of available versions on the Clusters page of the QDS UI. You can still launch clusters running a version marked deprecated, but:

  • No new features or bug fixes will be applied to this version.
  • The version will no longer be eligible for Qubole Customer Support; tickets will not be addressed.