QDS Components: Supported Versions and Cloud Platforms¶
The following table shows the currently supported open source versions of QDS Components, and the Cloud platforms on which they run.
Qubole is discontinuing support for HBase. You can no longer create HBase clusters.
|QDS Component||Currently Supported Versions||Supported on (Cloud Platforms)|
|Cascading||All||AWS, Azure, Oracle|
|Hadoop 2||2.6.0||AWS, Azure, Oracle|
|HBase||1.0||AWS (support being discontinued)|
||AWS, Azure, Oracle|
||AWS, Azure. Oracle for Spark only|
|2.6.0||AWS, Azure, Oracle|
|Presto||0.142 (to be deprecated), 0.157 (stable), and 0.174 (unstable)||AWS|
Airflow supports only 2.7 and later
See Can I use Python 2.7 for Hadoop tasks? for more information
|Scala||2.1.0 for Spark versions older than 2.0.0||AWS|
|2.1.1 for Spark 2.0.0 and later||AWS, Azure, Oracle|
|Spark: see Spark Version Support||1.3.1, 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 2.0.0, 2.0.2, 2.1.0||AWS|
|2.1.0, 2.0.2||Azure, Oracle|
|1.4.6||AWS, Azure, Oracle|
|Zeppelin (notebooks)||1.0||AWS, Azure, Oracle|
Spark Version Support¶
QDS supports two major versions of Spark, 1.x and 2.x. As of April 2017, Qubole began phasing out support for older versions, as explained in the sections that follow.
Explanation of Terms¶
Qubole will phase out support of older Spark versions in two stages: deprecation and sunset. Each Spark version can be in one of three states in terms of QDS support:
- Active: Fully supported; no current timeline for obsolescence.
- Deprecated: You can still launch clusters running this version, but:
- No new features or bug fixes will be applied to this version.
- The version will no longer be eligible for Qubole Customer Support; tickets will not be addressed.
- The version will sunset 90 days from the deprecation date.
- Sunset: The version is no longer available in the QDS UI, and no longer eligible for Qubole Customer Support. You can no longer launch clusters running this version. Qubole will make a best effort not to break compatibility with versions in this state, but compatibility is not guaranteed.
Spark 1.x Deprecation¶
For Spark major version 1.x, QDS is deprecating and will eventually sunset versions older than 1.6.2, starting April 14, 2017:
|Spark Versions||Deprecation Date||Sunset Date|
|1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1||April 14, 2017||July 31, 2017|