Understanding Hive Versions
QDS supports the following versions of Hive:
Hive 1.2.0 (deprecated) (works with Tez 0.7.0 and Hadoop 2.6.0)
Hive 2.1.1 (works with Tez 0.8.4 and Hadoop 2.6.0)
Hive 2.3 (works with Tez 0.8.4 and Hadoop 2.6.0)
Hive 3.1.1 (beta) (works with Tez 0.9.1 and Hadoop 3.0)
Using Different Versions of Hive on QDS
The various ways of running Hive using these versions are described in Understanding Different Ways to Run Hive. Hadoop 2 clusters are also known as Hadoop 2 (Hive) clusters.
Note
You can configure the Pig version on an Hadoop 2 (Hive) cluster. Pig 0.11 is the default version. Pig 0.15 and Pig 0.17 (beta) are the other supported versions. You can also choose between MapReduce and Tez as the execution engine when you set the Pig 0.17 (beta) version. Pig 0.17 (beta) is only supported with Hive 2.1.1.
Use the different versions of Hive on QDS as described below:
Hive Version |
QDS Server |
Hive-on-Coordinator |
HiveServer 2 |
Multi-instance HiveServer2 |
|---|---|---|---|---|
(deprecated) |
Default |
Self-serve Support |
Not Supported |
|
2.1.1 |
Self-serve Support |
Default |
Self-serve Support |
Contact QDS Support to enable |
2.3 |
Default |
Self-serve Support |
Contact QDS Support to enable |
|
3.1.1 (beta) |
N.A |
N.A |
Contact QDS Support to enable. When Hive 3.1.1 (beta) is enabled, it uses HiveServer2 by default. |
Contact QDS Support to enable |
Note
LLAP from the Hive open source is not verified in Qubole’s Hive 2.1.1.
The
/media/ephemeral0/hive1.2/metastore.propertiesfile has been deleted from Hive 2.3 onwards. Remove the dependency on themetastore.propertiesfile if you use this version. Hive 2.3 uses Java 8 while running on QDS servers. It is also compatible with Java 7.Hive 3.1.1 (beta) uses Java 8.
To configure a version, select it from the Hive Version drop-down list when you configure a Hadoop 2 (Hive) cluster in QDS. Create a ticket with Qubole Support for an account-wide configuration. This enables all clusters in the account to use Hive on Coordinator. For more information, see Enabling Hive on the Cluster Coordinator.
To use Hive Server 2, enable Hive Server 2 under the Advanced Configuration tab of the Hadoop 2 (Hive) cluster. For more information, see Configuring a HiveServer2 Cluster. You can set the versions through the API as described in engine_config for Enabling HiveServer2 on a Hadoop 2 (Hive) Cluster.
To run Hive queries on Hive-on-coordinator, refer to How to Enable Hive-on-Coordinator. If Hive 2.x queries do not run on QDS servers by default, they run on the coordinator by default.
If you enable Hive Server 2, Hive queries run through HiveServer2.