Understanding Hive Versions

QDS supports the following versions of Hive:

  • Hive 1.2.0 (works with Tez 0.7.0 and Hadoop 2.6.0)
  • Hive 2.1.1 (works with Tez 0.8.4 and Hadoop 2.6.0)
  • Hive 2.3 (works with Tez 0.8.4 and Hadoop 2.6.0)
  • Hive 3.1.1 (beta) (works with Tez 0.9.1 and Hadoop 3.0)

Using Different Versions of Hive on QDS

The various ways of running Hive using these versions are described in Understanding Different Ways to Run Hive. Hadoop 2 clusters are also known as Hadoop 2 (Hive) clusters.

Note

You can configure the Pig version on an Hadoop 2 (Hive) cluster. Pig 0.11 is the default version. Pig 0.15 and Pig 0.17 (beta) are the other supported versions. You can also choose between MapReduce and Tez as the execution engine when you set the Pig 0.17 (beta) version. Pig 0.17 (beta) is only supported with Hive 2.1.1.

Use the different versions of Hive on QDS as described below:

Hive Version QDS Server Hive-on-Master HiveServer 2 Multi-instance HiveServer2
1.2.0 Default Running Hive on the Master Node Self-Serve Support Not Supported
2.1.1 Contact QDS Support to enable Default Self-Serve Support Contact QDS Support to enable
2.3 Default Running Hive on the Master Node Self-Serve Support Contact QDS Support to enable
3.1.1 (beta) N.A N.A Contact QDS Support to enable. When Hive 3.1.1 (beta) is enabled, it uses HiveServer2 by default. Contact QDS Support to enable

Note

  • LLAP from the Hive open source is not verified in Qubole’s Hive 2.1.1.
  • The /media/ephemeral0/hive1.2/metastore.properties file has been deleted from Hive 2.3 onwards. Remove the dependency on the metastore.properties file if you use this version. Hive 2.3 uses Java 8 while running on QDS servers. It is also compatible with Java 7.
  • Hive 3.1.1 (beta) uses Java 8.

To configure a version, select it from the Hive Version drop-down list when you configure a Hadoop 2 (Hive) cluster in QDS. Create a ticket with Qubole Support for an account-wide configuration. This enables all clusters in the account to use Hive on Master. For more information, see Enabling Hive on the Cluster Master.

To use Hive Server 2, enable Hive Server 2 under the Advanced Configuration tab of the Hadoop 2 (Hive) cluster. For more information, see Configuring a HiveServer2 Cluster. You can set the versions through the API as described in engine_config for Enabling HiveServer2 on a Hadoop 2 (Hive) Cluster.

To run Hive queries on Hive-on-master, refer to How to Enable Hive-on-Master. If Hive 2.x queries do not run on QDS servers by default, they run on the master by default.

If you enable Hive Server 2, Hive queries run through HiveServer2.