Upcoming Engine Versions Deprecation and Expiry FactSheet¶
The following section provides upcoming engine versions’ deprecation and expiry factsheet:
Hive Version 1.2 Deprecation¶
Hive 1.2 has been live on QDS since 2015. The lifecycle of a supported version on a data platform usually lasts 24 months. With this perspective, Qubole will deprecate Hive 1.2 from March, 2020.
Open source Hive has stopped support of Hive 1.2 in these aspects:
- No new features are ported into Hive version 1.2
- No security patches are available
Qubole has added support for Hive versions 2.1.1, 2.3, and the latest 3.1.1 (beta) version of Hive.
With the end of support of Hive 1.2 in 6 months, Qubole urges its users to move to higher Hive versions (preferably Hive 2.3) as appropriate.
New Features supported in Hive Version 2.3¶
Features that are supported in Hive version 2.3 are:
- Vectorized Query Execution
- Predicate Pushdown for Parquet
- Dynamic Runtime Filtering
- Better scalability with Application TimelineServer 1.5
- Multi-Instance HiveServer2
- CBO Improvements
- Support for Apache Ranger & AWS Glue
- Native JDBC Storage Handler
- Support for GROUPING function
- HiveServer2 UI
- Significantly better performance with fewer resources
For migrating from Hive version 1.2 to Hive version 2.3, perform the following steps:
- Clone Hadoop (Hive) clusters configured with Hive version 1.2.
- Change the Hive version to 2.3 in such clusters.
- Test accounts with the new Hive version.
To know more, see Understanding Hive Versions.
FAQs around Hive 1.2 Deprecation¶
Would Hive 1.2 queries run in later Hive versions supported by Qubole?
Qubole-introduced features will continue to work as 1.2, however, OSS features will behave as Apache Hive 2.3. You must own the responsibility of testing of queries compatibility from Hive 1.2 to higher versions.
Does Hive metastore DB Schema require an upgrade?
Qubole will take care of upgrading the schema of Hive metastore DBs managed by Qubole and recommends customers to upgrade the custom Metastore DB.
Should I upgrade the custom thrift server (if in use) ?
Yes, as Hive versions 2.1.1/2.3 client does not support Hive 1.2 HMS. You should upgrade the custom thrift server.
Does it impact a Spark/Presto user, who is using custom Hive 1.2 Metastore Schema or Thrift Server?
No, it does not impact a Spark/Presto user, who is using custom Hive 1.2 Metastore schema or thrift server.
What is exactly impacted by Hive 1.2 deprecation?
Hive 1.2 deprecation only impacts Hadoop (Hive) clusters. Qubole does not intend to deprecate use of Hive 1.2 schema in custom Hive metastores.
Currently I am using Hive on MapReduce, should I move to Tez?
Though Hive on MapReduce is supported in Qubole Hive 2.1.1 / 2.3, it is deprecated in Open Source Hive. Qubole strongly recommends moving to Tez.
This will also set the path for Hive 3 upgrade as MapReduce is deprecated in Qubole Hive 3.1.1 (beta).
Is it possible to move to Hive 3.1.1 directly?
Currently, Hive 3.1.1, Hadoop 3, and Tez 0.9.1 are in the beta phase. So Qubole recommends moving a test/staging workload first to avoid unforeseen issues.