Hive

Changes in Hive provides more description on the new features/enhancements.

New Features

  • HIVE-1880: With this release, Qubole supports external authentication to use HiveServer2 in Hive 2.1.

Bug Fixes

  • HIVE-2655: The issue in which HiveServer2 failed with intermittent error - Illegal Operation state transition from CLOSED to ERROR - has been resolved.

  • HIVE-2671: Avro schema caching has been disabled, md5sum will be refreshed every time the schema file is accessed.

  • HIVE-2762: The issue in which a Hive query had an intermittent failure, has been resolved.

  • HIVE-2865: Qubole Hive has added hive.security.authorization.enabled, a configuration parameter to the restricted list to avoid a user from setting it during the run time.

    To disable hive.security.authorization.enabled that is set at the account level (when HiveServer2 is enabled on the cluster), you can override it in Hive Settings > Override Hive Configuration under the Advanced Configuration tab of the cluster UI. It is applicable only to Hive 1.2 version. It will be supported on Hive 2.1 shortly.

  • HIVE-2879: The custom metastore connection issues have been resolved.

  • HIVE-2943: Qubole Hive has increased the protobuf message size from 64MB to 1GB for reading ORC file footers/metadata.

  • HIVE-3030: Tez can now handle a scenario where a single key-value pair size does not fit into any of the buffer blocks. This resolved the issue with Hive query failing with BufferOverFlowError when Tez is used.

  • HIVE-3032: The issues where Insert Overwrites were failing when hive.allow.move.on.s3=true and table location contains = has been resolved.

    This issue occurred in cases where table location is longer/smaller than the partition path generated (dynamically when hive.allow.move.on.s3=true) and as a result it was producing invalid partition specs.

  • HIVE-3035: A dynamic write to a partitioned table fails if a partition already exists with some custom location. Making this case pass by skipping the exception block if hive.qubole.dynpart.move.check is set to false.

  • HIVE-3041: Thrift metastore exceptions which were leading to to grant/create/drop commands to hang (in Hive 2.1.1) and fail with null error (in Hive1.2) when the object already exists in the metastore are resolved.

  • HIVE-3042: The issue where a Hive query can get stuck during job submission has been resolved.

  • HIVE-3063: Fixed an issue causing memory leak when reading from a compressed file using native library. Related OpenSource Hadoop JIRA - HADOOP-14376.

  • HIVE-3133: This is a fix to handle empty vector column batches in the float and double reader when vectorised ORC reader is used.

  • HIVE-3166: The issue that occured while reading a Hive bootstrap file has been fixed.

  • HIVE-3172: Fixed the NullPointerException in the HiveServer2 webUI Historical SQL Operations section.

  • QTEZ-261: The issue in which a Hive-on-Tez job displayed a duplicate entry in the output has been resolved.

  • QTEZ-265: This is a fix for repairing the levelDB when it encounters missing/corrupted-level DB state files to avoid the failure in starting TimeLineServer.

Enhancements

  • HIVE-2126: Logs for dropping tables are available now.

  • HIVE-2884: Qubole Hive has now added alerts and dashboards in Datadog for HiveServer2 Metrics.

  • HIVE-2945: Qubole Hive has upgraded OpenX JSON SerDe to 1.3.8-qds-0.3.0.

  • HIVE-3167: Fixed a race condition which caused queries to fail with State transition Exception. Related Open Source JIRAs: HIVE-15563 and HIVE-17352.