Hive

Hive 2.1 is Generally Available

Hive 2.1 is now generally available. Cluster Restart Required

HAProxy on the Cluster Coordinator Node

INFRA-1016 and INFRA-603: QDS now uses HAProxy on the cluster Coordinator node to balance the load when there are multiple connections between the cluster and a QDS-managed Hive Metastore. This removes a single point of failure and provides more stability. Via Support

If you are using a Bastion node, you must allow access on ports 20001-20005, in addition to 7000, for incoming TCP traffic from the cluster Coordinatorr node.

Improvements

  • QHIVE-1633: The logs for Hive Tez jobs display split computation, progress state, and completion state.

  • QHIVE-3727: QDS sets the value of hive.metastore.drop.partitions.batch.size to 1000, so partitions are dropped in batches of 1000. You can change this default.

Bug Fixes

  • QHIVE-3145: Hive now supports special characters in the metastore password when hive.use_encoded_metastore_password is enabled.

  • QHIVE-3582: Pruning columns resulted in incorrect sequencing of columns in SelectOperator; this problem has been resolved.

  • QHIVE-3535: Improves startup time for Hive clusters by bringing up daemons in parallel.

  • QHIVE-3641: The container memory for Reduce tasks is determined by mapreduce.reduce.memory.mb rather than mapreduce.map.memory.mb when Tez is the execution engine. This resolves an issue that caused overriding default Map or Reduce tasks at the Hive query level to fail.

  • QTEZ-345: The Tez application UI is now accessible when the node’s hostname is unresolvable and a MapReduce job is running.