HAProxy on the Cluster Master Node¶
INFRA-1016 and INFRA-603: QDS now uses HAProxy on the cluster Master node to balance the load when there are multiple connections between the cluster and a QDS-managed Hive Metastore. This removes a single point of failure and provides more stability. Via Support
If you are using a Bastion node, you must allow access on ports 20001-20005, in addition to 7000, for incoming TCP traffic from the cluster Master node.
- QHIVE-1633: The logs for Hive Tez jobs display split computation, progress state, and completion state.
- QHIVE-3727: QDS sets the value of
hive.metastore.drop.partitions.batch.sizeto 1000, so partitions are dropped in batches of 1000. You can change this default.
- QHIVE-3145: Hive now supports special characters in the metastore password when
- QHIVE-3535: Improves startup time for Hive clusters by bringing up daemons in parallel.
- QHIVE-3582: Pruning columns resulted in incorrect sequencing of columns in
SelectOperator; this problem has been resolved.
- QHIVE-3641: The container memory for Reduce tasks is determined by
mapreduce.map.memory.mbwhen Tez is the execution engine. This resolves an issue that caused overriding default Map or Reduce tasks at the Hive query level to fail.
- QTEZ-345: The Tez application UI is now accessible when the node’s hostname is unresolvable and a MapReduce job is running.