Understanding the HiveServer2 Metrics

HiveServer2 is available on Hadoop2 (Hive) clusters. You can enable HS2 on a Hadoop 2 (Hive) cluster as described in Configuring a HiveServer2 Cluster.

QDS supports Datadog monitoring for HS2. You can configure the Datadog monitoring service at the cluster level as described in Advanced configuration: Modifying Cluster Monitoring Settings.

For more information on configuring the Datadog monitoring service at the account level in Control Panel > Account Settings, see Configuring your Access Settings using IAM Keys or Managing Roles.

Qubole also provides a default dashboard on Datadog and alerts to monitor HS2 metrics. If you want to customize the threshold values or alerts about other metrics, you can set such alerts/values. For information on how to create alerts and configure email notifications, see the Datadog Alerts description.

This section describes:

• Metrics associated with the Query Lifecycle Execution

• Metrics associated with Active Queries

• Metrics associated with Execution Engines

• Metrics associated with HS2 Sessions

• Metrics associated with the Memory

• Metrics associated with the Garbage Collection

• Default Dashboard for HS2 Metrics

Metrics associated with the Query Lifecycle Execution

Metric	Description
hive.hs2.waiting_compile_ops.count	It denotes the number of queries waiting to be compiled on HS2.
hive.hs2.submitted_queries.count	It denotes the number of queries submitted on HS2.
hive.hs2.compiling_queries.count	It denotes the number of queries being compiled on HS2.
hive.hs2.executing_queries.count	It denotes the number of queries being executed on HS2.
hive.hs2.failed_queries.count	It denotes the number of queries failed on HS2.
hive.hs2.succeeded_queries.count	It denotes the number of queries succeeded on HS2.

Metrics associated with Active Queries

Metric	Description
active_calls_hive.hs2.executing_queries	It denotes the number of queries currently executed on HS2.
active_calls_hive.hs2.compiling_queries	It denotes the number of queries currently compiled on HS2.
active_calls_hive.hs2.submitted_queries	It denotes the number of queries currently submitted on HS2.

Metrics associated with Execution Engines

Metric	Description
hive.hs2.mapred_tasks.count	It denotes the number of queries that have run with MapReduce as the execution engine.
hive.hs2.tez_tasks.count	It denotes the number of queries that have run with Tez as the execution engine.

Metrics associated with HS2 Sessions

Metric	Description
hive.hs2.session.open_sessions	It denotes the number of open sessions on HS2.
hive.hs2.session.active_sessions	It denotes the number of active sessions on HS2.
hive.hs2.session.avg_open_session_time	It denotes the average session time of open sessions on HS2.
hive.hs2.session.avg_active_session_time	It denotes the average session time of active sessions on HS2.

Metrics associated with the Memory

Metric	Description
hive.hs2.memory.total_used	It denotes the total memory used by HS2.
hive.hs2.memory.heap_used	It denotes the total heap memory used by HS2.
hive.hs2.memory.pools_CMS-Perm-Gen_usage	It denotes the total Permanent Generation (PermGen) used by HS2.

Metrics associated with the Garbage Collection

Metric	Description
hive.hs2.gc.ConcurrentMarkSweep_count	It denotes the number of ConcurrentMarkSweep GC events.
hive.hs2.gc.ConcurrentMarkSweep_time	It denotes the time taken by ConcurrentMarkSweep GC events.
hive.hs2.gc.ParNew_count	It denotes the number of ParNew GC events.
hive.hs2.gc.ParNew_time	It denotes the time taken by ParNew GC events.

Default Dashboard for HS2 Metrics

The default dashboard that is available on Datadog contains these HS2 metrics:

• HS2 GC Time

• HS2 Memory Usage

• Active Queries

• HS2 Session Details

• Query Counters

Here is a sample of the default dashboard.