HMS Total Memory Used > 90%¶
This runbook shows steps to perform upon receiving an alert that the Hive Metastore Server’s memory usage is above 90%.
- Check the dashbord HMS Memory Usage. Specifically look for a trend in the metric
hive.hms.memory.pools_CMS-Perm-Gen_usage, along with heap trends.
- Restart the HMS process in the interim if failures are not contained.
- HMS logs are available on the coordinator node: “/media/ephemeral0/logs/hive1.2/hive_ms.log”
- Look for any evident errors(do basic grep and count of errors).
- Look at the dashboards defined above. (“title” is the name of the dashboard).
Restart of Process¶
sudo monit summaryto check the status of the process.
sudo monit stop metastore1_2to stop the process.
sudo monit start metastore1_2to start the process.