13. Why are so many idle mappers seen on an Hadoop Cluster?

This is because the number of mappers does not change until a job completes.

This is not a problem in itself, but problems can arise that are usually related to the Task Limits. Check the hard limit for total tasks, which is based on the Job Tracker’s maximum heap size settings.

Possible problems include:

  • The JobTracker is hitting OOM (out-of-memory) errors because there are too many tasks.
  • The JobTracker cannot schedule more mappers because too many jobs or tasks are already running.
  • A big job is never scheduled because it requires more mappers and reducers than the total task limit allows.