Qubole SchedulerΒΆ

The QDS scheduler allows you to configure jobs and specify the intervals at which they run, and provides additional capabilities that make it a powerful tool for automating your workflow. Job characteristics you can specify include:

  • The job type (Hive Query, Hadoop Job, etc.) and query parameters
  • The job start and end date and time, and its time zone and frequency
  • Javascript macros to be included
  • The Hadoop Fair Scheduler pool (Hadoop 1) or queue (Hadoop 2) to be used
  • The number of concurrent jobs to allow
  • Hive and AWS S3 dependencies, to ensure that the job runs only when the data it needs is available

For more information, see the following topics: