Using Default or Custom Failure Handling¶
Airflow executors submit tasks to Qubole and keep track of them. These executors (task-instances) also register heartbeats with the Airflow database periodically. A task-instance is marked as zombie if it fails to register the heartbeat in a configured amount of time.
The Airflow scheduler checks for zombie processes in the system and if necessary invokes
the failure-handler for the task in question. Qubole ships a default failure-handler
with the Qubole operator; this checks for the corresponding Qubole command and kills
it if necessary. If the command succeeds, the handler changes the task-instance state to
You can override this behaviour by providing a custom failure-handler in the task definition, as shown in the following example:
def my_failure_handler(context): """ custom logic to handle command failures """ hive_task = QuboleOperator(task_id='hive_show_table', command_type='hivecmd', query='show tables', on_failure_callback=my_failure_handler, dag=dag)