Mapping of Cluster and Command TypesΒΆ

The following table shows which types of commands and queries run on each type of Qubole cluster.

Note

To see which Cloud platforms support each type of cluster, and for version information, see QDS Components: Supported Versions and Cloud Platforms.

Qubole has deprecated Hadoop 1 as-a-service. For more information, see Hadoop 1 is Deprecated.

Command Type Airflow Cluster Support Hadoop-1 Cluster Support Hadoop-2 Cluster Support Presto Cluster Support Spark Cluster Support
Data Export Not Applicable Optional Optional Not Applicable Not Applicable
Data Import Not Applicable Optional Optional Not Applicable Not Applicable
DB Query Not Applicable Not Applicable Not Applicable Not Applicable Not Applicable
Hadoop command No Yes Yes Yes No
Hive query No Yes Yes Yes No
Pig query No Yes Yes Yes No
Presto query No No No Yes No
Query Export No Yes Yes Yes No
Redshift command Not Applicable Not Applicable Not Applicable Not Applicable Not Applicable
Refresh Table command Not Applicable Not Applicable Not Applicable Not Applicable Not Applicable
Shell command Yes Yes Yes Yes Yes
Spark command No No No No Yes

Note

Cluster support for the workflow command depends on the type of commands in the workflow. DB Query, Data Export, Data Import, Redshift, and Refresh Table commands run without a cluster.

Run only Hive DDL queries on a Presto cluster. Running Hive DML queries on a Presto cluster is not recommended as it can lead to unexpected behavior.