Overview

The Qubole Data Service (QDS) is accessible via REST APIs.

To write and test applications that use these APIs, you can use any HTTP client in any programming language to interact with Qubole Data Service. The detailed syntax of the calls is described in subsequent sections.

You can also use the QDS Python SDK or the QDS Java SDK to interact with the Qubole Data Service. The Python SDK also provides an easy-to-use command-line interface.

Note

Qubole now supports only HTTPS. All HTTP requests are now redirected to HTTPS. This is aimed at better security for Qubole users.

Access URL

All URLs referenced in the documentation have the following base:

https://api.qubole.com/api/${V}/

where ${V} refers to the version of the API. Valid values of ${V} are currently v1.2, and latest.

However, Qubole now supports other access URLs or endpoints that you can use to access QDS that are described in Supported Qubole Endpoints.

cURL is a useful tool for testing out Qubole REST API calls from the command line.

The Qubole API is served over HTTPS and Qubole redirects all HTTP access to HTTPS. Qubole now supports only HTTPS access, which is aimed at better security for Qubole users. When using HTTPS, ensure that the certificates on the client machine are up-to-date.

Authentication

API calls must be authenticated with a Qubole API Token.

Navigate to Control Panel in UI at https://api.qubole.com/v2/control-panel and click the My Accounts tab on the left pane. Click Show for the account and copy the API token that is displayed.

Set the value of this API token to the AUTH_TOKEN environment variable when running the API examples via curl.

API Types

The Qubole REST APIs are broadly divided into the following categories:

Account API

These APIs let you to create a new account, edit and delete an existing account, view users, and set Hive bootstrap. See Account API for more information.

Apps API

These APIs allow you to create, list, and delete a Spark Job Server App. For more information, see Apps API.

Cluster API

These APIs allow you to create a new cluster, edit, restart, and delete an existing clusters. These APIs also allow you to view the list of clusters in a Qubole account, and add and remove nodes in some clusters. See Cluster API for more information.

Command API

The Command APIs let you submit queries and commands, check the status of commands, retrieve results and logs, or cancel commands. See Command API for more information. The Qubole Data Service currently supports these command types:

Command Template API

Use a commmand template API to run a command more than one time and run the command with different parameter values. For more information, see Command Template API.

Custom Metastore API

These APIs allow you to connect to a custom metastore, edit and view the metastore details. For more information, see Custom Metastore API.

DbTap API

A DbTap identifies an external end point for import/export of data from QDS, such as a MySQL instance. The DbTap APIs let you create, view, edit or delete a DbTap. For more information, see DbTap API.

Editions API

This API is mainly used to self start the QDS-on-AWS cloud agents free trial. For more information, see Editions API.

Group API

These APIs allow you to create groups, add/delete users in a group, and assign/unassign roles to a group in a Qubole account. For more information, see Groups API.

Hive Metadata API

These APIs provide a set of read-only views that describe your Hive tables and the metadata. For more information, see Hive Metadata API.

Notebook API

These APIs allow you to create a notebook, clone, configure, run, import, and delete a notebook. For more information, see Notebook API.

Object Policy API

These APIs allow you to create object-level access policies for notebooks and clusters. For more information, see Object Policy API.

Reports API

These APIs let you view aggregated statistical and operational data for your commands. For more information, see Reports API.

Role API

These APIs allow you to create and delete a role to perform a set of actions in a Qubole account. For more information, see Roles API.

Scheduler API

The Scheduler APIs let you schedule any command or workflow to run at regular intervals. For more information, see Scheduler API.

Sensor API

These APIs allow you to create file and partition sensors to monitor the file and Hive partition’s availability. For more information, see Sensor API.

Users API

These APIs let you view invite a user to a Qubole account and enable/disable users in a Qubole account. For more information, see Users API.