Submit a Presto Command

POST /api/v1.2/commands/

This API is used to submit a Presto command.

Note

Presto is currently supported on AWS, Azure, and GCP Cloud platforms; see QDS Components: Supported Versions and Cloud Platforms. Presto supports querying tables backed by the storage handlers.

Required Role

The following users can make this API call:

• Users who belong to the system-user or system-admin group.

• Users who belong to a group associated with a role that allows submitting a command. See Managing Groups and Managing Roles for more information.

Parameters

Note

Parameters marked in bold below are mandatory. Others are optional and have default values.

Parameter	Description
query	Specify Presto query to run. Either the query or the script_location is required.
script_location	Specify a S3 path where the presto query to run is stored. Either the query or the script_location is required. The AWS storage credentials stored in the account are used to retrieve the script file.
command_type	Presto command
label	Specify the cluster label on which this command is to be run.
retry	Denotes the number of retries for a job. Valid values of retry are 1, 2, and 3. Caution Configuring retries will just do a blind retry of a Presto query. This may lead to data corruption for non-Insert Overwrite Directory (IOD) queries.
name	Add a name to the command that is useful while filtering commands from the command history. It does not accept & (ampersand), < (lesser than), > (greater than), “ (double quotes), and ‘ (single quote) special characters, and HTML tags as well. It can contain a maximum of 255 characters.
tags	Add a tag to a command so that it is easily identifiable and searchable from the commands list in the Commands History. Add a tag as a filter value while searching commands. It can contain a maximum of 255 characters. A comma-separated list of tags can be associated with a single command. While adding a tag value, enclose it in square brackets. For example, {"tags":["<tag-value>"]}.
macros	Denotes the macros that are valid assignment statements containing the variables and its expression as: macros: [{"<variable>":<variable-expression>}, {..}]. You can add more than one variable. For more information, see Macros.
timeout	It is a timeout for command execution that you can set in seconds. Its default value is 129600 seconds (36 hours). QDS checks the timeout for a command every 60 seconds. If the timeout is set for 80 seconds, the command gets killed in the next minute that is after 120 seconds. By setting this parameter, you can avoid the command from running for 36 hours.

Note

You can add a session property, qubole_max_raw_input_datasize=1TB to limit the total bytes scanned. Queries that exceed this limit fail with the RAW_INPUT_DATASIZE_READ_LIMIT_EXCEEDED exception. This ensures rogue queries do not run for a very long time.

Examples

Show tables

curl  -i -X POST -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"query":"show tables;", "command_type": "PrestoCommand"}' "https://api.qubole.com/api/v1.2/commands"

Note

The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.

Note

For a given Presto query, a new Presto Query Tracker is displayed in the Logs tab when:

• A cluster instance that ran the query is still up.

• The query info is still present in the Presto server in that cluster instance. The query information is periodically purged from the server.

If any of the above 2 conditions is not met, the older Presto Query Tracker is displayed in the Logs tab.

Sample Response

{ "command":
  { "query": "show tables", },
  "qbol_session_id": 0000,
  "created_at": "2014-01-21T16:01:09Z",
  "user_id": 00,
  "status": "waiting",
  "command_type": "PrestoCommand",
  "id": 4850,
  "progress": 0,
  "meta_data": {
    "results_resource": "commands/4850/results",
    "logs_resource": "commands/4850/logs"
  }
}

Count Number of Rows

export QUERY="select count(*) as num_rows from miniwikistats;" curl -X POST -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{\"query\":\"$QUERY\", \"command_type\" : \"PrestoCommand\" }' "https://api.qubole.com/api/v1.2/commands/"

Sample Response

{
  "command": { "query": "select count(*) as num_rows from miniwikistats;", },
  "qbol_session_id": 0000,
  "created_at": "2014-10-11T16:54:57Z",
  "user_id": 00,
  "status": "waiting",
  "command_type": "PrestoCommand",
  "id": 4852,
  "progress": 0,
  "meta_data": {
    "results_resource": "commands/4852/results",
    "logs_resource": "commands/4852/logs"
  }
}