Submit a Refresh Table Command

POST /api/v1.2/commands/

This command API can be used to refresh only a Hive table. This API can be mainly used when a Hive partition or directory is extensively used to write data and when Hive tables must be refreshed regularly.

Required Role

The following users can make this API call:

Users who belong to the system-user or system-admin group.
Users who belong to a group associated with a role that allows submitting a command. See Managing Groups and Managing Roles for more information.

Parameters

Note

Parameters marked in bold below are mandatory. Others are optional and have default values.

Parameter	Description
db_name	Database name that contains the Hive table, which is to be refreshed.
hive_table	Name of the Hive table that is to be refreshed
loader_stable	Checks if a Hive directory or partition is fully loaded.
loader_stable_mult	It is the time in minutes to wait before a directory is considered loaded.
template	s3import template is used to refresh tables. The template is used to differentiate the `refresh table` command from other Hive commands that use generic template.
name	Add a name to the command that is useful while filtering commands from the command history. It does not accept & (ampersand), < (lesser than), > (greater than), “ (double quotes), and ‘ (single quote) special characters, and HTML tags as well. It can contain a maximum of 255 characters.
tags	Add a tag to a command so that it is easily identifiable and searchable from the commands list in the Commands History. Add a tag as a filter value while searching commands. It can contain a maximum of 255 characters. A comma-separated list of tags can be associated with a single command. While adding a tag value, enclose it in square brackets. For example, `{"tags":["<tag-value>"]}`.
macros	Denotes the macros that are valid assignment statements containing the variables and its expression as: `macros: [{"<variable>":<variable-expression>}, {..}]`. You can add more than one variable. For more information, see Macros.
timeout	It is a timeout for command execution that you can set in seconds. Its default value is 129600 seconds (36 hours). QDS checks the timeout for a command every 60 seconds. If the timeout is set for 80 seconds, the command gets killed in the next minute that is after 120 seconds. By setting this parameter, you can avoid the command from running for 36 hours.

Request

curl -i -X POST -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" \
-d ' {"db_name":"default", "hive_table":"default_qubole_memetracker", "loader_stable":"1",
"loader_stable_mult":"Minutes", "template":"s3import"}' \
"https://api.qubole.com/api/v1.2/commands"

Note

The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.

Sample Response

HTTP/1.1 200 OK
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json; charset=utf-8
Date: Mon, 16 Nov 2015 18:49:20 GMT
ETag: "b2a6a723e8b7e931ff87e44feacc9a2f"
Server: nginx/1.6.2 + Phusion Passenger 4.0.53
Set-Cookie: _tapp_session=bd0070f3344489b9a306c8c072cdc71c; path=/; HttpOnly
Set-Cookie: qbol_user_id=1574; path=/
Status: 200 OK
X-Powered-By: Phusion Passenger 4.0.53
X-Rack-Cache: invalidate, pass
X-Request-Id: 91b57e2a21319a4b51fb354378869fb0
X-Runtime: 0.422908
X-UA-Compatible: IE=Edge,chrome=1
Content-Length: 865
Connection: keep-alive

{"status":"waiting","qbol_session_id":null,"progress":0,"uid":2081,"account_id":632,"end_time":null,
"start_time":null,"command_type":"HiveCommand","command":{"sample":false,"approx_aggregations":false,"md_cmd":null,
"query":"use default ; alter table default_qubole_memetracker recover partitions;","approx_mode":false,
"loader_table_name":"default.default_qubole_memetracker","retry":0,"script_location":null,"loader_stable":1},
"created_at":"2015-11-16T18:49:20Z","num_result_dir":0,"submit_time":1447699760,"pid":null,"can_notify":false,
"qlog":null,"resolved_macros":null,"label":"default","user_id":1574,"saved_query_mutable_id":null,
"command_source":"API","name":null,"pool":null,"timeout":null,"template":"s3import",
"path":"/tmp/2015-11-16/632/402620","id":402620,"meta_data":{"results_resource":"commands/402620/results",
"logs_resource":"commands/402620/logs"}}