All Commands Report
- GET /api/v1.2/reports/all_commands
This API retrieves the All Commands report containing the query metrics in JSON format.
Note
The following points are related to a report API:
If the difference between start date and end date is more than 60 days, then the system defaults to 1 month window from the current day’s date.
If either start date or end date is not provided, then the system defaults to 1 month window from the current day’s date.
If you want to get data for a window more than 2 months, then create a ticket with Qubole Support.
Required Role
The following users can make this API call:
Users who belong to the system-user or system-admin group.
Users who belong to a group associated with a role that allows viewing commands reports. See Managing Groups and Managing Roles for more information.
Parameters
Parameter |
Description |
|---|---|
start_date |
The date from which you want the report (inclusive). This parameter supports the timestamp in the UTC timezone (YYYY-MM-DDTHH:MM:SSZ) format. The date cannot be earlier than 180 days. |
end_date |
The date until which you want the report (inclusive). The API default is today. This parameter also supports timestamp in the UTC timezone (YYYY-MM-DDTHH:MM:SSZ) format. |
offset |
The starting point of the results. The API default is 0. |
limit |
The number of results to fetch. The API default is 10. |
sort_column |
The column used to sort the report. The valid choices are |
by_user |
Report only those queries which are created by the current user. By default all queries by the current account are reported. |
|
It enables you to filter queries based on the following status values: done, error, running, waiting, cancelled, and cancelling. |
Response Parameters
Parameter |
Description |
|---|---|
start_date |
The starting date of the report. This parameter filters on the |
end_date |
The ending date of the report. This parameter filters on the |
sort_column |
The sort column used. |
queries |
Contains an array of query-related parameters and values as provided in the following table. |
An array of parameters and values associated with the queries parameter.
id |
The query ID of the command. |
created_at |
The time when the query was submitted. |
submitted_by |
The Email address of the user who created the query. |
command_type |
The type of the command (HiveCommand, PrestoCommand, and so on.) |
command_summary |
The summary of the command (query/latin_statements/script_location, and so on.) |
status |
The status of the command (whether it succeeded or failed, and so on.) |
label |
It is the label of the cluster on which the command is run. |
end_time |
Denotes the time at which the query execution completed. |
cpu |
The total cumulative CPU, (in ms), consumed by this command. It signifies the cumulative CPU time spent by all cluster nodes in the cluster that processed the command. |
fs_bytes_read |
The total bytes read by this command. |
fs_bytes_written |
The total bytes written by this command. |
Examples
Goal
To get the default report.
curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Accept: application/json" \
"https://api.qubole.com/api/v1.2/reports/all_commands"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Goal
To get the report for commands executed only by the current user.
curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Accept: application/json" \
"https://api.qubole.com/api/v1.2/reports/all_commands?by_user"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Goal
To get the report for commands executed during a specific time period and sorted by total bytes read.
curl -i -X GET -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Accept: application/json" \
"https://api.qubole.com/api/v1.2/reports/all_commands?start_date=2017-06-10&end_date=2017-07-10&sort_column=fs_bytes_read"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Sample Response
{
"sort_column": "fs_bytes_read",
"start_date": "2017-06-10T00:00:00Z",
"end_date": "2017-07-10T00:00:00Z",
"queries": [
{
"id": 79520285,
"created_at": "2017-06-19T08:56:06Z",
"submitted_by": "[email protected]",
"command_type": "HiveCommand",
"command_summary": "select count(*) from default_qubole_memetracker where month=\"2008-08\";",
"status": "done",
"end_time": "2017-06-19 09:05:12 +0000",
"tags": "",
"fs_bytes_read": 4429520896,
"fs_bytes_written": 8,
"cpu": null
},
{
"id": 78826792,
"created_at": "2017-06-15T07:01:23Z",
"submitted_by": "[email protected]",
"command_type": "HiveCommand",
"command_summary": "SELECT * FROM `tpcds_orc_1000`.`customer`",
"status": "done",
"end_time": "2017-06-15 07:02:53 +0000",
"tags": "JDBC 1.0.7",
"fs_bytes_read": 683353856,
"fs_bytes_written": 1629612160,
"cpu": 74910
},
{
"id": 78828387,
"created_at": "2017-06-15T07:11:24Z",
"submitted_by": "[email protected]",
"command_type": "HiveCommand",
"command_summary": "SELECT COUNT(* ) FROM `tpcds_orc_1000`.`customer`",
"status": "done",
"end_time": "2017-06-15 07:12:03 +0000",
"tags": "JDBC 1.0.7-SNAPSHOT",
"fs_bytes_read": 683353856,
"fs_bytes_written": 9,
"cpu": 25680
},
{
"id": 78830962,
"created_at": "2017-06-15T07:23:20Z",
"submitted_by": "[email protected]",
"command_type": "HiveCommand",
"command_summary": "SELECT * FROM `tpcds_orc_1000`.`customer`",
"status": "done",
"end_time": "2017-06-15 07:24:44 +0000",
"tags": "JDBC 1.0.7",
"fs_bytes_read": 683353856,
"fs_bytes_written": 1629612160,
"cpu": 76200
},
{
"id": 78652381,
"created_at": "2017-06-14T08:22:17Z",
"submitted_by": "[email protected]",
"command_type": "HiveCommand",
"command_summary": "set hive.cli.print.header=false;\nset hive.resultset.use.unique.column.names=false; \nSELECT * FROM `table`.`calcs`",
"status": "done",
"end_time": "2017-06-14 08:24:14 +0000",
"tags": "ODBC 1.0.0.1001",
"fs_bytes_read": 23357610,
"fs_bytes_written": 7470759,
"cpu": 4330
},
{
"id": 78825573,
"created_at": "2017-06-15T06:54:44Z",
"submitted_by": "[email protected]",
"command_type": "HiveCommand",
"command_summary": "SELECT * FROM `table`.`calcs`",
"status": "done",
"end_time": "2017-06-15 06:57:54 +0000",
"tags": "JDBC 1.0.7",
"fs_bytes_read": 23357610,
"fs_bytes_written": 7470759,
"cpu": 4320
}
]
}