Edit a QDS Account using API version 2¶
-
PUT/api/v2/accounts/¶
Use this API to edit an existing QDS account parameters. All parameters are optional.
Required Role¶
The following users can make this API call:
- Users who belong to the system-user/system-admin groups.
- Users who belong to a group associated with a role that allows editing an account. See Managing Groups and Managing Roles for more information.
Parameters¶
| Parameter | Description |
|---|---|
| compute_type | Denotes the type of compute credentials for the account. Its value must be CUSTOMER_MANAGED. It is mandatory when you update compute settings globally from the current account. |
| access_mode | Denotes the access type you want to use to create an account. The possible values are IAM_ROLES and IAM_KEYS (default). If the access mode is IAM_KEYS, the acc_key, secret, compute_access_key and compute_secret_key parameters are mandatory if you want to switch the account mode from IAM_ROLES to IAM_KEYS. If the access mode is IAM_ROLES, the new role_arn parameter is mandatory if you want to switch the account mode from IAM_KEYS to IAM_ROLES. |
| role_arn | The Role ARN to which the external ID of the account is mapped. For more information on IAM Roles, see the AWS documentation for IAM Roles . To get the external ID for the current account, refer View an Account . |
| compute_access_key | It is the AWS Access Key used as a compute credential. It is mandatory when you update compute settings globally from the current account. |
| compute_secret_key | It is the AWS Secret Key used as a compute credential. It is mandatory when you update compute settings globally from the current account. |
| storage_access_key | This is the Storage Access Key for your S3 storage credentials. It is used to edit the AWS Access Key and is mandatory when you update storage settings. |
| storage_secret_key | This is the Storage Secret Key for your S3 storage credentials. It is used to edit the AWS Secret Key and is mandatory when you update storage settings. |
| defloc | It is the default location on Amazon S3 for storing logs, results and caching data. It can be any location on S3 buckets. It is mandatory when you update storage settings. Change the Default Location as it is recommended and the purpose is highlighted in this warning. Warning To avoid adverse effects on the command latency and performance, specify a separate S3 bucket for each QDS account and avoid providing long prefixes and sharing the same S3 bucket for all accounts. |
| notification_email_list | This parameter is used to add emails that would receive any change in QDS settings. |
| CacheQuotaSizeInGB | The default value of Amazon S3 cache size is 25 GB. It can be more than 25 GB and does not have a maximum limit. It is set to maintain a size-limited cache for files stored on the local device and and once the cache size limit is reached, Qubole clears excess files from the local cache. Qubole’s caching filesystem buffers data locally and asynchronously uploads files to HDFS and S3. The S3 Cache Size is applicable to Hadoop and Spark clusters. |
| idle_cluster_timeout | The default cluster timeout is 2 hours. Optionally, you can configure it between 0 to 6 hours that is
the value range is 0-6 hours. The unit of time supported is only hour. If the timeout is set at account
level, it applies to all clusters within that account. However, you can override the timeout at cluster
level. The timeout is effective on the completion of all queries on the cluster. Qubole terminates a
cluster in an hour boundary. For example, when idle_cluster_timeout is 0, then if there is any node
in the cluster near its hour boundary (that is it has been running for 50-60 minutes and is idle even after
all queries are executed), Qubole terminates that cluster. |
| idle_cluster_timeout_in_secs | After enabling the aggressive downscaling feature on the QDS account, the Cluster Idle Timeout can be
configured in seconds. Its minimum configurable value is Note This feature is only available on a request. Contact the account team to enable this feature on the QDS account. |
| idle_session_timeout | If there is no activity on the UI, Qubole logs you out after a specified interval. The default value is 1440 minutes (24 hours). Use this option to configure the time interval in minutes if you do not want the default idle session timeout. The maximum value is 10800 minutes (1 week). |
| show_s3_download_link | By default this parameter is set to false. To download files from an S3 location, set it to true. |
| show_s3_upload_link | By default this parameter is set to false. To upload files to an S3 location, set it to true. |
| persistent_security_groups | This parameter is used to add a security group that overrides the account-level settings. You must provide a persistent security group when you configure outbound communication from cluster nodes to pass through a Internet proxy server. |
update_all_clusters_compute_creds |
When this parameter is set to true, Qubole pushes new settings to all clusters and vice versa. Note While transferring access settings from IAM Keys to IAM Roles or vice-versa, Qubole pushes the new settings to all clusters irrespective of the value of this parameter. |
| node_bootstrap_file | A file that is executed on every node of every cluster in the account at boot time. You can use this to customize your cluster nodes by setting up environment variables, installing required packages, and so on. Note The account node bootstrap is executed before the cluster node bootstrap. |
Request API Syntax¶
curl -X PUT -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"<an account parameter" : "<new value>", ......}' "https://api.qubole.com/api/v2/accounts/"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Sample Request¶
Here is a sample request API to edit a QDS account.
curl -X PUT -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{
"account":
{
"compute_type":"CUSTOMER_MANAGED",
"compute_access_key" : "$COMPUTE_ACCESS_KEY",
"compute_secret_key": "$COMPUTE_SECRET_KEY",
"access_mode": "$ACCESS_MODE",
"storage_access_key" : "$STORAGE_ACCESS_KEY",
"storage_secret_key" : "$STORAGE_SECRET_KEY",
"defloc": "$DEFLOC",
"idle_cluster_timeout": 2,
"idle_session_timeout": 4320,
"show_s3_download_link":true,
"show_s3_upload_link":true,
"node_bootstrap_file": "node_bootstrap.sh"
"persistent_security_groups": "default"
}
}' "https://api.qubole.com/api/v2/accounts/"
Sample Response¶
{
"CacheQuotaSizeInGB": 50,
"account_plan_id": 3701,
"allow_qubole_access": true,
"allowed_email_domain": null,
"ami_overrides": null,
"aws_availability_zone": "Any",
"aws_region": "us-east-1",
"cloud_provider": "AWS",
"compute_access_key": "$COMPUTE_ACCESS_KEY",
"compute_secret_key": "$COMPUTE_SECRET_KEY",
"compute_type": "CUSTOMER_MANAGED",
"compute_userid": null,
"compute_validated": true,
"created_at": "2018-08-13T08:54:12Z",
"custom_hadoop_config": null,
"custom_metastore_dbtap_id": null,
"customer_ssh_key": null,
"datadog_api_token": null,
"datadog_app_token": null,
"default_cluster_tags": [],
"default_pool": null,
"defloc": "$DEFLOC",
"deleted_at": null,
"disabled": 0,
"disallow_cluster_termination": false,
"email_timeout_secs": null,
"encrypted_ephemerals": false,
"fair_scheduler_config": null,
"ganglia_monitoring": false,
"gce_project_id": null,
"gce_sa_client_email": null,
"gce_sa_private_key_file": null,
"gce_storage_project_id": null,
"gce_storage_sa_client_email": null,
"gce_storage_sa_private_key": null,
"gce_storage_sa_private_key_file": null,
"gcs_storage_system_bucket": null,
"gcs_system_bucket": null,
"hadoop_initial_nodes": 1,
"hadoop_master_type": "m1.large",
"hadoop_max_nodes": 1,
"hadoop_slave_request_type": "hybrid",
"hadoop_slave_type": "m1.xlarge",
"hive_bootstrap_base_loc": null,
"iam_access_key": null,
"iam_location": null,
"iam_secret": null,
"id": 3399,
"idle_cluster_timeout": 2,
"idle_cluster_timeout_in_secs": null,
"idle_session_timeout": 4320,
"info": null,
"initialized": true,
"is_early_access": null,
"join_confirm": "CT92Q7ZXCHFRH9JZ3399",
"kms_cmk_id": null,
"maximum_bid_price_percentage": 100,
"maximum_spot_instance_percentage": 50,
"mstore_password": null,
"name": "Test",
"nezha_ds_set_id": null,
"node_bootstrap_file": "node_bootstrap.sh"
"persistent_security_groups": "default",
"private_ssh_key": "$PVT_SSH_KEY",
"public_ssh_key": "$PUBLIC_SSH_KEY",
"receive_email_notifications": true,
"restrict_ssh_access": false,
"storage_access_key": "$ACCESS_KEY",
"storage_secret_key": "$STORAGE_SECRET_KEY",
"show_s3_download_link": true,
"show_s3_upload_link": true,
"sse_algo": null,
"sse_key": null,
"state": "success_create",
"status": null,
"storage_external_id": "$STORAGE_EXTERNAL_ID",
"storage_role_arn": "$STORAGE_ROLE_ARN",
"storage_type": "CUSTOMER_MANAGED",
"storage_validated": true,
"storage_validation_result": null,
"stripe_customer_id": null,
"timeout_for_spot_request": 10,
"updated_at": "2018-08-13T09:53:27Z",
"white_label_layout_id": null
}