Resize a Cluster¶

PUT /api/v2.2/clusters/<<Cluster ID>/<Cluster Label>>¶

Use this REST API to change the minimum and maximum size of a running cluster.

Note

Set the configuration option, push to true for changes to be effective immediately without a cluster restart.

Modifying the Minimum Cluster Size¶

Any Hadoop cluster running on Qubole has a minimum and maximum cluster size between which the cluster autoscales depending on the load. The minimum and maximum cluster size can be changed for a running cluster as a pushable configuration.

As a prerequisite, mapred.refresh.min.cluster.size=true must be set in the cluster to modify a running cluster’s minimum size. This property is set to true by default in an Hadoop cluster. Ensure that the property is not set to false or removed.

Required Role¶

The following roles can make this API call:

A user who is part of the system-admin group.
A user invoking this API must be part of a group associated with a role that allows editing an existing cluster’s configuration. See Managing Groups and Managing Roles for more information.

Request API Syntax¶

curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"initial_nodes": "<value>"}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/${cluster_id}/"

Note

The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.

Parameters¶

Note

Parameters marked in bold are mandatory. Others are optional and have default values.

Parameter	Description
cluster_id	It is the cluster ID.
cluster_info	This contains the cluster nodes details. See Node Configuration Options used to Minimize Cluster Nodes.

Sample Request¶

This sample request is for minimizing the nodes for a running cluster that has 3711 as its ID.

curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"initial_nodes":2}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/3711/"

Node Configuration Options used to Minimize Cluster Nodes¶

Parameter	Description
initial_nodes	The number of minimum cluster nodes in a cluster.
push	Set this parameter to `true` to change the minimum number of nodes for a running cluster. If `push` is not set to `true`, changes become effective only after the cluster is restarted.

Modifying the Maximum Cluster Size¶

This API is used to modify the maximum number of cluster nodes in a running cluster.

Required Role¶

The following roles can call this API:

A user who is part of the system-admin group can call this API request.
A group user who is assigned with a user-defined role that allows editing an existing cluster’s configuration.

Request API Syntax¶

curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"max_nodes": "<value>"}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/${cluster_id}/"

Note

The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.

Parameters¶

Note

Parameters marked in bold are mandatory. Others are optional and have default values.

Parameter	Description
cluster_id	It is the cluster ID.
cluster_info	This contains the cluster nodes details. See Node Configuration Options used to Maximize Cluster Nodes.

Node Configuration Options used to Maximize Cluster Nodes¶

Parameter	Description
max_nodes	Use this parameter to change the maximum number of cluster nodes.
push	Set this parameter to `true` to change the maximum number of nodes for a running cluster. If `push` is not set to `true`, changes become effective only after the cluster is restarted.

Sample Request¶

This sample request is for maximizing the nodes for a running cluster that has 3850 as its cluster ID.

curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"max_nodes":1000}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/3850/"