Resize a Cluster
- PUT /api/v2.2/clusters/<<Cluster ID>/<Cluster Label>>
Use this REST API to change the minimum and maximum size of a running cluster.
Note
Set the configuration option, push to true for changes to be effective immediately without
a cluster restart.
Modifying the Minimum Cluster Size
Any Hadoop cluster running on Qubole has a minimum and maximum cluster size between which the cluster autoscales depending on the load. The minimum and maximum cluster size can be changed for a running cluster as a pushable configuration.
As a prerequisite, mapred.refresh.min.cluster.size=true must be set in the cluster to modify a running cluster’s
minimum size. This property is set to true by default in an Hadoop cluster. Ensure that the property is not set to
false or removed.
Required Role
The following roles can make this API call:
A user who is part of the system-admin group.
A user invoking this API must be part of a group associated with a role that allows editing an existing cluster’s configuration. See Managing Groups and Managing Roles for more information.
Request API Syntax
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"initial_nodes": "<value>"}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/${cluster_id}/"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Parameters
Note
Parameters marked in bold are mandatory. Others are optional and have default values.
Parameter |
Description |
|---|---|
cluster_id |
It is the cluster ID. |
cluster_info |
This contains the cluster nodes details. See Node Configuration Options used to Minimize Cluster Nodes. |
Sample Request
This sample request is for minimizing the nodes for a running cluster that has 3711 as its ID.
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"initial_nodes":2}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/3711/"
Node Configuration Options used to Minimize Cluster Nodes
Parameter |
Description |
|---|---|
initial_nodes |
The number of minimum cluster nodes in a cluster. |
push |
Set this parameter to |
Modifying the Maximum Cluster Size
This API is used to modify the maximum number of cluster nodes in a running cluster.
Required Role
The following roles can call this API:
A user who is part of the system-admin group can call this API request.
A group user who is assigned with a user-defined role that allows editing an existing cluster’s configuration.
Request API Syntax
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"max_nodes": "<value>"}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/${cluster_id}/"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Parameters
Note
Parameters marked in bold are mandatory. Others are optional and have default values.
Parameter |
Description |
|---|---|
cluster_id |
It is the cluster ID. |
cluster_info |
This contains the cluster nodes details. See Node Configuration Options used to Maximize Cluster Nodes. |
Node Configuration Options used to Maximize Cluster Nodes
Parameter |
Description |
|---|---|
max_nodes |
Use this parameter to change the maximum number of cluster nodes. |
push |
Set this parameter to |
Sample Request
This sample request is for maximizing the nodes for a running cluster that has 3850 as its cluster ID.
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json"
-d '{"cluster_info": {"max_nodes":1000}, "push": "true"}' \
"https://api.qubole.com/api/v2.2/clusters/3850/"