Resize a Cluster¶
-
PUT
/api/v1.3/clusters/
(string: id_or_label)¶
Use this REST API to change the minimum and maximum size of a running cluster.
Note
Set the configuration option, push
to true
for changes to be effective immediately without
a cluster restart.
Modifying the Minimum Cluster Size¶
Any Hadoop cluster running on Qubole has a minimum and maximum cluster size between which the cluster autoscales depending on the load. The minimum and maximum cluster size can be changed for a running cluster as a pushable configuration.
As a prerequisite, mapred.refresh.min.cluster.size=true
must be set in the cluster to modify a running cluster’s
minimum size. This property is set to true
by default in an Hadoop cluster. Ensure that the property is not set to
false
or removed.
Required Role¶
The following users can make this API call:
- Users who belong to the system-admin group.
- Users who belong to a group associated with a role that allows editing an existing cluster’s configuration. See Managing Groups and Managing Roles for more information.
Request API Syntax¶
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"node_configuration": {"initial_nodes": "<value>"}, "push": "true"}' \
"https://api.qubole.com/api/v1.3/clusters/${cluster_id}/"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Parameters¶
Note
Parameters marked in bold are mandatory. Others are optional and have default values.
Parameter | Description |
---|---|
cluster_id | It is the cluster ID. |
node_configuration | This contains the cluster nodes details. See Node Configuration Options used to Minimize Cluster Nodes. |
Sample Request¶
This sample request is for minimizing the nodes for a running cluster with 3711 as its ID.
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"node_configuration": {"initial_nodes":2}, "push": "true"}' \
"https://api.qubole.com/api/v1.3/clusters/3711/"
Node Configuration Options used to Minimize Cluster Nodes¶
Parameter | Description |
---|---|
initial_nodes | The number of minimum cluster nodes in a cluster. |
push | Set this parameter to true to change the minimum number of nodes for a
running cluster. If push is not set to true , changes become effective only after
the cluster is restarted. |
Modifying the Maximum Cluster Size¶
This API is used to modify the maximum number of cluster nodes in a running cluster.
Required Role¶
The following roles can call this API:
- Users who belong to the system-admin group can call this API request.
- A group user who is assigned with a user-defined role that allows editing an existing cluster’s configuration.
Request API Syntax¶
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"node_configuration": {"max_nodes": "<value>"}, "push": "true"}' \
"https://api.qubole.com/api/v1.3/clusters/${cluster_id}/"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Parameters¶
Note
Parameters marked in bold are mandatory. Others are optional and have default values.
Parameter | Description |
---|---|
cluster_id | It is the cluster ID. |
node_configuration | This contains the cluster nodes details. See Node Configuration Options used to Maximize Cluster Nodes. |
Node Configuration Options used to Maximize Cluster Nodes¶
Parameter | Description |
---|---|
max_nodes | Use this parameter to change the maximum number of cluster nodes. |
push | Set this parameter to true to change the maximum number of nodes for a running
cluster. If push is not set to true , changes become effective only after
the cluster is restarted. |
Sample Request¶
This sample request is for maximizing the nodes for a running cluster with 3850 as its ID.
curl -i -X PUT -H "X-AUTH-TOKEN: $AUTH_TOKEN" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"node_configuration": {"max_nodes":1000}, "push": "true"}' \
"https://api.qubole.com/api/v1.3/clusters/3850/"