Understanding Cluster Operations

The following section explains cluster operations such as start, terminate, add, clone, and delete a cluster.

How to Start or Terminate a Cluster

The current state of the cluster is indicated by the icon on the extreme left of each row in the cluster table. It is a green circle for a running cluster, a red circle for a terminated cluster and an arrow pointing up or down for clusters that are starting up or terminating, respectively.

  • Click the Start button to start a cluster or restart a stopped cluster.

    A dialog box prompts for confirmation. Click OK to start a cluster. Click Cancel if you do not want to start a cluster.

    Caution

    You must not manually launch instances in the security groups created by Qubole when a cluster is active. Launching the instances in the Qubole security groups would result in the incorrect display of the number of nodes in the Clusters UI page.

  • Click the Stop button to terminate a running cluster.

    A dialog box prompts for confirmation. Click OK to terminate a cluster. Click Cancel if you do not want to terminate a cluster.

Click the refresh icon RefreshIcon that is on the top-right corner of the Clusters page to refresh the clusters status.

How to Add a Cluster

To add a cluster, click the New button on the Clusters page.

The Create New Cluster page is displayed. Enter the following details to create a new cluster:

Click Save to create the new cluster.

How to Clone a Cluster

Cloning may be preferable to creating a new cluster in many cases since most of the fields will be copied from an existing cluster.

To clone a cluster, click the ellipse icon listed against the cluster.

Select Clone from the list of options as shown in the following figure.

../../_images/cloneNdefaultCluster.png

The Clone a Cluster page is displayed. Enter a new label for the cluster. Do the required modifications and click Save to clone the cluster. The label is the only mandatory field to be changed when you clone a cluster.

Clicking Clone takes you to the Edit Clusters page.

How to Modify a Cluster

To edit a cluster, click the Edit button available in the Action column.

The Edit Cluster page is displayed. The current configuration of the cluster is displayed on this page. You can make the desired modifications to it. See Modifying Cluster Configuration for more information. Click Save to save the modifications.

How to Push Configuration Changes to a Cluster

Most cluster changes take effect only when a cluster is restarted, but some can be pushed to a running cluster. Changes to the following cluster attributes can be pushed to a running cluster:

  • The maximum size of the cluster
  • The minimum size of the cluster
  • The Fair Scheduler configuration
  • The default Fair Scheduler pool
  • The maximum Spot bid price (AWS)
  • The Hadoop configuration variables (AWS)

To push configuration changes to a running cluster, click the ellipse icon listed against the cluster.

Select Push from the list of options:

../../_images/cloneNdefaultCluster.png

The resulting Edit Cluster page shows all settings of the cluster; Pushable fields are marked with a P. QDS allows modifying editable fields besides the pushable fields. However, other editable fields (without a P) get effective only after the cluster restart. After doing the changes, you can click one of the two options:

  • Udate and Push - To push the changes of all pushable fields into the running clusters. Other editable (non-pushable) fields’ changes get effective after the cluster restart. Once you click Update and Push, a dialog is displayed that prompts you to confirm before pushing the corresponding settings into the running cluster. Click Update and Push again.
  • Update only - To edit the cluster configuration that would get effective only after the running cluster restarts.

How to Delete a Cluster

Note

Running clusters and the cluster labeled default cannot be deleted.

To delete a cluster, click the ellipse icon listed against the cluster.

Select Delete from the list of options as shown in the following figure.

../../_images/ClusterPush.png

A dialog box is displayed prompting for confirmation. Click Ok to delete a cluster. Click Cancel if you do not want to delete a cluster.

Warning

Once a cluster is deleted, it cannot be retrieved. So, be cautious when deleting a cluster.

Switching Clusters

Clusters are identified using labels. When you want to switch the workload of one cluster to another, you can reassign the clusters’ labels. To do this, hover the mouse on the 3 vertical dots that is to the left of the cluster label and drag it to another cluster.

You can also click view the commands that are run on a specific cluster by clicking on the label name.