Validate Compute Credentials on a Cluster
- GET /api/latest/clusters/<cluster ID>/validate_compute_permissions
Use this API to validate compute credentials on a QDS cluster in a Qubole environment. Ensure to use the correct environment URL where the QDS cluster is created. For more information on the compute credentials that get validated, see Configuring your Access Settings using IAM Keys or Configuring your Access Settings using IAM Roles.
Request API Syntax
curl -X GET -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
"https://<Qubole Environment>/api/latest/clusters/<cluster ID>/validate_compute_permissions"
Here <Qubole Environment> is the Qubole environment in which you have the QDS account. For example, https://api.qubole.com
is a Qubole environment.
Note
Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Sample API Request
Here is the sample request to validate compute credentials on a cluster with 3400 as its ID in the api.qubole.com
environment.
curl -X GET -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
"https://api.qubole.com/api/latest/clusters/3400/validate_compute_permissions"
Understanding Different Permission Validation Status
Status |
Description |
|---|---|
Error |
It implies that the permission is missing and it is mandatory. The functionality does not work as explained in the Description field. |
Warning |
It implies that the permission is missing and it is optional. You can perform some workaround steps as explained in the Description field. |
Sample Response of a Successful Validation
You can also refer to Sample of a Validation-error Response.
{
result: [
{
status: "Success",
msg: "",
perm_list: {
Key Pair: {
ec2:DescribeKeyPairs: {
status: "Success",
message: "",
description: "For checking for the existence of key pair",
UIOrder: 1
},
ec2:ImportKeyPair: {
status: "Success",
message: "",
description: "To get SSH keys to node, so that Qubole or customer can access it, by SSHing",
UIOrder: 2
}
},
Security Group: {
ec2:CreateSecurityGroup: {
status: "Success",
message: "",
description: "Qubole uses this permission to create qubole specific security group which is attached to cluster launched by Qubole. Please make sure to provide access or if you do not wish to provide these permissions then create create security group manually before launching the cluster. Please note if you do not wish to provide permissions to attach rules then attach them manually before launching the cluster. Please check the documentation for details.",
UIOrder: 1
},
ec2:DescribeSecurityGroup: {
status: "Success",
message: "",
description: "Describe security group is needed to check if qubole related security group is present and check the details.",
UIOrder: 2
},
ec2:AuthorizeSecurityGroupEgress: {
status: "Success",
message: "",
description: "Qubole uses this permissions to attach inbound / outbound rules to Qubole defined security group while bringing up the cluster. Please make sure to provide access or you can avoid giving permission and attach rules manually before launching the cluster. Please note if you are not providing access to Create Security Group as well then please create the create security group manually as well. Please check the documentation for details.",
UIOrder: 3
},
ec2:AuthorizeSecurityGroupIngress: {
status: "Success",
message: "",
description: "Qubole uses this permissions to attach inbound / outbound rules to Qubole defined security group while bringing up the cluster. Please make sure to provide access or you can avoid giving permission and attach rules manually before launching the cluster. Please note if you are not providing access to Create Security Group as well then please create the create security group manually as well. Please check the documentation for details.",
UIOrder: 4
},
ec2:DeleteSecurityGroup: {
status: "Success",
message: "",
description: "Qubole automatically terminates qubole defined cluster security group while terminating the cluster. If you do not wish to provide permission then you can manually delete the security group.",
UIOrder: 5
}
},
Instance: {
ec2:RunInstances: {
status: "Success",
message: "",
description: "This give qubole to run the instance, to bring up cluster",
UIOrder: 1
},
ec2:RequestSpotInstances: {
status: "Success",
message: "",
description: "For requesting Spot Node",
UIOrder: 2
},
ec2:CancelSpotInstanceRequests: {
status: "Success",
message: "",
description: "For cancelling request for Spot Node",
UIOrder: 3
},
ec2:CreateTags: {
status: "Success",
message: "",
description: "This is used for billing purposes to identify the number of instances spun up by Qubole.",
UIOrder: 4
},
ec2:DeleteTags: {
status: "Success",
message: "",
description: "This is used to delete tags post termination of the cluster.",
UIOrder: 5
},
ec2:StopInstances: {
status: "Success",
message: "",
description: "To Stop an running instance",
UIOrder: 6
},
ec2:ModifyInstanceAttribute: {
status: "Success",
message: "",
description: "To Modify the instance",
UIOrder: 7
},
ec2:StartInstances: {
status: "Success",
message: "",
description: "Restart after modifying instance",
UIOrder: 8
},
ec2:TerminateInstances: {
status: "Success",
message: "",
description: "In downscaling terminate",
UIOrder: 9
}
}
}
}
]
}
Sample of a Validation-error Response
Understanding Different Permission Validation Status describes the different errors.
{
status: "Error",
messsage: "",
permission_list: {
Key Pair: {
ec2:DescribeKeyPairs: {
status: "Error",
message: "AWS::EC2::Errors::AuthFailure exception listing EC2 key pairs-AWS was not able to validate the provided access credentials",
description: "For checking for the existence of key pair",
UIOrder: 1
},
ec2:ImportKeyPair: {
status: "Warning",
message: "AWS::EC2::Errors::AuthFailure exception importing keypair 'canopykey' into EC2 - please import manually or contact [email protected] was not able to validate the provided access credentials",
description: "To get SSH keys to node, so that Qubole or customer can access it, by SSHing",
UIOrder: 2
}
},
Security Group: {
ec2:CreateSecurityGroup: {
status: "Success",
message: "",
description: "Qubole uses this permission to create qubole specific security group which is attached to cluster launched by Qubole. Please make sure to provide access or if you do not wish to provide these permissions then create create security group manually before launching the cluster. Please note if you do not wish to provide permissions to attach rules then attach them manually before launching the cluster. Please check the documentation for details.",
UIOrder: 1
},
ec2:DescribeSecurityGroup: {
status: "Error",
message: "AWS::EC2::Errors::AuthFailure exception while describing test Security Group-AWS was not able to validate the provided access credentials",
description: "Describe security group is needed to check if qubole related security group is present and check the details.",
UIOrder: 2
},
ec2:AuthorizeSecurityGroupEgress: {
status: "Warning",
message: "Could not validate as there was an error while creating Security Group",
description: "Qubole uses this permissions to attach inbound / outbound rules to Qubole defined security group while bringing up the cluster. Please make sure to provide access or you can avoid giving permission and attach rules manually before launching the cluster. Please note if you are not providing access to Create Security Group as well then please create the create security group manually as well. Please check the documentation for details.",
UIOrder: 3
},
ec2:AuthorizeSecurityGroupIngress: {
status: "Success",
message: "",
description: "Qubole uses this permissions to attach inbound / outbound rules to Qubole defined security group while bringing up the cluster. Please make sure to provide access or you can avoid giving permission and attach rules manually before launching the cluster. Please note if you are not providing access to Create Security Group as well then please create the create security group manually as well. Please check the documentation for details.",
UIOrder: 4
},
ec2:DeleteSecurityGroup: {
status: "Warning",
message: "Could not validate as there was an error while creating Security Group",
description: "Qubole automatically terminates qubole defined cluster security group while terminating the cluster. If you do not wish to provide permission then you can manually delete the security group.",
UIOrder: 5
}
},
Instance: {
ec2:RunInstances: {
status: "Success",
message: "",
description: "This give qubole to run the instance, to bring up cluster",
UIOrder: 1
},
ec2:RequestSpotInstances: {
status: "Error",
message: "AWS::EC2::Errors::AuthFailure exception while requesting spot EC2 instance-AWS was not able to validate the provided access credentials",
description: "For requesting Spot Node",
UIOrder: 2
},
ec2:CancelSpotInstanceRequests: {
status: "Success",
message: "",
description: "For cancelling request for Spot Node",
UIOrder: 3
},
ec2:CreateTags: {
status: "Warning",
message: "Could not validate as there was an error spawning test ec2 instance",
description: "This is used for billing purposes to identify the number of instances spun up by Qubole.",
UIOrder: 4
},
ec2:DeleteTags: {
status: "Warning",
message: "Could not validate as there was an error spawning test ec2 instance",
description: "This is used to delete tags post termination of the cluster.",
UIOrder: 5
},
ec2:StopInstances: {
status: "Warning",
message: "Could not validate as there was an error spawning test ec2 instance",
description: "To Stop an running instance",
UIOrder: 6
},
ec2:ModifyInstanceAttribute: {
status: "Warning",
message: "Could not validate as there was an error spawning test ec2 instance",
description: "To Modify the instance",
UIOrder: 7
},
ec2:StartInstances: {
status: "Success",
message: "",
description: "Restart after modifying instance",
UIOrder: 8
},
ec2:TerminateInstances: {
status: "Success",
message: "",
description: "In downscaling terminate",
UIOrder: 9
}
}
}
}