Connect to a Custom Metastore
- POST /api/v1.2/custom_metastores
Use this API to connect to the custom metastore through the QDS. Currently, QDS supports connecting to only MySQL type of meta stores. For information on how to connect/edit to a metastore through the QDS UI, see Connecting to a Custom Hive Metastore.
Note
This feature is not enabled by default. To enable it, create a ticket with Qubole Support.
Required Role
The following users can make this API call:
Users who belong to the system-user or system-admin group.
Users who belong to a group associated with a role that allows submitting a command. See Managing Groups and Managing Roles for more information.
Parameters
Note
Parameters marked in bold below are mandatory. Others are optional and have default values.
Parameter |
Description |
|---|---|
db_name |
This is the metastore name to be connected to the QDS. |
db_host |
It is the IP address or hostname of the metastore. |
db_user |
It is the user name to login to the metastore. |
db_passwd |
Password to login to the metastore. |
db_port |
It is the TCP Port to connect on to. If it is not specified, then the default port 3306 is used. |
qubole_managed |
It is set to false by default as you cannot use Qubole_managed Hive metastore and a custom metastore at the same time. |
enable_cluster_access |
Set it to |
use_bastion_node |
Set this parameter to |
bastion_node_public_dns |
It is the Public DNS address of the bastion node through which you can connect to the metastore.
It becomes mandatory when |
bastion_node_user |
It is the user ID that is used to log into the Bastion node through QDS.
It becomes mandatory when |
bastion_node_private_key |
It is the private key of the Bastion node.
It becomes mandatory when |
Request API Syntax
Here is the API syntax for the REST API call to connect to a metastore that is not within a VPC.
curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "<Metastore Name>", "db_host": "<n.n.n.n/hostname>", "db_port":"<Port>", "db_user":"user_name", "db_passwd":"<Password>"}' \
"https://api.qubole.com/api/v1.2/custom_metastores/"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Here is the API syntax where you want to connect to a metastore that is in a VPC through the Bastion Node.
curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "<Metastore Name>", "db_host": "<n.n.n.n/hostname>", "db_port":"<Port>", "db_user":"user_name", "db_passwd":"<Password>",
"use_bastion_node":"true", "bastion_node_public_dns" : "<bastion node Public DNS>", "bastion_node_user" : "<bastion node user name>",
"bastion_node_private_key" : "<Bastion Node Private Key>"}' \ "https://api.qubole.com/api/v1.2/custom_metastores/"
Note
The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.
Sample Requests
Here is a sample API call to connect to a custom metastore that is not in a VPC.
curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "HiveMetaStoreA", "db_host": "10.10.10.1", "db_user":"EC2-User", "db_passwd":"Met@1_P@ss"}' \
"https://api.qubole.com/api/v1.2/custom_metastores/"
Here is a sample request API to connect to a metastore that is within a VPC.
curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "HiveMetaStoreA", "db_host": "10.10.10.1", "db_user":"EC2-User", "db_passwd":"Met@1_P@ss",
"use_bastion_node":"true", "bastion_node_public_dns" : "10.10.10.8", "bastion_node_user" : "BastionAdmin",
"bastion_node_private_key" : "<Bastion Node Private Key>"}' \
"https://api.qubole.com/api/v1.2/custom_metastores/"