Connect to a Custom Metastore

POST /api/v1.2/custom_metastores

Use this API to connect to the custom metastore through the QDS. Currently, QDS supports connecting to only MySQL type of meta stores. For information on how to connect/edit to a metastore through the QDS UI, see Connecting to a Custom Hive Metastore.

Note

This feature is not enabled by default. To enable it, create a ticket with Qubole Support.

Required Role

The following users can make this API call:

  • Users who belong to the system-user or system-admin group.

  • Users who belong to a group associated with a role that allows submitting a command. See Managing Groups and Managing Roles for more information.

Parameters

Note

Parameters marked in bold below are mandatory. Others are optional and have default values.

Parameter

Description

db_name

This is the metastore name to be connected to the QDS.

db_host

It is the IP address or hostname of the metastore.

db_user

It is the user name to login to the metastore.

db_passwd

Password to login to the metastore.

db_port

It is the TCP Port to connect on to. If it is not specified, then the default port 3306 is used.

qubole_managed

It is set to false by default as you cannot use Qubole_managed Hive metastore and a custom metastore at the same time.

enable_cluster_access

Set it to true if the cluster has direct access to the custom metastore.

use_bastion_node

Set this parameter to true when the metastore that you want to connect to is within a VPC and is accessed through the Bastion node. It is set to false by default.

bastion_node_public_dns

It is the Public DNS address of the bastion node through which you can connect to the metastore. It becomes mandatory when use_bastion_node is set to true.

bastion_node_user

It is the user ID that is used to log into the Bastion node through QDS. It becomes mandatory when use_bastion_node is set to true.

bastion_node_private_key

It is the private key of the Bastion node. It becomes mandatory when use_bastion_node is set to true.

Request API Syntax

Here is the API syntax for the REST API call to connect to a metastore that is not within a VPC.

curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "<Metastore Name>", "db_host": "<n.n.n.n/hostname>", "db_port":"<Port>", "db_user":"user_name", "db_passwd":"<Password>"}' \
"https://api.qubole.com/api/v1.2/custom_metastores/"

Note

The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.

Here is the API syntax where you want to connect to a metastore that is in a VPC through the Bastion Node.

curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "<Metastore Name>", "db_host": "<n.n.n.n/hostname>", "db_port":"<Port>", "db_user":"user_name", "db_passwd":"<Password>",
    "use_bastion_node":"true", "bastion_node_public_dns" : "<bastion node Public DNS>", "bastion_node_user" : "<bastion node user name>",
    "bastion_node_private_key" : "<Bastion Node Private Key>"}' \ "https://api.qubole.com/api/v1.2/custom_metastores/"

Note

The above syntax uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.

Sample Requests

Here is a sample API call to connect to a custom metastore that is not in a VPC.

curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "HiveMetaStoreA", "db_host": "10.10.10.1", "db_user":"EC2-User", "db_passwd":"Met@1_P@ss"}' \
"https://api.qubole.com/api/v1.2/custom_metastores/"

Here is a sample request API to connect to a metastore that is within a VPC.

curl -X POST -H "X-AUTH-TOKEN:<AUTH-TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" \
-d '{"db_name" : "HiveMetaStoreA", "db_host": "10.10.10.1", "db_user":"EC2-User", "db_passwd":"Met@1_P@ss",
    "use_bastion_node":"true", "bastion_node_public_dns" : "10.10.10.8", "bastion_node_user" : "BastionAdmin",
    "bastion_node_private_key" : "<Bastion Node Private Key>"}' \
    "https://api.qubole.com/api/v1.2/custom_metastores/"