Creating a Custom Hive MetastoreΒΆ

Qubole supports using a custom Hive metastore on the Qubole account. By default, each account comes with the Qubole Hive metastore but if you want to use a custom metastore, then you create a new metastore (if you do not have a metastore).


Migrating Data from Qubole Hive Metastore to a Custom Hive Metastore describes how to migrate the data from the Qubole-managed Hive metastore to the custom-managed Hive metastore. Connecting to a Custom Hive Metastore describes how to connect to a custom metastore.

If you face any intermittent lock or dead lock issues, see Intermittent Lock and Deadlock Issues in Hive Metastore Migration to resolve them.

Create a custom Hive metastore by performing these steps:

  1. Log into MySQL and create the metastore database and tables as shown in the example below.

     > mysql -uroot
    mysql> CREATE DATABASE <database-name>;
    mysql> USE <database-name>;
    mysql> SOURCE <metastore-schema-script>
  2. Create a MySQL user and grant access to the metastore database as illustrated in the example below.

    mysql> CREATE USER 'hiveUser'@'localhost' IDENTIFIED BY 'hivePassword';
    mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hiveUser'@'localhost';
    mysql> GRANT ALL ON hive_metastore.* TO 'hiveUser'@'localhost';
  3. Configure the hive-site.xml file with the above metastore properties, the database and user.

  4. Run the SQL scripts to create default tables for different Hive versions listed below:

  5. After creating the default tables in the custom Hive metastore, create a ticket with Qubole Support to enable the metastore.

  6. When the custom Hive metastore is enabled on the QDS account, you can connect to the custom Hive metastore as described in Connecting to a Custom Hive Metastore.