Migrating Data from Qubole Hive Metastore to a Custom Hive Metastore¶
By default, a QDS account comes with the Qubole-managed Hive metastore.
Qubole provides you an option to switch to a custom Hive metastore as described in Connecting to a Custom Hive Metastore.
Creating a Custom Hive Metastore describes how to create a custom Hive metastore from the very beginning.
To migrate the data from the Qubole-managed Hive metastore when you decide to a custom Hive metastore, the following section provides the steps.
Ensure that there is a downtime during the data migration from Qubole-managed Hive metastore to the custom Hive metastore.
Migrating Data from the Qubole-managed Hive Metastore¶
Perform these steps to migrate data from Qubole-managed Hive metastore to the custom Hive metastore:
If you face any intermittent lock or dead lock issues, see Intermittent Lock and Deadlock Issues in Hive Metastore Migration to resolve them.
- Ensure that you have a database instance that is appropriate for the workload.
- For security reasons, RDS is recommended to be in a private subnet and its bastion node can be in a public subnet.
- Create a ticket with Qubole Support requesting for the Qubole Hive metastore data dump.
- After receiving the DB dump from the Qubole Support, push the DB dump on the RDS instance that you have identified (refer to step 1).
- It is an optional step. For steps to create a metastore from the very beginning, see Creating a Custom Hive Metastore.