Understanding the parameters used for Snowflake¶
There are certain parameters for adding a Snowflake data warehouse as a data store to QDS, and for reading and writing to the Snowflake data warehouse. You should review these parameters and understand their significance in adding a data store, and performing read and write operations on Snowflake data.
The REST API parameters and parameters in the QDS UI are used to add or connect to the Snowflake data store. The parameters in the Qubole Dataframe API for Apache Spark (Dataframe API) are used to read data from or write data to the Snowflake data store.
The following table lists the parameters for Snowflake, the corresponding parameters in the UI, REST API, and the Qubole Dataframe API for Apache Spark, and their role in different use cases.
| Parameters | Corresponding Parameters in the QDS UI | Corresponding Parameters in the REST API | Use Case: Connecting to the Data Store | Corresponding Parameters in the Dataframe API | Use Case: Accessing the Snowflake Data by using the Dataframe API |
|---|---|---|---|---|---|
| Name of the catalog. This is a mandatory parameter to make all the data stores accessible through Spark clusters. | Catalog Name | catalog_name | Specifies a catalog name that is used in the Dataframe API to access Snowflake data. | catalog-name | catalog-name is mapped to the following user authentication credential fields: host name (Snowflake account URL), user name, and password. Therefore, when executing the Dataframe API, only catalog-name is specified and user authentication credentials are not specified. |
| Name of the data store that is created in QDS. | Data Store Name | name | Users can explore the data of the specified data store in the Explore page. | Not applicable | Users can connect to any database and warehouse that is associated with the added Snowflake account. |
| Type of database | Database Type | db_type | Value = ‘snowflake’. | Not applicable | Not applicable |
| Name of the database | Database Name | db_name | Users can explore data of the specified database in the Explore page. If the user wants to explore data of any other database, then the user should edit the value of the database field for the data store or add a new data store with the required database name. | database-name | Users can execute Spark commands on any database that is associated with the added Snowflake account by specifying the database name using this parameter. |
| Name of the warehouse | Warehouse Name | warehouse field in the extra_configs parameter. | Users can explore data of the specified warehouse in the Explore page. If the user wants to explore data of any other warehouse, then the user should edit the value of the warehouse field for the data store or add a new data store with the required warehouse name. | snowflake-virtual-warehouse-name | Users can execute Spark commands on any warehouse that is associated with the added Snowflake account by specifying the warehouse name using this parameter. |
| Name of the host | Host Address | db_host | Required to connect to the data store. | Not applicable | Not applicable as catalog-name is mapped to the user authentication credentials. |
| User name | Username | db_user | Required to connect to the data store. | Not applicable | Not applicable as catalog-name is mapped to the user authentication credentials. |
| Password | Password | db_passwd | Required to connect to the data store. | Not applicable | Not applicable as catalog-name is mapped to the user authentication credentials. |