Understanding the parameters used for Snowflake

There are certain parameters for adding a Snowflake data warehouse as a data store to QDS, and for reading and writing to the Snowflake data warehouse. You should review these parameters and understand their significance in adding a data store, and performing read and write operations on Snowflake data.

The REST API parameters and parameters in the QDS UI are used to add or connect to the Snowflake data store. The parameters in the Qubole Dataframe API for Apache Spark (Dataframe API) are used to read data from or write data to the Snowflake data store.

The following table lists the parameters for Snowflake, the corresponding parameters in the UI, REST API, and the Qubole Dataframe API for Apache Spark, and their role in different use cases.

Parameters

Corresponding Parameters in the QDS UI

Corresponding Parameters in the REST API

Use Case: Connecting to the Data Store

Corresponding Parameters in the Dataframe API

Use Case: Accessing the Snowflake Data by using the Dataframe API

Name of the catalog. This is a mandatory parameter to make all the data stores accessible through Spark clusters.

Catalog Name

catalog_name

Specifies a catalog name that is used in the Dataframe API to access Snowflake data.

catalog-name

catalog-name is mapped to the following user authentication credential fields: host name (Snowflake account URL), user name, and password. Therefore, when executing the Dataframe API, only catalog-name is specified and user authentication credentials are not specified.

Name of the data store that is created in QDS.

Data Store Name

name

Users can explore the data of the specified data store in the Explore page.

Not applicable

Users can connect to any database and warehouse that is associated with the added Snowflake account.

Type of database

Database Type

db_type

Value = ‘snowflake’.

Not applicable

Not applicable

Name of the database

Database Name

db_name

Users can explore data of the specified database in the Explore page. If the user wants to explore data of any other database, then the user should edit the value of the database field for the data store or add a new data store with the required database name.

database-name

Users can execute Spark commands on any database that is associated with the added Snowflake account by specifying the database name using this parameter.

Name of the warehouse

Warehouse Name

warehouse field in the extra_configs parameter.

Users can explore data of the specified warehouse in the Explore page. If the user wants to explore data of any other warehouse, then the user should edit the value of the warehouse field for the data store or add a new data store with the required warehouse name.

snowflake-virtual-warehouse-name

Users can execute Spark commands on any warehouse that is associated with the added Snowflake account by specifying the warehouse name using this parameter.

Name of the host

Host Address

db_host

Required to connect to the data store.

Not applicable

Not applicable as catalog-name is mapped to the user authentication credentials.

User name

Username

db_user

Required to connect to the data store.

Not applicable

Not applicable as catalog-name is mapped to the user authentication credentials.

Password

Password

db_passwd

Required to connect to the data store.

Not applicable

Not applicable as catalog-name is mapped to the user authentication credentials.