Setting Additional Configuration¶
The advanced driver configuration and the configuration to create and list the Spark Apps are described in the following section.
To set the advanced configuration, perform these steps:
On Qubole ODBC Driver DSN Setup, click Advanced Configuration. The following dialog is displayed.
You can change the following properties under Advanced Configuration:
Maximum length of varchar column: If you do not want to use the default value, which is 100000.
However, some tools require the maximum value to be set below a particular limit.
Example: For Microsoft SQL Server, this value has to be set to 8000 or less than 8000.
Data Encoding type: You can choose the encoding type of the data as
Windows-1252. The default type is
Schema Name: Enter the schema name to filter to that schema while fetching metadata so that only the related
TABL_SCHEMconfigured is exposed.
Catalog Name: Specify the data catalog name.
UseS3: It is selected by default and it is set to bypass the QDS Control Plane for getting results directly from the S3 location.
Bucket Region: Specify the AWS region in which you have the S3 object storage for final results (default storage location (defloc)), which you use to read data from or write data into it. If you do not specify the AWS region, then it defaults to
Stream Results: Select it to enable Presto FastStreaming, which enables streaming of results directly from AWS S3 in the ODBC driver. This is in contrast to the earlier behavior wherein the driver waits for the query to finish before downloading any results from the QDS Control Plane or from S3. The streaming behavior can help the BI tool performance as results are displayed as soon as they are available in S3. Presto FastStreaming for ODBC driver is supported in Presto versions 0.193 and 0.208. As streaming cannot be used with Presto Smart Query Retry, the Presto FastStreaming feature automatically disables Presto Smart Query Retry.
Create a ticket with Qubole Support to enable the Presto FastStreaming feature on the account.
Append Limit for SqlPrepare: This configuration is now deprecated.
Virtual Hosted-Style URL: It is enabled by default. Set it to
falseto enable path-style URLs that allow a dot in the (
.) S3 bucket name of the default storage location (defloc). It is only supported from ODBC Driver version 2.2.1.
Listing and Creating Spark Apps¶
You can get the list of Spark Apps available with the entered API token and endpoint.
On Qubole ODBC Driver DSN Setup, choose Spark as Data Source Type. Ensure that the correct API token and endpoint have been entered. Click List Available Spark Apps. The Spark Apps dialog box is displayed.
Select one of the listed apps represented by the App ID or click CREATE NEW SPARK APP to create a new spark app.
For the new app, you can change the configuration value of
spark.dynamicAllocation.maxExecutors. Its default value is 10.