Spark
This section explains how to configure and use Spark on a Qubole cluster. It covers the following topics:
- Introduction
- Supported Interfaces for Spark
- Understanding Spark Cluster Worker Node Memory and Defaults
- Running a Simple Spark Application
- Composing Spark Commands in the Analyze Page
- Composing Spark Commands in the Workbench Page
- Accessing Data Stores through Spark Clusters
- Connecting to Redshift Data Source from Spark
For details about supported Spark versions and deprecated Spark versions, see QDS Components: Supported Versions and Cloud Platforms.
For more information about the features in older supported versions of Spark, see Apache Spark Documentation.
What’s New |
|---|
Qubole supports the latest Apache Spark 3.0.0 version. Qubole bundles various performance, cost optimization, and usability changes on top of the open source Apache Spark 3.0.0 to provide the most performant, open, and secure Apache spark distribution. For more information about these features, see Spark 3 on Qubole. |