Data Visualization in Jupyter Notebooks

Jupyter Notebooks provide a data visualization framework called Qviz that enables you to visualize dataframes with improved charting options and Python plots on the Spark driver. Qviz provides a display() function that enables you to plot charts, such as table chart, pie chart, line chart, and area chart for the following data types:

• Spark dataframes

• pandas dataframes

• SQL (%%sql) magic

You can also create visualization for custom plots directly on the Spark driver by using the supported Python libraries.

Note

The display() function is supported only on PySpark kernels.

The Qviz framework provides various options to visualize data and customize the charts.

• Visualizing Spark Dataframes
• Visualizing pandas dataframes
• Visualizing SQL
• Visualising Using Python Plotting Libraries
• Using Qviz Options