Visualising Using Python Plotting Libraries
You can visualize Python on the Spark driver by using the display(<dataframe-name>)
function.
The following Python libraries are supported:
plotly
matplotlib
seaborn
altair
pygal
leather
Note
The display()
function is supported only on PySpark kernels.
Using plotly
The following image shows the visualization of the plotly plot.

Using matplotlib
import pandas as pd
import matplotlib.pyplot as plt
plt.switch_backend('agg')
sdf = spark.sql("select * from default_qubole_airline_origin_destination limit 10")
data = sdf.toPandas()
data['distance'] = pd.to_numeric(data['distance'], errors='coerce')
data.plot(kind='bar', x='dest', y='distance', color='blue')
display(plt)
The following image shows the visualization of the matplotlib plot.

Using seaborn
The following image shows the visualization of the seaborn plot.

Using altair
The following image shows the visualization of the altair plot.

Using pygal
The following image shows the visualization of the pygal plot.

Using leather
The following image shows the visualization of the leather plot.

For other plot types, refer to the PlotExamplesPySpark.ipynb in the Example Notebooks of the Jupyter notebooks.