Spark Structured Streaming¶
In this release, Spark Structured Streaming has the following enhancement.
Enhancements¶
- SPAR-4230: For the optimized performance, users must specify the
--conf spark.sql.streaming.stateStore.rocksDb.localDir=<tmp-path>option when using RocksDB based state storage for structured streaming. This prevents creation of multiple temporary directories, which can result in high consumption of disk space.
Bug Fixes¶
There are no bug fixes in this version.