Understanding Data Management on QDS

Qubole always caches metadata, command results, and the notebook paragraphs-results. Notebook paragraph-results and commands are stored in Amazon S3 and cached on Qubole serves.

Qubole caches:

  • Metadata for one day that is 24 hours
  • Command results for 7 days
  • Notebook paragraph-results for 30 days

Encrypting Cached Data

The cached data are encrypted and stored in Qubole servers. Create a ticket with Qubole Support to decrypt the encrypted-cached-data.

Encrypting Data at Rest

Data-at-rest simply means the data that is not moving. It can be in the server and client side. Qubole protects data-at-rest on Amazon S3.

Enabling Encryption for Data at Rest on Amazon S3 describes how to encrypt data on Amazon S3 by configuring different types of options.

Enable AWS Key Management Service Encryption in QDS describes the AWS Key Management Service that is used to protect the client-side data through encryption.