Enabling Encryption of Ephemeral Data in QDS Clusters

Azure

See Encryption for Data at Rest on Azure.

AWS

Qubole has enabled SSE-KMS encryption for the data at rest on EBS volumes. For enabling block device encryption on the cluster nodes’ ephemeral (local) storage, create a ticket with Qubole Support.

Note

Qubole would continue to honor the block-device encryption that was set in R57 and earlier versions through the UI option in the cluster configuration.

The block device encryption includes HDFS and any intermediate output generated by Hadoop. It is set up on the local devices before the node joins the cluster; this can increase the time it takes to bring up and upscale the cluster.

Note

The SSE-KMS/SSE-C encryption is only supported for instance types described in: EBS Encryption. You must only use the supported instances described in the above referenced topic (EBS Encryption) if you require the SSE-KMS/SSE-C encryption for EBS volumes.

For custom KMS keys, you must provide additional permissions to the IAM Role that are mentioned in Sample Policies for EBS Encryption.

Qubole has enabled SSE-KMS encryption on EBS volumes by default.

Note

You cannot view the block device encryption settings in the AWS console. You can verify it by running the lsblk command on cluster instances.