1. What AWS regions are supported by Qubole?

Qubole currently supports following AWS regions:

• US East (N.Virginia) [us-east-1]

• US East (Ohio) [us-east-2]

• US West (N. California) [us-west-1]

• US West (Oregon) [us-west-2]

• Canada (Central) [ca-central-1]

• EU (Ireland) [eu-west-1]

• EU (London) [eu-west-2]

• Asia Pacific (Mumbai) [ap-south-1]

• Asia Pacific (Singapore) [ap-southeast-1]

• Asia Pacific (Tokyo) [ap-northeast-1]

• Asia Pacific (Seoul) [ap-northeast-2]

• South America (Sao Paulo) [sa-east-1]

Qubole also supports EU (Frankfurt); this requires a separate Qubole account, as explained below.

1.1. How do I use QDS in the EU (Frankfurt) AWS Region?

This section provides a brief introduction to using eu-central-1.qubole.com.

This version of QDS:

• Runs entirely in the local EU (Frankfurt) AWS region .

• Requires a separate Qubole account. If you already have an api.qubole.com account, you need to create a new account to use eu-central-1.qubole.com.

Why Use this QDS version?

eu-central-1.qubole.com ensures that all the data you work on stays in the local AWS region. Examples of why this may be useful or necessary include:

• Compliance: you may need to meet data residency requirements.

• Latency: if you primarily work in this region, improved latency may be a consideration: the QDS REST endpoint may be closer to your computation and storage resources.

Useful Documents

Make yourself familiar with the following documents before setting up a cluster in the EU:

AWS documentation:

• Global Infrastructure

• Regions and Availability Zones

  • AWS Regions

  • Describing Your Regions and Availability Zones

• AWS Security Credentials

• Setting Up with Amazon EC2

• Your VPC and Subnets

• Amazon Elastic Block Store (Amazon EBS)

• Amazon EC2 FAQs

Qubole documentation:

• Configuring Clusters

• Configuring a Cluster in a VPC with Public and Private Subnets

• Understanding Cluster Network Security Characteristics

Guidelines for Setting Up a Cluster

The following guidelines apply to setting up any QDS cluster:

• Make sure your cluster routing table is set up as described under Understanding Cluster Network Security Characteristics (this is the default configuration).

  • In particular, if you make changes, make sure TCP port 22 remains open to allow SSH connections from QDS tunnel servers. Tunneling with Bastion Nodes for Private Subnets in an AWS VPC lists the IP addresses of the Qubole tunnel servers.

• Use the default security configuration if possible.

  • If it’s not possible for your organization to use the default configuration, make sure you thoroughly understand the issues described in the links on this page and know how to proceed. If you need a custom security configuration and are not fully confident you can configure it correctly, contact Qubole Support before you start.

• If you move the cluster from EC2-Classic to a VPC, or vice versa, make sure you delete the old security group. (If the old group exists with the same name, QDS will use it and the cluster will fail to start.)

• Make sure that starting the cluster will not put your AWS account over the quota for the instance type you’ve chosen. (Remember the quota is AWS-account-wide; it includes any non-Qubole instances you may have.)

• Make sure your AWS credentials, and IAM credentials if any, are properly configured.

• If you use AWS instances with limited local storage (such as c3), be careful about configuring Elastic Block Store (EBS) volumes for the cluster (via EBS volume count on the Add New Cluster page of the QDS Control Panel):

  • If you don’t configure enough storage, jobs may run slowly or fail.

  • If you configure too much, you could possibly exceed your AWS account’s volume limit. (You can request an increase if necessary.)