Uploading a File to Amazon S3 Buckets

This document is intended to guide a new user to upload a file into Amazon S3 buckets.

Configure CORS Policy on an Amazon S3 Bucket

You must configure the Cross Origin Resource Sharing (CORS) policy on an Amazon S3 bucket to which you want to upload a file.

If you are a new user, you can configure the CORS policy on the default storage Amazon S3 bucket that is set in Control Panel. Click the Account Settings tab in the Control Panel page.

Copy the default location, mentioned against the Default location (for the created data) that is listed below Storage Settings.

Perform these steps to configure CORS policy on an Amazon S3 bucket:

  1. Login to Amazon S3.

  2. Navigate to the default bucket, example.bucket.

    Note

    In the step above, example.bucket is a sample bucket name. Replace this with the name of your default bucket.

  3. Go to Permissions and click Add CORS Configuration.

  4. Repeat steps 2-3 to set the CORS policy on the Amazon S3 buckets to which you want to upload files.

A sample CORS configuration XML file is as shown below.

Note

The syntax below uses https://api.qubole.com as the endpoint. Qubole provides other endpoints to access QDS that are described in Supported Qubole Endpoints on Different Cloud Providers.

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>https://api.qubole.com</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Upload a File to an Amazon S3 Bucket from the Explore UI

Note

You must have object Storage resource permission for the role assigned to you for uploading and downloading data. For more information, see Resources, Actions, and What they Mean.

Perform these steps to upload a file into an Amazon S3 bucket using Qubole UI:

  1. Navigate to the Explore page. Click My Amazon S3 from the drop-down list.

    ../../_images/ExploreDropdown1.png
  2. Select the default storage location, example.bucket, from the drop-down list.

Note

Ensure the CORS policy has been set on the example.bucket bucket.

  1. Click the file upload icon FileUploadIcon. The file upload dialog is displayed.

Note

The maximum file size limit for uploads is 4GB.

  1. Click Browse to the location of the file and select the file. Click Submit. A success message is displayed if the upload is successful. An error message is displayed if the upload is unsuccessful.

Go to the corresponding Amazon S3 bucket and check to confirm the uploaded file is listed in the bucket.

Upload/Download a File from/to an Amazon S3 Location using Analyze and Notebooks UI

The section on the S3 tab describes how to upload/download a file to/from an Amazon S3 location through the Analyze UI.

Uploading and Downloading a File to or from a Cloud Location describes how to upload/download file to/from an Amazon S3 location through the Notebooks UI.