Introduction to Qubole Pipelines Service APIs (Pipelines APIs)

You can create streaming pipelines and perform various operations on these pipelines by using the APIs offered by Qubole Pipelines Services (Pipelines APIs).

Creating Streaming Pipelines

Note

You must ensure that the Pipelines resource is allowed for your role to create a streaming pipeline. Additionally, you must have the access to run Spark commands to run the streaming pipelines.

You can use the Create a Pipeline API to create a pipeline by using your own custom Jar files or code.

Alternatively, with the following APIs, you can create streaming pipelines by using your own custom Jar files or code:

1. Add properties.

2. Add your own code or JAR file.

3. Set alerts.

4. Start the pipeline.

To create and run a streaming pipeline by using the using out-of-box connectors and operators, use the Pipelines UI.

Managing Streaming Pipelines

Note

You must ensure that the Pipelines resource is allowed for your role to create a streaming pipeline.

After creating and running the pipelines, you can manage the streaming pipelines by using the following APIs:

• View a list of pipelines.

• View of list of events.

• Pause a pipeline.

• Delete a pipeline.

• View code for a pipeline.