Run Flow by ID API

You can incorporate Etlworks into your own automation workflow by delegating heavy-duty integration tasks and executing Flows on demand.

Flow can be executed asynchronously (default behavior, the response is returned to the client immediately) and synchronously (the response is returned to the client after the Flow is executed).

Authentication

Before making a call to the built-in API, the user must receive a JWT token from the authentication API.

Step 1. Use any user with the Operator, Editor, or Administrator role to call an Etlworks authentication endpoint and receive an access token.

Step 2. Use the access token received in Step 1 to call the Etlworks API endpoints. The access token must be submitted as a header parameter, as in: Authorization:Bearer access-token.

Note: Access tokens in Etlworks are short-lived and self-expiring. An access token gives you access to the APIs for approximately 10 minutes. It is recommended that you refresh the access token before each call to the API.

The API endpoint parameters

PATH: /rest/v1/flows/{flowId}/run{?optional_query_parameters}.
METHOD: POST.
HEADER: Authorization:Bearer access-token.
REQUEST BODY: JSON document with arbitrary attributes that are passed to the Flow as Flow variables and global variables and can be referenced as {parameter_name}. They can be also accessed programmatically.

The global variables can be used in the FROM and TO fields in the source-to-destination transformation, as well as Connection parameters. The Flow variables can be referenced in the source and destination queries.
```
{
  "parameter_name": "parameter_value",
  "parameter_name2": "parameter_value2"
}
```
REQUEST CONTENT TYPE: application/json.

Response

The response is a JSON document in the following Format:

{
  "flowId": flowId,
  "messageUuid": "UUID of the message if flow was executed by sending payload to the custom endpoint", 
  "auditId": "ID of the record in audit table if flow was executed by name or by ID", 
  "status": status of the flow,
  "exception": "exception-stack-trace"
}

The status can be one of the following:

0: started. Just started.
1: running. Flow is still running.
2: canceled. Flow has been canceled.
3: wasn't running. Flow has not been executed.
4: success. Flow executed successfully.
5: error. Flow executed with an error (exception).

Response Codes

200 for success
401 and 403 for not authorized
500 for an internal error

Optional Query parameters

Execute Flow in parallel

To allow multiple instances of the same Flow to run in parallel, set the allowParallel parameter to true.

PATH: /rest/v1/flows/{flowId}/run?allowParallel=true

Serialize execution

When the same API endpoint, which executes the same Flow, is called while the flow is already running, status 1 (running) will be returned and this new execution request ignored. If you wish request to be added to the queue and executed in the order that the Flows were called you need to add waitIfRunning=true query parameter.

Execute Flow synchronously

By default, when you call this endpoint, the Flow is executed asynchronously, meaning the response is returned immediately after the call. Use the optional query parameter sync=true to execute Flow synchronously.

PATH: /rest/v1/flows/{flowId}/run?sync=true

Read how to parse the response from API when the Flow is executed synchronously.

Send email notifications

There are several query parameters that you can specify to receive email notifications based on execution status or duration.

emailOnSuccess: boolean (default value:false). If set to true and Flow was successfully executed, an email notification will be sent to the specified list of emails. Example: emailOnSuccess=true&emails=support@etlworks.com.
emailOnFailure: boolean (default value: false). If set to true and Flow has failed during the execution, an email notification will be sent to the specified list of emails. Example: emailOnFailure=true&emails=support@etlworks.com.
expectedTime: number of minutes. If Flow execution lasts longer than the specified number of minutes, an email notification will be sent to the specified list of emails. For example, send a warning notification if Flow runs longer than 5 minutes: expectedTime=5&emails=support@etlworks.com.
emails: URL encoded string. The comma or whitespace-separated list of email addresses that should receive notifications. This parameter is required if any of the three above were used.

Console log capturing

Console log capturing related parameters:

debug: boolean (default value: true). By default (if not specified explicitly in the URL) this parameter value is true and that means that system will record and store console logs captured during the Flow execution. Console log can be later viewed on the Flow statistics screen. If you don't want console log to be recorded by the system for performance gains set debug=false.
logStep: number (must be equal or greater than 100). Extra checkpoint records will be added to the log after every specified number of processed data records. For this parameter to take effect, debug parameter must be true, so either not specified, or explicitly set to true. Example: logStep=1000.

Return error code

Typically, when the Flow is executed by calling this API endpoint, the server returns response code 200 as long as the call successfully went through, regardless of the status of the Flow which has been executed.

When the Flow is executed synchronously and finished with an error or has been canceled, it is possible to return an error code (500), together with an actual exception.

Note: If the call to the API is made from the other Flow and the API returned HTTP 500, the calling Flow will fail with an exception.

PATH:/rest/v1/flows/{flowId}/run?sync=true&returnErrorCode=true
RESPONSE CODE: 500.
RESPONSE:

{
  "flowId": flowId,
  "messageUuid": "UUID of the message", 
  "auditId": "ID of the record in audit table", 
  "status": 5 or 2,
  "exception": "exception-stack-trace"
}

Force execution

If the flow was canceled prior to this request, but the actual ETL processes is potentially still running, status 2 (canceled) will be returned and this new execution request ignored. If you wish to forcefully execute this flow anyway, then you need to add force=true query parameter.

Stop flow executed by calling run Flow by name API endpoint

If you have multiple queued Flows and attempt to stop the Flow that is currently running, all Flows, current and queued, will be stopped.

Articles in this section