Startup
Business
Enterprise
On-Premise
Add-on

Overview

The Integration Agent can run Flows created in the Etlworks Integrator. The Flows can be executed manually, scheduled to run automatically, and stopped.

Process

Flows can be added, removed, and scheduled when the specific Integration Agent is running. The Agent will reconfigure itself on the next communication session between the Agent and the Etlworks Integrator. The communication session is always initiated by the Agent as we assume that the Etlworks Integrator cannot access the Agent which is running behind the firewall. You can change the frequency at which the Agent and the Integrator communicate to each other by changing the configuration parameter Agent Ping Delay.

Here's how you can add Flows to run in the remote Integration Agent:

Step 1. In Etlworks Integrator, open the `Agents` window and select the already created Integration Agent.

Step 2. Add previously created Flow(s) to the Agent.

Step 3. Select one of the available schedule types.

Step 4. Configure the schedule if the Schedule type is not `Manual`

Execute flow every n seconds

Execute flow periodically using a cron-based schedule

Step 5. Optionally provide a description for the flow.

The description is an additional attribute available when adding flows to the Agent. It can be used as an informational field only but it is required if you wish to schedule the same flow multiple times with different flow variables. Read about parametrization.

Step 6. Optionally add flow variables.

Click the Editor icon next to the flow status link.

then add flow variables as key-value pairs.

Read about parametrization.

Pass secure parameters such as passwords in theSecure parametersbox:

Step 7. Optionally configure email notifications.

When creating or modifying a Schedule, it is possible to configure the email notifications.

Click Email button

Configure Notifications

The following notifications are available:

Send email on success

This notification is sent when the Flow is successfully executed.

Step 1. Enable On success.

Step 2. Use Emails to configure the email address(es) that will be receiving a notification.

Send email on failure

This notification is sent when the Flow is either failed or canceled.

Step 1. Enable On failure.

Step 2. Use Emails to configure the email address(es) that will be receiving a notification.

Send an email if Flow execution is taking longer than expected

This notification is sent when the Flow execution is taking longer than expected. The notification will be sent one time when the actual run time exceeds the configured threshold.

Step 1. Use Expected flow execution time to specify the expected Flow execution time in minutes.

Step 2. Use Emails to configure the email address(es) that will be receiving a notification.

Send an email and stop the flow if Flow execution is taking longer than maximum allowed time in minutes

This notification is sent and the flow is stopped when the Flow execution is taking longer than maximum allowed time in minutes. The notification will be sent one time when the actual run time exceeds the configured threshold.

Step 1. Use Stop flow after time to specify the maximum allowed Flow execution time in minutes.

Step 2. Use Emails to configure the email address(es) that will be receiving a notification.

The stop-flow signal will be send to the agent but it is not guaranteed that the flow will be actually stopped. There are cases when the flow is stuck when executing not cancellable IO operation (for example, opening the database connection with no configured timeout). In this cases the flow will continue running until it is finally able to stop or the agent is rebooted.

Filter flows by name, description, and tags

It is possible to filter flows, associated with the agent by

1. Tags.

2. Name and Description

Run the Flow manually

You can request to run any flow added to the Integrator Agent. If the Flow is not already running it will be executed on the next communication session between the Agent and the Etlworks Integrator. You can change the frequency at which the Agent and the Integrator communicate to each other by changing the configuration parameter Agent Ping Delay.

The Run button is disabled if the Agent is offline.

Stop the running Flow

You can request to stop any running flow added to the Integrator Agent. If the Flow is running it will be stopped on the next communication session between the Agent and the Etlworks Integrator. You can change the frequency at which the Agent and the Integrator communicate to each other by changing the configuration parameter Agent Ping Delay.

The Stop button is disabled if the Agent is offline.

Parametrization

It is possible to add flow variables to the flow executed by the Integration Agent. Flow variables can be referenced as {VARNAME} anywhere in the code.

Examples:

Flow variables can be converted to global variables.

Read how to use global variables.

Read how to add flow variables to the flow executed by Integration Agent.

Monitor the Flow

Flow Statistics dashboard

Each time the Flow is executed by the Integration Agent (manually or automatically by the scheduler) it reports stats, metrics, and optionally console log for that specific run back to the Etlworks Integrator.

Unlike flows running in the cloud Etlworks Integrator, the tasks and the log for the flows executed by the Agent are not updated in real-time.

To access the history of runs as well as starts, metrics, and log for each run click the status link next to the flow added to the Integration Agent.

The standard Etlworks Flow Statistics dashboard will be displayed.

Flow Execution Log

Read how to enable and configure log upload.

When the log is available it can be viewed and downloaded from the Flow Statistics dashboard.

Articles in this section

Schedule and Run Flows in the Integration Agent

Overview

Process

Step 1. In Etlworks Integrator, open the `Agents` window and select the already created Integration Agent.

Step 2. Add previously created Flow(s) to the Agent.

Step 3. Select one of the available schedule types.