- Startup
- Business
- Enterprise
- On-Premise
- Add-on
Overview
The Integration Agent can run Flows created in the Etlworks Integrator. The Flows can be executed manually, scheduled to run automatically, and stopped.
Process
Flows can be added, removed, and scheduled when the specific Integration Agent is running. The Agent will reconfigure itself on the next communication session between the Agent and the Etlworks Integrator. The communication session is always initiated by the Agent as we assume that the Etlworks Integrator cannot access the Agent which is running behind the firewall. You can change the frequency at which the Agent and the Integrator communicate to each other by changing the configuration parameter Agent Ping Delay
.
Here's how you can add Flows to run in the remote Integration Agent:
Step 1. In Etlworks Integrator, open the Agents
window and select the already created Integration Agent.
Step 2. Add previously created Flow(s) to the Agent.
Step 3. Select one of the available schedule types.
Step 4. Configure the schedule if the Schedule type is not Manual
.
Execute flow every n seconds
Execute flow periodically using a cron-based schedule
Step 5. Optionally provide a description for the flow.
The description is an additional attribute available when adding flows to the Agent. It can be used as an informational field only but it is required if you wish to schedule the same flow multiple times with different flow variables. Read about parametrization.
Step 6. Optionally add flow variables.
Click the Editor
icon next to the flow status
link.
then add flow variables as key-value pairs.
Read about parametrization.
Pass secure parameters such as passwords in theSecure parameters
box:
Filter flows by name, description, and tags
It is possible to filter flows, associated with the agent by
1. Tags.
2. Name and Description
Run the Flow manually
You can request to run any flow added to the Integrator Agent. If the Flow is not already running it will be executed on the next communication session between the Agent and the Etlworks Integrator. You can change the frequency at which the Agent and the Integrator communicate to each other by changing the configuration parameter Agent Ping Delay
.
The Run
button is disabled if the Agent is offline.
Stop the running Flow
You can request to stop any running flow added to the Integrator Agent. If the Flow is running it will be stopped on the next communication session between the Agent and the Etlworks Integrator. You can change the frequency at which the Agent and the Integrator communicate to each other by changing the configuration parameter Agent Ping Delay
.
The Stop
button is disabled if the Agent is offline.
Parametrization
It is possible to add flow variables to the flow executed by the Integration Agent. Flow variables can be referenced as {VARNAME}
anywhere in the code.
Examples:
- Parameterization of the source and destination queries.
- Parametrization of the command-line (shell) scripts.
Flow variables can be converted to global variables.
Read how to use global variables.
- Parameterization of the Connections
- Parameterization of transformations
- Parameterization of file operations
Read how to add flow variables to the flow executed by Integration Agent.
Monitor the Flow
Flow Statistics dashboard
Each time the Flow is executed by the Integration Agent (manually or automatically by the scheduler) it reports stats, metrics, and optionally console log for that specific run back to the Etlworks Integrator.
Unlike flows running in the cloud Etlworks Integrator, the tasks and the log for the flows executed by the Agent are not updated in real-time.
To access the history of runs as well as starts, metrics, and log for each run click the status
link next to the flow added to the Integration Agent.
The standard Etlworks Flow Statistics
dashboard will be displayed.
Flow Execution Log
By default, the Agent does not upload the Flow execution log into the cloud Etlworks Integrator instance. Read how to enable log upload.
When the log is available it can be viewed and downloaded from the Flow Statistics
dashboard.
Comments
0 comments
Please sign in to leave a comment.