Running flows on a remote Integration Agent

Integration Agent can execute data integration flows created in the Etlworks web UI.

The flows are bundled with the connections, formats and other settings in a single JSON file and placed under the AGENT_HOME/config folder.

Step 1. Create flow using Etlworks Web UI.

Step 2. In Integrator open the Agents window and add a new agent (if Agent already exists skip this step).

Step 3. Add flow created in step 1 to the Agent and schedule it. 

You can keep the default scheduling pattern * * * * * since you are going to be configuring the actual schedule in the Agent's configuration file.

You can have as many flows added to the single agent as you want, also as many agents as you have a license for.

Step 4. Click Download Scenario.

Step 4. Copy downloaded JSON file into AGENT_HOME/config folder.

Step 5. Add the following properties to the AGENT_HOME/config/integration-agent.properties file:

 etl.request.flow_name.cron=cron-based pattern or time in milliseconds
 etl.request.flow_name.file=name of the downloaded JSON file
 etl.request.flow_name.enabled=true

Example

 etl.request.test.cron=* * * * *
 etl.request.test.file=copy files.json
 etl.request.test.enabled=true 

In the example above:

• test is the name of the scenario. Can be any name as long as it is the same for all 3 properties
• * * * * * is a schedule in the cron format (in this particular case it means every minute). You can also use a time interval in milliseconds.
• copy files.json is the name of the file downloaded from the Agent Web UI.
• true means the scenario is enabled. You can disable it at any time by setting this value to false.

You can have as many scenarios in the integration-agent.properties file as you need, each running in its own parallel thread, using its own schedule. Each scenario must have a unique name.

etl.request.copy.cron=* * * * *
etl.request.copy.file=copy files.json
etl.request.copy.enabled=true

etl.request.load.cron=10000
etl.request.load.file=load.json
etl.request.load.enabled=true

You can make any configuration changes in the integration-agent.properties file (such add, remove, enable and disable flows, as well as change the schedule) without stopping the agent.