Parameterization of file operations

Overview

In Etlworks the FROM (source) and TO (destination) fields in a file operation can be dynamically configured using variables. This allows you to perform file operations like copy, move, delete, or rename with flexible paths that change based on runtime conditions.

How it works

To use variables, simply enter the {variable name} in any FROM or TO field. The system will automatically replace the {variable_name} with the value of the corresponding global variable during execution, enabling dynamic file paths based on the current context.

Use one of the following methods to set global variables:

• Set global variables in a Script : Define and assign values to global variables using a script.
• Database Loop Parameters : Set variables based on the results of a database query.
• File Loop Parameters : Loop through the files matching the wildcard name and set variables to file name.
• Flow variables set at the flow level : Set specific variables within the flow configuration.
• Variables set when executing flow manually : When running a flow manually, variables can be entered directly before the flow starts.
• Variables set when configuring the schedule : When setting up a scheduled flow, variables can be defined as part of the schedule configuration.
• Variables set when configuring the flow to be executed by Integration Agent : If the flow is executed by an Integration Agent, variables can be set in the agent’s flow configuration.
• URL parameters in call-flow-by-name API :Pass variables through URL parameters when triggering a flow by name.
• Payload in call-flow-by-ID API : Provide variables via the payload when triggering a flow by ID.
• User-defined API URL parameters : Use parameters defined in your custom API endpoints.
• HTTP Preprocessor : Set variables dynamically using an HTTP preprocessor script.

Example

A real-life example would be copying a response from the web service to the file with a new name calculated as constant_{current_timestamp}.json. The {current_timestamp} is a global variable set using a JavaScript.