Parameterization of transformations

Overview

In Etlworks, the FROM (source) and TO (destination) fields in a source-to-destination transformation can be dynamically configured using variables. This allows for flexible and reusable transformations by replacing static values with dynamic ones during runtime.

How it works

Simply enter the {variable name} in any FROM or TO field. During execution, the system will automatically replace the {variable_name} with the value of the corresponding global variable, making your transformations adaptable to different contexts or datasets.

Use one of the following methods to set global variables:

• Set global variables in a Script : Define and assign values to global variables using a script.
• Database Loop Parameters : Set variables based on the results of a database query.
• File Loop Parameters : Loop through the files matching the wildcard name and set variables to file name.
• Flow variables set at the flow level : Set specific variables within the flow configuration.
• Variables set when executing flow manually : When running a flow manually, variables can be entered directly before the flow starts.
• Variables set when configuring the schedule : When setting up a scheduled flow, variables can be defined as part of the schedule configuration.
• Variables set when configuring the flow to be executed by Integration Agent : If the flow is executed by an Integration Agent, variables can be set in the agent’s flow configuration.
• URL parameters in call-flow-by-name API :Pass variables through URL parameters when triggering a flow by name.
• Payload in call-flow-by-ID API : Provide variables via the payload when triggering a flow by ID.
• User-defined API URL parameters : Use parameters defined in your custom API endpoints.
• HTTP Preprocessor : Set variables dynamically using an HTTP preprocessor script.

Example

A real-life example would be transforming a response from the web service to the file with a new name calculated as constant_{current_timestamp}.json. The {current_timestamp} is a global variable set using a JavaScript.