Parameterization of the Connections

Parameterization of the Connections

In Etlworks, most connection parameters—including host, port, credentials, and URLs—can be dynamically configured using tokens. Tokens are defined using the {variable_name} syntax and are replaced at runtime with the values of matching global variables.

Using Tokens in Connections

To parameterize a connection:

• Simply enter {variable_name} in any supported connection field.

• At runtime, Etlworks replaces the token with the corresponding global variable value.

This allows you to reuse connections across different flows, environments, or accounts by changing only the variables—not the connection itself.

Dynamic URL

You can use tokens in connection URLs, particularly useful for HTTP connections.

Example

https://my.api.com/metrics/?apiKey=111-222-333&periodStart={period_start}&periodEnd={period_end}

Process

Step 1: Create a Connection with Tokens

Create a connection (e.g., HTTP) that includes one or more {tokens} in fields such as the URL.

Step 2. Create a flow.

Use this connection as the source (FROM) or destination (TO) in your flow.

Step 3. Set Global Variables

Ensure that all required variables are set before the flow executes. You can set them using any of the following methods:

• Set global variables in a Script : Define and assign values to global variables using a script.
• Database Loop Parameters : Set variables based on the results of a database query.
• File Loop Parameters : Loop through the files matching the wildcard name and set variables to file name.
• Flow variables set at the flow level : Set specific variables within the flow configuration.
• Variables set when executing flow manually : When running a flow manually, variables can be entered directly before the flow starts.
• Variables set when configuring the schedule : When setting up a scheduled flow, variables can be defined as part of the schedule configuration.
• Variables set when configuring the flow to be executed by Integration Agent : If the flow is executed by an Integration Agent, variables can be set in the agent’s flow configuration.
• URL parameters in call-flow-by-name API :Pass variables through URL parameters when triggering a flow by name.
• Payload in call-flow-by-ID API : Provide variables via the payload when triggering a flow by ID.
• User-defined API URL parameters : Use parameters defined in your custom API endpoints.
• HTTP Preprocessor : Set variables dynamically using an HTTP preprocessor script.

Parameterization of the Database Connections

By default, Etlworks opens all database connections before a flow starts. This can cause issues if a database connection includes tokens, since global variables are typically set after the flow begins.

Use connection options that delay connection initialization until after global variables are set (e.g., dynamically create connections using scripts or avoid using tokenized connections for early validation steps).

Read how to configure database connections in a way that supports tokenization.