Overview

This flow runs scripts and command-line programs as part of an automation or orchestration workflow. It supports two execution modes:

Local execution (no connection configured)

The script runs directly on the operating system hosting the Etlworks instance.

The script is executed using the user account running Etlworks, typically the Tomcat user.

All standard interpreters available on the host can be used, for example:

sh or bash on Linux and macOS
cmd or PowerShell on Windows
python and python3
node
java -jar
docker
kubectl
mysql, psql, sqlplus
spark-submit

Remote execution over SSH

When an SSH connection is selected, the script is executed on the remote host associated with that connection.

Etlworks uses a secure SSH channel for transferring inline scripts and running the interpreter on the remote machine.

Inline scripts and script files

If you enter script text, it is encoded, transferred, and executed inline using the interpreter you select.
If you provide a full path and enable Is Path to Script, Etlworks executes the script file that already exists on the local or remote host.

This flexibility makes the flow suitable for orchestration, DevOps automation, CI tasks, triggering external services, and running real Python programs with full package support.

It covers the same use cases as Airflow operators such as BashOperator, SSHOperator, PythonOperator, and KubernetesPodOperator.

When to use this Flow type

Use this flow when you need to:

Run shell, Python, PowerShell, SQL CLI, or other command-line scripts as part of an Etlworks workflow
Execute commands on the local server hosting Etlworks
Execute commands on a remote Linux, macOS, or Windows machine over SSH
Orchestrate multi-step pipelines similar to Airflow
Trigger external apps, APIs, listeners, daemons, or services
Run Python programs using fully installed Python with pip packages
Invoke container commands, kubectl commands, or cloud CLI tools
Perform Git actions such as clone, commit, or push
Manage infrastructure or automation tasks using docker, aws, gcloud, az, kubectl, and others
Start long-running background scripts using Daemon mode

Creating the flow

Step 1. Optional: Create and test an SSH connection

If you intend to execute scripts on a remote host, create an SSH connection and verify connectivity.

If you leave the connection field empty, the script will run locally.

Step 2. Select the flow type

In the Flow list, type script and select:

Execute Script Local or Remote via SSH

Step 3. Select an SSH connection (optional)

If you want to run the script on a remote host, choose an SSH connection.

If you do not select a connection, the script will execute locally.

Step 4. Provide the script or script path

Under Script to execute or Path to Script, enter either:

The full script text
The absolute path to a script file on the local or remote host

If you enter a path, enable Is Path to Script.

Step 5. Configure optional parameters

See the next section for a full reference.

Step 6. Optionally add flow variables

Flow variables can be referenced inside the script using {VAR_NAME}.

Parameter reference

Script to execute or Path to Script

Enter the script text or a full path.

Inline scripts are encoded and executed using the selected interpreter.

Paths execute the existing script file.

Is Path to Script

Enable when the value above is a full path to a script file.

Script Type

Select the interpreter for execution. Examples:

sh, bash for Linux and macOS
cmd, powershell for Windows
python, python3, node
java -jar for running JAR files
docker, kubectl, aws, gcloud, az
psql, mysql, sqlplus
spark-submit

If set to any, the script is executed exactly as written.

Timeout (ms)

Maximum time allowed for the script to complete. After timeout, the process is forcibly terminated.

Success mask

If provided, and the script output contains this string, the execution is considered successful even if the exit code is non-zero.

Error mask

If provided, and the output contains this string, the execution is considered failed even if the exit code is zero.

Execute before

JavaScript or Python code to run before the main script.

This can be used to prepare variables, perform checks, or disable execution dynamically.

Flow Variables

Specify additional key-value pairs.

Variables can be referenced as {VAR_NAME} inside the script.

Keep log

If enabled, Etlworks saves stdout and stderr in the flow log.

Daemon

Runs the script continuously in a loop until stopped or an exception occurs.

Useful for watchers, listeners, or background agents.

On Exception, Exception Mask, Execute on Exception

Configure how the flow reacts to errors, including ignoring errors, matching specific exception patterns, or running code when an exception occurs.

Execute if Error

If enabled, the script is executed only when a previous step in the same nested flow fails.

Airflow-style orchestration examples

Replace Airflow BashOperator

echo "Hello from Etlworks"

Script Type: bash

Connection: none (local)

Runs exactly like Airflow BashOperator.

Replace Airflow PythonOperator using system Python

python3 /opt/app/scripts/process_data.py

or inline:

import json
print("Processing")

Script Type: python3

Install Python packages and run a program

Local host:

pip3 install --user pandas requests
python3 my_job.py

Remote host:

pip3 install pandas requests
python3 job.py

Run Node.js programs

Script Type: node

console.log("Node running");

Run PowerShell scripts on Windows

Script Type: powershell

Write-Host "Processing data"

Kubernetes automation

Script Type: kubectl

kubectl apply -f deployment.yaml

Cloud orchestration

Script Type examples: aws, gcloud, az

aws s3 sync /data s3://my-bucket

Database CLI automation

psql:

psql -h host -U user -d db -c "VACUUM"

mysql:

mysql -e "CALL refresh_data()"

sqlplus:

sqlplus user/pass@db @script.sql

Running Python scripts

Using Python installed on the host

You can run Python scripts locally or remotely exactly as if running them from a terminal.

Install packages (local)

pip3 install pandas requests

Install packages (remote)

pip3 install pandas requests

Use virtual environments

Create a venv:

python3 -m venv /opt/env
source /opt/env/bin/activate
pip install pandas requests

Run it from Etlworks:

python3 /opt/env/bin/python my_script.py

Inline Python