- Starter
- Business
- Enterprise
- On-Premise
- Add-on
Overview
This article covers the complete lifecycle of the remote Integration Agent: creating and registering an Agent, installing it natively or in Docker, licensing and connectivity, configuration, service control, updates, migration from legacy installations, JVM maintenance, backup and restore, connectors, and monitoring.
Keep three different command surfaces distinct:
- Integration Agent CLI — etlworks-agent.bat (Windows) / ./etlworks-agent (Linux and macOS), located directly in the Agent home. This is the tool documented here. The files under bin are implementation files and backward-compatible entry points; you should not need them.
- Full application host CLI — etlworks-cli, which manages the Etlworks application itself. Its commands and flags are not interchangeable with the Agent CLI; see Running Etlworks on-premise.
- Built-in application CLI/API — commands executed through the Etlworks instance.
A native Agent does not require the full application stack (Tomcat, PostgreSQL, Redis) on its host — it is a standalone process that communicates outbound with your Etlworks instance.
Modern (10.x) and legacy (9.x) Agents
Starting with 10.0.0, the Agent uses a managed installation layout with a single lifecycle CLI, a private Java runtime, and transactional updates. Legacy 9.x Agents use the older flat layout with per-task scripts (install.cmd/install.sh, integrationagent.exe). Modern instructions are the default throughout this article; applicable legacy procedures are kept in the clearly labeled legacy section, and the move from legacy to modern is the legacy installation migration.
Distinguish four different identities when planning any operation: the product version (for example 10.0.0; published Agent release names can legitimately retain a -SNAPSHOT suffix — use the actual published identity), the Java target (a modern release is built for Java 25 or Java 8; packages and update archives are target-specific), the installation format (modern managed layout vs legacy flat layout — a Java 8 package built for 10.x is not the same as a legacy 9.x installation), and the component versions (Agent payload, CLI, updater, and JVM are separate components that update separately).
Which operation do I need?
| Operation | Purpose |
|---|---|
| Full native installation | Initialize a managed installation and register the Agent and updater services |
| Ordinary Agent update — automated (updater service) or manual (downloaded ZIP) | Replace the Agent payload within a compatible managed installation |
| Independent updater refresh | Install or update the updater separately |
| Legacy migration bridge | Convert an eligible legacy 9.x installation to the modern managed layout |
| Private JVM patch | Change the approved JVM patch within the installed Java major |
| Docker image replacement | Update a containerized Agent's software and JVM |
| Persistent-state restore | Restore configuration and Agent state from an explicit backup |
Three rules prevent most confusion: an ordinary Agent update does not replace the CLI, the updater, or the JVM; a Java update does not migrate Java 8 to Java 25 (that requires a new installation or the migration); and a Docker update replaces the image — it is not an in-container native ZIP update.
Add new Integration Agent
Step 1. Open the Agents window. Click the Add Agent button.
Step 2. Enter the Agent name, click Save.
Note: Duplicate agent copies the Agent definition (name, configuration) and creates a new registration identity — it does not clone a running installation. Each installed Agent must run with its own downloaded registration; do not run multiple installations with the same registration identity.
Managing Integration Agents
The Agents page allows you to monitor, control, and maintain all Integration Agents connected to your Etlworks environment. From this page you can view the status and health of each Agent; start, stop, or restart Agents; monitor Agent version, OS, IP address, and updater status; suspend Agents temporarily; update Agents individually or in bulk; duplicate or delete Agents; and access Agent diagnostics and logs.
Table and Card Views
The Agents page supports two visual layouts: Card view and Table view. Switch between them using the view selector in the toolbar.
Card View
(default)
Card view displays each Agent as an individual card showing the Agent name, operating system, Agent version, IP address, Agent status (Online / Offline), and updater status (Running / Stopped). Card quick actions: start or stop the Agent, restart, download logs, view Agent status, suspend or resume, update, and delete. Card footer tools use compact icons with tooltips; the suspend switch keeps its Suspend/Resume tooltip and accessible label.
Table View
Table view displays Agents in a compact grid with the same key information per row. It is optimized for environments with many Agents and works well with sorting, filtering, and bulk operations. Switching between views does not affect Agent configuration or operation.
Context Menu (Right-Click Menu)
Right-click an Agent in the grid to open the context menu. For a single Agent: Open agent (configuration screen), View status (monitoring dashboard), Duplicate agent, Delete agent. With multiple Agents selected, the menu also provides Bulk update selected agents, Advanced options, and Delete selected agents.
Bulk Operations on Agents
Select one or more Agents, open the Actions menu or the right-click context menu, and choose Bulk update selected agents, Advanced options (the Bulk Actions dialog), or Delete selected agents.
Advanced Bulk Actions
The Advanced bulk actions dialog supports: Name prefix, Name suffix, Name regex replace, Add tags, Remove tags, and Set tags. Bulk operations help maintain consistent naming and tagging across large deployments.
Set up a new Agent: the complete sequence
Setting up an Agent always follows the same sequence. Each step links to its detailed section:
Step 1. Create and save the Agent in the appropriate Etlworks instance and tenant — Add new Integration Agent.
Step 2. Download the Agent package: select the Agent in the Agents window, click Download, choose the operating system, and download the package matching your platform and Java target — or plan a Docker deployment with a published image instead.
Step 3. Download the Agent's license: Download → License. The same file is used on every OS.
Step 4. Install: extract the package into a dedicated directory and run the installer for your platform — Linux, Windows, or macOS (or foreground/no-service for supervised environments). Installation registers the Agent and updater services but starts neither, and requires no license.
Step 5. Install the license, start, and verify — Install the license and start the Agent: place the downloaded .alic file in AGENT_HOME/license/, start the updater and the Agent, confirm the Agent is Online, and run a representative flow.
Licensing (modern Agents)
- An app-managed Agent requires its downloaded per-Agent .alic file in AGENT_HOME/license/.
- The shared integration-agent-online.lic and client.ks defaults are bundled inside the modern Agent runtime — you no longer copy them manually, and an empty external license directory does not hide the bundled defaults. Existing external shared-license/keystore overrides keep their priority.
- Installing the services without a .alic file is supported (useful for pre-provisioning); successful app-managed startup still requires valid registration and connectivity.
- Updating the Agent runtime never replaces your external registration files.
Note: The legacy instruction to delete *.alic/*.chk files before installing a new license applies to legacy installations. On a modern Agent, simply place the downloaded .alic file in license/; the Agent legitimately rewrites its activation-cache files at startup and polling.
Connectivity requirements
- The application URL embedded in the downloaded registration must be reachable from the Agent host. If the URL changes, regenerate and re-download the registration — editing a local URL property is not necessarily a substitute.
- In Docker, localhost refers to the container itself, not the host. A normally resolvable, reachable DNS name needs no special configuration; host-name overrides (Compose extra_hosts) are only for names that do not resolve from inside the container.
- Communication is outbound only — the Agent and updater poll the instance. No inbound listener is required for remote actions, and CLI lifecycle operations do not require enabling the optional Agent web dashboard.
- If the instance sits behind a reverse proxy or uses a private TLS certificate, the Agent host must trust that certificate chain. Configure the HTTP Proxy parameters when outbound traffic must pass a proxy, including authentication where enforced.
- Private JVM downloads are a separate network requirement: native installation downloads a checksum-pinned Azul Java package over HTTPS from the vendor. Allow that, or use the offline archive option.
Modern installation layout
| Directory | Contents |
|---|---|
| bin/ | CLI implementation files (CLI JAR and compatibility launchers). The customer-facing launcher — etlworks-agent.bat / etlworks-agent — sits directly in the Agent home root |
| config/ | Configuration (integration-agent.properties and related files) |
| license/ | Your per-Agent .alic registration (shared defaults are bundled in the runtime) |
| data/ | Agent working data |
| jdbc/ | Bundled connector/driver folders, refreshed by updates |
| jdbc-custom/ | Your driver overlays — survives updates; use this to override bundled drivers |
| lib/ | Engine and shared libraries (including SQL Server JDBC) |
| logs/ | IntegrationAgent.log (Agent) and UpdateService.log (updater) |
| backups/ | Explicit backups and retained recovery material |
| downloads/ | Download staging |
| update/ | The update inbox — place update ZIPs here |
| state/ | Installation and JVM selections (install.properties) |
| jvm/versions/ | Private Azul Java runtimes |
| updater/ | The updater service |
Points to understand:
- There is one active Agent payload — no application release history is retained on disk.
- Native packages do not include Docker tooling and do not create a plugin directory; an existing customer-created plugin directory remains supported and is preserved and backed up. Docker has its own plugin persistence and initialization.
- Updates replace each bundled jdbc folder as a complete bundle, retain extra folders absent from the bundle (for example a customer jdbc/salesforce), and apply the jdbc-custom overlay. Keep user-managed drivers in jdbc-custom, not directly in jdbc.
Native installation (10.x)
Extract the full Agent archive for your platform and Java target into a new, dedicated directory (for example /opt/etlworks-agent or C:\etlworks\integration-agent) and run the installation from that Agent home. The full installer already includes the updater — no separate updater archive is needed for a fresh installation.
What installation does — and does not do. Modern native installation registers both the Agent and updater services but starts neither:
- Agent boot policy: manual — it does not start on OS boot; start it explicitly through the CLI or the Etlworks UI.
- Updater boot policy: automatic — it starts on OS boot once you have started it the first time.
- Immediately after installation, installed=true, running=false (and a nonzero exit code from status) is the expected state, not an error.
- install --start is rejected; startup is always an explicit step.
The installer downloads a checksum-pinned private Azul Java runtime into jvm/versions. It does not use apt/yum/dnf to install Java, does not change system Java or JAVA_HOME, and validates the downloaded package's SHA-256 and version before use.
install options (both platforms unless noted):
- --user <account> — the service account, on supported Unix service managers (Linux/macOS). It is not a Windows account/password setting.
- --max-memory <MB> — maximum Agent heap; minimum 256 MB (default: 70% of system RAM).
- --java-archive <path> — offline JVM archive for the bootstrap; it must match the pinned runtime manifest.
- --no-service — initialize the Agent runtime without registering services (foreground/container use).
- --java-home <path> and --release <path> — advanced explicit runtime/layout overrides. --release is not a downloadable-version selector.
Install on Linux
Prerequisites:
- Root privileges for service registration; a running systemd.
- An existing service account with suitable directory permissions — the installer does not create one. Prefer a dedicated account and a dedicated directory.
- polkit with JavaScript rules support, when the updater runs as a non-root account (the installer writes a narrowly scoped rule authorizing only that account to start/stop/restart the Agent unit).
- curl or wget, sha256sum or shasum, and tar; working CA trust for the HTTPS JVM download.
- A glibc-based x64 or ARM64 distribution — the supplied JVM packages do not support Alpine/musl, and the installer rejects it before downloading.
Install and start (from the Agent home):
cd /opt/etlworks-agent
sudo tar -zxvf integration-agent-<version>-java25-linux.tar.gz
sudo ./etlworks-agent install --user etlworks
sudo ./etlworks-agent start
sudo ./etlworks-agent updater startetlworks is an example of an existing account to run the services under — substitute your own. Without --user, the CLI uses the sudo caller's account. Install the Agent and updater under the same service account. Optional: --max-memory <MB> sets the maximum Agent heap (default: 70% of system RAM).
Continue with Install the license and start the Agent. Local configuration lives in config/integration-agent.properties; most settings can instead be managed from the app.
Install on Windows
From an elevated PowerShell or Command Prompt, in the extracted Agent home:
cd C:\etlworks\integration-agent
etlworks-agent.bat install
etlworks-agent.bat start
etlworks-agent.bat updater startetlworks-agent.bat sits directly in the Agent home and invokes PowerShell internally — you do not need to open PowerShell or change the execution policy to use the CLI. PowerShell remains optional; from a PowerShell window use:
Set-Location C:\etlworks\integration-agent
.\etlworks-agent.bat statusExtract all files from the package together — the launcher, CLI JAR, and manifests belong to one build and must not be mixed across versions.
If Windows blocks the CLI scripts
Even from an elevated prompt, built-in Windows script policies and third-party protection software can stop the CLI scripts (.bat and .ps1) from running. Resolve this before installing or updating.
Built-in Windows policies (PowerShell execution policy and blocked downloaded files)
- Start the CLI through etlworks-agent.bat — the wrapper handles the execution policy for its child process.
-
Files downloaded from the internet carry the Mark of the Web, and Windows can refuse to run them with errors such as "running scripts is disabled on this system" or "... is not digitally signed". Extracting a downloaded ZIP with Windows Explorer propagates the mark to every file. After extracting, unblock the files — right-click each file → Properties → Unblock, or for the whole Agent home:
powershell.exe -NoProfile -Command "Get-ChildItem -Path C:\etlworks\integration-agent -Recurse | Unblock-File" - If the execution policy is enforced by Group Policy, the Bypass flag cannot override it. Check with Get-ExecutionPolicy -List: a value under MachinePolicy or UserPolicy comes from Group Policy — ask your Windows administrator to allow the Agent CLI scripts before installing.
Protection software (antivirus / EDR)
Security products can block or quarantine command-line scripts even for Administrators. Typical symptoms: the window closes immediately, Access is denied, or the CLI reports a required helper file is missing (a sign a file was quarantined). Work with your security team to allow the Agent home folder (for example C:\etlworks\integration-agent) before installation or updates, restore anything quarantined, then re-extract the complete package — the CLI requires every file from it — and re-run the command.
Service names: the Agent installs as IntegrationAgent (startup type Manual) and the updater as EtlworksUpdateService (startup type Automatic). The Windows service uses the unmodified Apache Procrun executable with the private Java runtime's absolute path — installing another Java on the host does not change the Agent's runtime.
Install on macOS
macOS uses launchd: installation writes the LaunchDaemon definitions but defers loading until explicit start. Run the same install/start/updater start commands with sudo from the extracted macOS archive. The updater runs at load; the Agent is started explicitly. Validate your deployment on the target machine — macOS receives the same managed layout, but qualify it in your environment before production use.
Install the license and start the Agent
Installation registers the services but starts nothing and requires no license. To bring the Agent online:
- Download the Agent's license: in the Agents window select the Agent, click Download, then License. The same file is used on every OS.
- Place the downloaded .alic file in AGENT_HOME/license/. Nothing else needs to be copied — the shared license files are bundled in the modern runtime.
-
Start the updater and the Agent:
cd /opt/etlworks-agent sudo ./etlworks-agent updater start sudo ./etlworks-agent startWindows, from an elevated terminal:
cd C:\etlworks\integration-agent etlworks-agent.bat updater start etlworks-agent.bat start - Verify: the Agent shows Online in the Agents window (allow up to a minute for the first ping), and the updater shows Running. Then run a representative flow.
Foreground execution (no services)
For containers, test rigs, and supervised environments, initialize the runtime only and run in the foreground:
Linux
cd /opt/etlworks-agent
./etlworks-agent install --no-service
./etlworks-agent runWindows
cd C:\etlworks\integration-agent
etlworks-agent.bat install --no-service
etlworks-agent.bat runForeground execution uses the selected private runtime. Process supervision and restart policy are your responsibility, and your supervisor's shutdown signal handling and timeout must allow the Agent's own shutdown timeout to complete.
Service control and diagnostics
Command syntax:
<launcher> <command> [subcommand] [options]where <launcher> is etlworks-agent.bat on Windows and ./etlworks-agent on Linux/macOS (with sudo where needed). Put commands and subcommands before options, and quote paths that contain spaces. Run from the Agent home; service start/stop/install operations require root/Administrator, read-only commands do not.
- help, --help, -h, or no arguments display the general help. Per-command help is not available.
- version / --version display the CLI's build identity.
- --verbose prints exception details for ordinary CLI failures (not accepted by the migration parser).
- --home <directory> is an advanced target-home override for commands that support it; normal usage is to run the launcher in the intended Agent home.
- Options are command-specific, not universal — for example --skip-confirm applies only to confirmation-bearing operations.
Note — older 10.x installations without the root launcher: fresh native installers include the root launchers, and migrated installations retain them. If an older 10.x installation lacks them, copy only the matching root launcher from a rebuilt native installer into the Agent home (on Linux/macOS, then run chmod +x etlworks-agent). Regular update ZIPs do not add launchers — do not extract a full installer over an existing installation.
cd /opt/etlworks-agent
./etlworks-agent help
./etlworks-agent version
./etlworks-agent status
sudo ./etlworks-agent start
sudo ./etlworks-agent stop
sudo ./etlworks-agent restart
./etlworks-agent updater status
sudo ./etlworks-agent updater start
sudo ./etlworks-agent updater stop
sudo ./etlworks-agent updater restart
./etlworks-agent health
./etlworks-agent doctor
./etlworks-agent java status
./etlworks-agent logs --lines 200
./etlworks-agent logs --followWindows — run from a Command Prompt as Administrator (PowerShell is optional; use .\etlworks-agent.bat there):
cd C:\etlworks\integration-agent
etlworks-agent.bat help
etlworks-agent.bat version
etlworks-agent.bat status
etlworks-agent.bat start
etlworks-agent.bat stop
etlworks-agent.bat restart
etlworks-agent.bat updater status
etlworks-agent.bat updater start
etlworks-agent.bat updater stop
etlworks-agent.bat updater restart
etlworks-agent.bat health
etlworks-agent.bat doctor
etlworks-agent.bat java status
etlworks-agent.bat logs --lines 200
etlworks-agent.bat logs --followWhat each answer means — these are different questions:
- status — native installation and running state of the service (installed=, running=, release=). A stopped service returns a nonzero exit code — expected right after installation.
- health — an instantaneous installed/running check of the native service. It does not prove flows execute or that the application is reachable.
- version — the CLI/build identity on disk. Cross-check the actually running Agent's version in the Etlworks UI.
- java status — the Agent/updater JVM selections and the state of any JVM maintenance transaction.
- doctor — installation diagnostics: home, platform, architecture, and running-JVM details. It reports the launcher-resolved home.
- logs — prints the last 100 lines of the Agent log (logs/IntegrationAgent.log) by default; --lines <number> changes the count, --follow tails live (Ctrl+C stops), and --file <path> selects another log, for example --file UpdateService.log for the updater log.
There is no generic monitor command — combine status, logs --follow, and the Agents page in the UI.
Exit codes for automation:
| Code | Meaning |
|---|---|
| 0 | Successful command (asynchronous JVM maintenance may only have been queued) |
| 1 | Command failure |
| 2 | Unknown command |
| 3 | status reports the Agent/updater not installed or not running |
| 4 | health reports unhealthy |
Status and health checks do not prove that flows or online licensing are working.
Configure the Integration Agent
Configure the Agent from the app: in the Agents window select the Agent, open the Parameters tab, modify parameters, and click Save. A running Agent picks up most configuration at runtime on its next polling cycle; a stopped Agent applies changes on the next start. JVM-level settings (memory, Java options) always require a restart — not every parameter changes dynamically.
Local configuration lives in config/integration-agent.properties; app-managed configuration takes precedence for the parameters managed through the UI.
Configuration parameters
Configures the SMTP sender used for Agent email notifications: SMTP Host, SMTP Port (default 587), SMTP User, SMTP Password, From, To (comma-separated recipients), CC, BCC, TLS Enabled (default on).
Notifications
Enable email notifications for specific events (the SMTP sender must be configured). All are disabled by default: On Success (each successful flow execution), On Fail (each failed flow execution), On Shutdown, On License Issue (license verification problems), On Status Change (any status change, including reaching and clearing warning/critical thresholds).
HTTP Proxy
Routes the Agent's HTTP traffic through a proxy: Proxy Host, Proxy Port, Proxy User, Proxy Password, Proxy Auth Scheme (http, https, or ntlm), and Proxy Exceptions (comma-separated hosts that bypass the proxy).
Web Server
The optional embedded web server displays the local Agent dashboard: Web Server Enabled, Host (default localhost), Port (default 8081). When managed from the app, the dashboard is disabled by default. The dashboard is a local convenience — CLI lifecycle operations and remote updater actions do not require it.
Monitoring
Warning and critical thresholds for the Agent health status: Memory Utilization High Threshold (%) (default 80), Memory Utilization Critical Threshold (%) (default 90), Error Ratio High Threshold (%) (default 10), Error Ratio Critical Threshold (%) (default 25).
Logging
- Upload console log files — uploads each flow execution log to the Etlworks instance, where it is visible in the Flow Statistics dashboard. Enabled by default.
- Maximum log file size to upload (MB) — default 10. Larger log files are not uploaded; network proxies can impose their own upload limits.
dbt
Physical locations for dbt and Git flows running on this Agent: dbt Executable, dbt Project Root, dbt Package Cache Root, Git Executable, Git Work Root. See Run dbt and Git flows on the Integration Agent.
Miscellaneous
- Shutdown time out (ms) — how long the Agent waits for running flows to finish before shutting down; default 300000 (5 minutes). The Agent first asks running flows to stop; most stop quickly, busy flows can take minutes. Your service manager's or container's own stop timeout is a separate, outer limit — set it above this value (the Docker template uses a 6-minute grace period).
- Agent ping delay (seconds) — delay between the Agent's calls to the Etlworks instance for configuration and status reporting; default 60. Lower values make the Agent more responsive but increase chatter.
Reupload flow stats and logs
Agents automatically re-upload statistics and logs for flows executed during host-instance downtime (webhooks fire too, if configured). To prevent many Agents from flooding the instance at once: Failed audit upload check delay (minutes) — delay between re-send attempts, default 60 (the first check after Agent start is always 3 minutes); Failed audit re-upload throttling (seconds) — delay between consecutive uploads, default 5.
Memory configuration
The default maximum Agent heap is 70% of system RAM. Three supported ways to change it:
- From the UI (requires the updater running): open the Agent's Status dashboard, navigate to Memory Max, click Configure, enter the maximum memory in megabytes or auto (70%), and click Update. The updater reconfigures and restarts the Agent; verify the new value in the Status screen afterward.
- At installation: install --max-memory <MB> (validated; megabytes).
- In Docker: the container's memory limit also bounds the effective heap — size the container and the Agent memory setting together.
Modern installations have no install.cmd/install.sh to edit — those files belong to legacy installations. Do not add unsupported JAVA_OPTS or Docker environment variables; memory changes take effect after a restart.
Required Java 25 option for Snowflake and Databricks
On Java 25 Agents, the Snowflake and Databricks Arrow result readers require the JVM option --add-opens=java.base/java.nio=ALL-UNNAMED. Current launchers, Windows service installation, and current Docker images supply it automatically; Java 8 builds omit it. An ordinary update ZIP does not refresh an older CLI/service configuration — an existing Java 25 installation created before this option needs its CLI/service configuration updated (or, in Docker, a current image) and a full Agent restart. Symptom when missing: Snowflake/Databricks reads fail during Arrow initialization with java.nio.Buffer.address access errors in the Agent log.
Update an Agent: automated (updater service)
There are two workflows for updating a 10.x Agent, and they differ in who executes the update: automated — the updater service performs the update, triggered from the Etlworks UI (this section); and manual — you download the update ZIP and run the CLI on the host yourself (next section).
Agent updates are performed by the updater service on the Agent host. The Update button is available on saved Agent detail pages and Agent cards: it is highlighted blue with an Update available annotation when a newer compatible published release exists, and neutral otherwise. The annotation distinguishes Update available, Up to date, and Newer than published; unknown release information is never shown as up to date.
Two update sources
The update dialog (single and bulk) offers two explicit source buttons:
- Download and install — select a published compatible version. Single-Agent updates can pick an explicit version; bulk automatic updates require a newer target. The updater downloads the release.
- Install local ZIP — you first download or copy exactly one compatible Agent update ZIP into the Agent home's update folder on each selected Agent's host. Nothing is downloaded by the app or the updater in this mode, and it works for same-version reinstalls. Each selected Agent needs its own staged ZIP, and the Agent's updater must be current enough to support local updates (older updaters are rejected for this mode).
Both modes require a running, non-busy updater — a stopped Agent can be updated as long as its updater is available. Installation is one queued action: there is no separate "check ZIP" step; the ZIP is validated on the Agent before the Agent is stopped, and a missing or invalid ZIP fails with the actual reason without stopping the Agent.
Update one or multiple Agents
Step 1. Select one or more Agents and click the Bulk Update button in the toolbar (or use Update on a single Agent). This opens the Bulk Agent Update dialog.
Step 2. Choose the update source and the version. The dialog shows the installed version, the release timestamp, and a link to the Release Notes.
Step 3. Review the validation results. Agents may be marked Eligible, No update required (already running the selected version), or ineligible with a reason (for example, Agent is offline with no running updater, an incompatible release, or a busy updater). Only eligible Agents will be updated.
Step 4 (optional). Click Download CSV for the eligibility report (Agent name, current version, target version, eligibility status).
Step 5. Click Update Eligible Agents. Each Agent's updater performs the update and restarts the Agent.
Step 6. Verify: the Agent status returns to Online and the new version appears on the Agent card — and, for an important release, run a representative flow.
Update requirements and compatibility
- The updater service must be installed and running on the Agent host.
- Modern direct updates require compatible identities: the application and the Agent's installed engine must both be at least 10.0.0. The app checks the actual release and engine identity, installation format, and Java target — a version label or Java target alone does not prove eligibility, and unknown or incompatible identities are rejected. Do not try to bypass these checks.
- Legacy Agents keep their legacy downloaded-update path (Agent 6.3.4-SNAPSHOT or newer with the bundled updater); they cannot receive local-ZIP updates, and they cannot be moved to 10.x by an ordinary update — that is the migration.
- A queued action is not a completed action — updates execute on the updater's next poll. Completion is reported by the updater after the CLI finishes; an unchanged-version heartbeat alone does not mark a reinstall successful.
Agent update reports
All update requests are stored. Open them with the Agent update request reports button in the Agents toolbar: view current and past requests, expand per-Agent results and failure details, and download reports as CSV. The number of stored requests and their retention (TTL in days) are configured under Settings → Agents.
Agent status indicators
Card and table views show the Agent Status (Online / Offline) and the Updater Status (Running / Stopped) including the updater's current activity. Two different things can look similar: updater liveness (is the updater polling?) and the last operation's result (did the last update fail?) are displayed separately — a failed operation shows a compact warning icon next to the Running/Stopped pill, and hovering or focusing it shows the actual reason reported by the updater. Older updaters that report only a generic failure show a hint to check the updater log instead; a missing-ZIP cause is never inferred from a generic failure.
Troubleshooting Agent updates
- Agent Offline, Updater Running — start the Agent from the UI and retry. If it cannot be started from the UI, start it on the host (sudo ./etlworks-agent start) and retry.
- Agent Offline, Updater Stopped — start the updater on the host (sudo ./etlworks-agent updater start), then the Agent, then retry.
- Agent Online, Updater Running, but the update fails — read the failure reason on the updater-status warning icon and in the update request report; check the updater log (logs/UpdateService.log). For local-ZIP updates, confirm exactly one complete ZIP is present in update and that it matches the Agent's Java target. If the problem persists, contact support with the updater log.
Update an Agent: manual (downloaded update ZIP)
In the manual workflow you obtain the update archive yourself and the CLI installs it — the CLI never downloads one. Download the complete, target-matching update ZIP for the release you want, then place it (for example integration-agent-<version>-java25-update.zip) in AGENT_HOME/update. Copy or download it with a temporary non-ZIP name (for example update.zip.part) and rename it to .zip only when the transfer is complete.
cd /opt/etlworks-agent
./etlworks-agent update --dry-run
sudo ./etlworks-agent update --skip-confirm
sudo ./etlworks-agent update-do-not-start --skip-confirmWindows, from an elevated terminal:
cd C:\etlworks\integration-agent
etlworks-agent.bat update --dry-run
etlworks-agent.bat update --skip-confirm
etlworks-agent.bat update-do-not-start --skip-confirm- Exactly one ZIP must be in the inbox by default; with several, select one with --archive <filename>. Files outside the inbox, directories, and symlinks are rejected.
- --dry-run validates the archive, release identity, and Java target without changing the service or consuming the archive. The Java target and installation metadata must match the installation — a Java 25 Agent requires the Java 25 update ZIP.
- The update stops the Agent, takes a verified transactional backup (configuration, license, data, plugins, custom JDBC, active libraries and drivers), replaces the release, refreshes service references, and starts the Agent. Verification requires the native service to remain running for five continuous seconds — no HTTP probe, and not proof that flows work. --health-timeout <milliseconds> bounds that verification; default 60000.
- An ordinary update starts the Agent even if it was stopped beforehand. update-do-not-start leaves it stopped and explicitly reports runtime health as unverified — start it later yourself.
- On success the consumed ZIP is deleted by default; --retain-archive moves it to update/applied/ instead. The temporary update backup is deleted after verification. On failure, the pending ZIP and recovery material are retained, and automatic rollback restores the runtime files and the original running state where possible — it does not blindly restore mutable data over newer state.
- There is no retained release history and no generic rollback-to-previous command: to downgrade within 10.x, explicitly install an older compatible 10.x update ZIP (downgrading to 9.x is not available), and consider what your flows may already have written to external systems in between.
The updater service
The updater (EtlworksUpdateService on Windows) executes remote actions queued from the Etlworks UI — start, stop, restart, update, reconfigure — by polling the instance; it requires no inbound connectivity. Modern full installers bundle it; installation registers it with automatic boot start, and you start it explicitly once (updater start).
Updater subcommands (updater without a subcommand defaults to status):
<launcher> updater install
<launcher> updater start
<launcher> updater stop
<launcher> updater restart
<launcher> updater status
<launcher> updater uninstall
<launcher> updater run- updater install registers the service without starting it; it supports --user, --release, and --java-home (bootstrap can use --java-archive). updater install --start is rejected — start explicitly.
- updater run runs the updater in the foreground; updater uninstall removes its service and retains data. There is no updater update command.
To refresh the updater independently of the Agent, extract the standalone updater archive (integration-agent-updater-<version>-java<N>.zip) over the Agent home while the services are stopped and run sudo ./etlworks-agent updater install (Windows: etlworks-agent.bat updater install). Ordinary Agent update ZIPs do not replace the updater, and the updater does not update itself.
Migrate a legacy 9.x Agent to 10.x
Legacy Agents (the flat Java 8 layout installed by 9.x installers) move to the modern managed layout through a staged, operator-run migration — deliberately not an automatic update. Not every 9.x variation is automatically eligible: the migration validates the actual installed layout, services, and updater, and fails closed before downtime when it finds unsupported customizations.
The migration has two stages:
Stage 1 — install the bridge update. The bridge is delivered and installed like an ordinary legacy Agent update — automatically through the updater service, or manually from a download. Installing it places the new 10.x migration tooling on the machine (the migration launcher and the new CLI, with the staged 10.x payload), but the Agent itself stays on its current pre-10.x version and keeps running; the UI continues to report the old version.
Stage 2 — initiate the migration from the CLI on the Agent machine. Stage 1 installed the new root CLI in the Agent home. An administrator logs in to the machine running the Agent, opens an elevated terminal in the Agent home, and runs etlworks-agent.bat migrate-legacy (Windows) or sudo sh ./etlworks-agent migrate-legacy (Linux). Only this step migrates the Agent to 10.x.
Stage 1: install the bridge update
Automated delivery (updater service)
Requires the legacy Agent's updater to be installed and running. In the Agents window select the legacy Agent — or several — and start an update (Update, or Bulk Update for many), choosing the published 10.x migration (bridge) release offered for legacy Agents. The updater downloads the bridge through its existing update mechanism, stages its files into the Agent home, and restarts the old Agent.
Manual delivery (download)
Use this when the updater path is unavailable. Download the bridge ZIP, extract it into a separate temporary directory, and copy the five flat files from its update/ directory — etlworks-upgrade.ps1, etlworks-upgrade.sh, etlworks-upgrade.target, the bundle ZIP, and its .sha256 — directly into the Agent home. Do not unpack the inner bundle yourself.
Bridge delivery does not migrate anything — whichever way it was delivered, the Agent still runs its old release and reports its old version. Whichever way the bridge was installed, migration starts only in stage 2 — per host, from the CLI.
Stage 2: initiate the migration from the CLI
Log in to the machine running the Agent. Stage 1 installed the new root CLI (etlworks-agent.bat / etlworks-agent) directly in the Agent home. Open a Command Prompt as Administrator (Windows) or a terminal (Linux), cd to the Agent home, and run migrate-legacy — optionally with --check first for a non-disruptive preflight. On Linux, invoke it through sh the first time: the legacy ZIP extractor does not preserve the executable bit.
Preflight (optional but recommended; non-disruptive):
cd C:\etlworks\integration-agent
etlworks-agent.bat migrate-legacy --checkcd /opt/etlworks/agent
sudo sh ./etlworks-agent migrate-legacy --checkApply (repeats every preflight check, then migrates):
etlworks-agent.bat migrate-legacysudo sh ./etlworks-agent migrate-legacyMigration options — --check, --apply, and --recover are mutually exclusive:
- No mode option — preflight followed by migration (the default shown above). --apply is its explicit equivalent.
- --check — preflight only.
- --recover — recover an interrupted migration using its retained bridge/staging/recovery files.
- --java-archive <path> — offline bootstrap JVM.
- --health-timeout <milliseconds> — native service verification timeout; default 120000.
- --stage-only — outer-launcher diagnostic that verifies and extracts the bridge without Java bootstrap or service changes.
Migration does not ask for interactive confirmation, and the strict migration parser rejects extra options such as --skip-confirm and --verbose. After a successful migration the root launcher is executable, so subsequent Linux commands run directly: sudo ./etlworks-agent status. The older etlworks-upgrade.ps1 / etlworks-upgrade.sh entry points delivered by the bridge remain as compatibility launchers; standard procedure uses migrate-legacy.
Recovery after an interrupted or failed migration (repeatable):
etlworks-agent.bat migrate-legacy --recoversudo sh ./etlworks-agent migrate-legacy --recoverImportant — there is no way back to 9.x. Once an Agent runs 10.x — whether installed fresh or migrated from 9.x — downgrading to a 9.x Agent is no longer available. The migration's recovery path exists only for a failed or interrupted migration; after a successful migration and its cleanup, returning to 9.x is not supported.
What to know before migrating:
- Finish active flows and schedule downtime. Do not queue remote update/start actions, and do not redeliver a bridge, while migration or recovery is active.
- Linux needs unzip and sha256sum/shasum (minimal images may need unzip installed first). Windows script execution policy applies to the launcher; use your approved unblocking procedure.
- --check verifies the layout, payload checksums, service identity/configuration, disk space, and backup readability; it can download and stage the private pinned Java under the staging directory and write protected workspace material, but it does not stop or reconfigure services. --stage-only only verifies and extracts the bridge. --java-archive <file> supports offline Java staging. The system package manager is never used for Java.
- Supported starting points are strict: the standard legacy Windows services (IntegrationAgent/EtlworksUpdateService under the selected home) or the standard legacy Linux service definitions. Custom accounts, JVM options, or layouts fail the preflight before downtime. Legacy macOS service migration and in-container migration are not supported.
- The migration stops the updater, then the Agent; backs up the legacy installation with verified checksums; removes the old services; installs the new layout (new CLI, lib/jdbc, updater, private JVM, state); and starts both services with the modern boot policy (updater automatic, Agent manual). Your configuration, licenses, state, data, and extra connector folders are preserved — bundled connector folders are replaced by the new release's bundles, with your overlays retained through jdbc-custom and extra folders kept in place. Old launchers and the private JRE are retired; nothing removes system-wide Java.
- After success, the temporary migration backup is cleaned up. Take an independent backup before migrating — after cleanup there is no automatic way back. A cleanup failure does not roll back the running services; rerun ./etlworks-agent migration-cleanup from the installed CLI.
- Failed or interrupted migrations retain their backup and staging; --recover validates the retained material and returns the old services to their original state. Recovery restores what migration changed — it does not rewind database writes or CDC progress. Do not run flows during the migration/rollback window.
- Native service success does not prove license validation, connectivity, or flow correctness — verify the Agent in the app and run a representative flow before resuming production schedules. Java 25 scripting differs from Java 8 (see compatibility); review Python-based flows in particular.
Private JVM maintenance
The Agent runs on a private, pinned Azul Java runtime under jvm/versions. Applying a JVM security patch is an explicit administrator operation within the installed Java major — it is not an unattended "download latest" feature, it cannot convert Java 8 to Java 25 (that requires a new installation or migration), and it does not touch the host's system Java or JAVA_HOME. Docker Agents are patched by image replacement instead; native JVM patching is rejected in Docker.
sudo ./etlworks-agent java update --manifest /path/to/approved-java25.properties --component both
./etlworks-agent java status
sudo ./etlworks-agent java rollback --component both
sudo ./etlworks-agent java recoverWindows, from an elevated terminal:
cd C:\etlworks\integration-agent
etlworks-agent.bat java update --manifest C:\releases\approved-java25.properties --component both
etlworks-agent.bat java status
etlworks-agent.bat java rollback --component both
etlworks-agent.bat java recover- java without a subcommand defaults to status. --component selects agent, updater, or both (default); the selected component's service must be installed. --health-timeout-ms <milliseconds> (default 120000) bounds service verification for java update and java rollback.
- The manifest is an approved pin supplied for the release: a local manifest file is trusted administrator input; an HTTPS manifest additionally requires its out-of-band --manifest-sha256 (redirects and plain HTTP are rejected). Omitting the manifest uses the already installed pin — it does not discover a newer release. --java-archive <file> supports offline staging with the same checksum checks (--archive is only a bootstrap compatibility alias — prefer --java-archive).
- The candidate runtime is downloaded/verified, extracted, and version-probed before any downtime; a staging failure changes nothing. Downgrades are rejected by update — use explicit rollback to return to the previous patch after a completed update.
- The switch runs through an independent privileged worker (not inside the updater). It stops the selected services, changes only the JVM selections, and restarts only the services that were running beforehand — accounts, JVM options, service policies, and boot settings are preserved. A restarted service must stay running for ten seconds.
- Submission means queued, not successful. Watch java status until the result is SUCCEEDED, ROLLED_BACK (the patch failed and the previous configuration was restored automatically), FAILED_BEFORE_SWITCH, or RECOVERY_REQUIRED. Diagnostics are in the displayed job directory (journal.properties, worker.log, stage/probe logs).
- An interrupted worker requires explicit java recover; there is no automatic reboot-triggered repair. While a transaction is pending, ordinary lifecycle changes are blocked — and do not delete the retained old JVM, journal, or helper files.
java install is an advanced bootstrap operation used before installation state exists; it is rejected for an installed Agent — use java update there.
Damaged selected runtime: if the active JVM is broken and the launcher itself cannot run, invoke the CLI directly with a retained working JVM of the same major:
<old-java>/bin/java -jar <agent-home>/bin/etlworks-agent-cli.jar java recover --home <agent-home>Run the Integration Agent in Docker
The official image runs the Agent and its bundled Java runtime as an ordinary Docker container. The Docker deployment model:
- Installation is ordinary Docker usage: docker pull, docker run, and container replacement. You do not download a native Agent installer, install Java on the host, obtain a host Agent CLI, or install Compose.
- Windows and macOS hosts use Docker Desktop with Linux containers.
- Persistent files live in host folders mounted into the container.
- No native Agent updater service runs inside the container — remote update actions do not apply. The Agent and its Java runtime are updated by replacing the image.
- Native installation, native update ZIPs, and the legacy bridge migration are not part of Docker installation or upgrades.
Choosing an image. Use a published version-specific tag — the pattern is etlworks/integration-agent:<version>-java25 — or an immutable digest. The examples below use <published-tag>; replace it with the actual released tag. Do not assume latest points to 10.x. Images are published for linux/amd64 and linux/arm64; choose an image available for your host architecture.
Create persistent host folders
The examples use /opt/etlworks/integration-agent (Linux/macOS) and C:\etlworks\integration-agent (Windows) — substitute another location if you prefer, consistently in every command.
Linux/macOS (creating directories under /opt may require elevated permissions):
mkdir -p /opt/etlworks/integration-agent/{config,license,data,plugin,jdbc,jdbc-custom,logs,backups}Windows Command Prompt:
mkdir C:\etlworks\integration-agent\config
mkdir C:\etlworks\integration-agent\license
mkdir C:\etlworks\integration-agent\data
mkdir C:\etlworks\integration-agent\plugin
mkdir C:\etlworks\integration-agent\jdbc
mkdir C:\etlworks\integration-agent\jdbc-custom
mkdir C:\etlworks\integration-agent\logs
mkdir C:\etlworks\integration-agent\backups| Host folder | Purpose |
|---|---|
| config | Agent configuration |
| license | Downloaded Agent license and applicable external licensing files |
| data | Staged data and other Agent working data |
| plugin | Additional or premium connector drivers |
| jdbc | Persistent JDBC driver folders managed during startup |
| jdbc-custom | Customer JDBC additions and overrides |
| logs | Agent log files |
| backups | Backup files stored by Agent operations |
Mount the individual folders, not the whole Agent home. Bind-mounting one host folder over the entire /opt/etlworks-agent would hide the runtime installed in the image. The entrypoint prepares ownership of the mounted folders before starting the Agent as its dedicated user; your host filesystem permissions (and Docker Desktop file-sharing settings) must permit access to them — fix the actual permission, rather than resorting to blanket chmod 777.
Install the Agent license
Create the Agent in the application and download its license as described in Install the license and start the Agent. Copy the downloaded .alic file into the host's license directory. Do not rename the license file.
The application URL recorded in the license must be reachable from inside the container:
- localhost inside the container refers to the container itself, not the host.
- A normal application hostname reachable through DNS or a reverse proxy needs no special Docker host mapping.
- When the application runs on the Docker host itself, use a hostname the container can actually reach.
- Only if a custom hostname specifically needs to resolve to the Docker host, add a mapping matching that hostname, for example: --add-host application.example.test:host-gateway. This mapping is conditional, not a standard part of the setup.
Pull and run the container
docker pull etlworks/integration-agent:<published-tag>Linux/macOS:
docker run -d --name integrationagent \
--init \
--restart unless-stopped \
--stop-timeout 360 \
-v /opt/etlworks/integration-agent/config:/opt/etlworks-agent/config \
-v /opt/etlworks/integration-agent/license:/opt/etlworks-agent/license \
-v /opt/etlworks/integration-agent/data:/opt/etlworks-agent/data \
-v /opt/etlworks/integration-agent/plugin:/opt/etlworks-agent/plugin \
-v /opt/etlworks/integration-agent/jdbc:/opt/etlworks-agent/jdbc \
-v /opt/etlworks/integration-agent/jdbc-custom:/opt/etlworks-agent/jdbc-custom \
-v /opt/etlworks/integration-agent/logs:/opt/etlworks-agent/logs \
-v /opt/etlworks/integration-agent/backups:/opt/etlworks-agent/backups \
-e JAVA_OPTS="-Xms100m -Xmx4000m" \
etlworks/integration-agent:<published-tag>Windows Command Prompt:
docker run -d --name integrationagent ^
--init ^
--restart unless-stopped ^
--stop-timeout 360 ^
-v "C:\etlworks\integration-agent\config:/opt/etlworks-agent/config" ^
-v "C:\etlworks\integration-agent\license:/opt/etlworks-agent/license" ^
-v "C:\etlworks\integration-agent\data:/opt/etlworks-agent/data" ^
-v "C:\etlworks\integration-agent\plugin:/opt/etlworks-agent/plugin" ^
-v "C:\etlworks\integration-agent\jdbc:/opt/etlworks-agent/jdbc" ^
-v "C:\etlworks\integration-agent\jdbc-custom:/opt/etlworks-agent/jdbc-custom" ^
-v "C:\etlworks\integration-agent\logs:/opt/etlworks-agent/logs" ^
-v "C:\etlworks\integration-agent\backups:/opt/etlworks-agent/backups" ^
-e JAVA_OPTS="-Xms100m -Xmx4000m" ^
etlworks/integration-agent:<published-tag>- Replace <published-tag> before running, and replace the host paths consistently with the folders you created.
- --restart unless-stopped controls automatic restart (including after host reboots).
- --stop-timeout 360 allows six minutes for graceful shutdown, so the Agent's own shutdown timeout can complete.
- The optional web dashboard does not need to be published for ordinary outbound communication with the application — see below if you want it.
Configuration
An empty mounted config folder is initialized automatically from the image defaults at startup: missing default configuration files are copied in, and existing files are preserved — including when you recreate the container with a newer image. You never need to extract configuration manually or download another package.
The primary host configuration file is <host-agent-home>/config/integration-agent.properties. To change configuration:
docker stop --time 360 integrationagentEdit the host file, then:
docker start integrationagentNote: existing configuration files are not automatically rewritten to include every setting introduced by future releases — consult release-specific configuration instructions when applicable.
Memory settings and the optional dashboard
JAVA_OPTS sets the Agent JVM options:
-e JAVA_OPTS="-Xms100m -Xmx4000m"- This example requests a 100 MB initial heap and a 4000 MB maximum heap — example settings, not the default. Without JAVA_OPTS, the runtime uses -XX:MaxRAMPercentage=70.0.
- Heap size is not the container's total memory usage. Docker memory limits are separate — leave room for native memory and other JVM overhead.
- Your options apply to the Agent JVM; the required Java 25 module-access arguments remain present automatically.
- Changing an environment variable requires recreating the container with the new setting — restarting an existing container retains its original environment.
Optional dashboard. The image configures the dashboard listener for container access; the relevant properties are:
web.server.enabled=true
web.server.host=0.0.0.0
web.server.port=8081To publish it only on the Docker host's loopback interface, add to docker run:
-p 127.0.0.1:8081:8081Then open http://localhost:8081 on that host. Publishing the port alone does not enable a dashboard that is disabled in your existing configuration, and changing published ports requires container recreation.
Premium connectors and JDBC drivers
Premium connectors and their licenses may be required separately. To install one:
Step 1. Copy the driver into the host's plugin folder.
Step 2. Open a container shell and activate the license:
docker exec -it integrationagent bash
cd /opt/etlworks-agent/plugin
java -jar name.of.the.driver.jar -licenseReplace the filename with the actual driver filename and follow the vendor's activation instructions. Vendors differ in where they store activation files — retain any vendor-specific files according to that vendor's instructions so they survive container replacement.
Step 3. Restart after installing or changing driver files:
docker restart --time 360 integrationagentJDBC customization: startup refreshes the image-managed bundled driver folders in jdbc; additional driver folders you add there are retained. To override a bundled driver, use jdbc-custom — edits made directly to bundled files in jdbc are not promised to survive an update.
Routine management and monitoring
| Task | Command |
|---|---|
| List containers, including stopped ones | docker ps -a |
| Start | docker start integrationagent |
| Stop gracefully | docker stop --time 360 integrationagent |
| Restart gracefully | docker restart --time 360 integrationagent |
| Show recent console logs | docker logs --tail 100 integrationagent |
| Follow console logs | docker logs --tail 100 --follow integrationagent |
| View resource usage | docker stats integrationagent |
| Open a shell | docker exec -it integrationagent bash |
Agent log files also persist in the host logs folder. Docker Desktop users can perform the corresponding lifecycle and log-viewing operations in its UI.
The image includes a health check — but container health alone does not prove successful Agent registration or successful flows. Confirm the Agent is Online in the application and test a representative flow.
Optional diagnostics inside a running container (these use the CLI already inside the image; no host CLI download is needed — and do not run host-oriented etlworks-agent docker ... commands inside the container):
docker exec integrationagent /opt/etlworks-agent/etlworks-agent version
docker exec integrationagent /opt/etlworks-agent/etlworks-agent health --containerBack up and update to a new 10.x image
Updating replaces the container with a new image while the persistent host folders carry your state. The procedure:
- Record the existing settings: image reference or image ID, mount paths, environment settings, published ports, and any other Docker settings you applied.
-
Pull the replacement image before stopping the running Agent:
docker pull etlworks/integration-agent:<new-published-tag> -
Coordinate downtime and stop gracefully:
docker stop --time 360 integrationagent - Back up the persistent host folders while the Agent is stopped (copy or archive the host Agent home).
-
Remove the stopped container:
docker rm integrationagent - Run the replacement image: rerun the complete creation command with the new image tag, the same host folders, and all your recorded settings.
- Verify: check the logs, container health, the Agent's Online status in the application, and a representative flow.
Make no mistakes about what Docker does and does not do here:
- Pulling an image does not update an existing container, and restarting an existing container does not switch its image.
- Removing a container does not delete the host bind-mounted folders.
- Reuse exactly the same host paths — an incorrect path can appear to lose the license or data.
- Preserve all custom environment, networking, resource, and port settings when recreating.
- Retain the previous image and your backup until the replacement is accepted.
- Direct Docker updates do not create verified backups or perform transactional rollback for you — the backup step above is yours.
Rollback is the same operation in reverse: stop and remove the replacement, recreate with the previous image and compatible persistent state, restoring the pre-update backup where needed for compatibility. An image rollback does not reverse changes flows already made in external systems.
Offline hosts: transfer the image with docker save / docker load; no Agent CLI is required for offline image replacement.
Upgrade a 9.x Docker Agent to 10.x
The important change for existing Docker Agents is the container home path:
Legacy container home: /agent
10.x container home: /opt/etlworks-agentYour existing host directories can stay where they are — update the container-side destination in each mount. Example:
Old: -v /path/to/agent/home/data:/agent/data
New: -v /path/to/agent/home/data:/opt/etlworks-agent/dataProcedure:
- Record the old container's settings and mounts.
- Pull the 10.x image.
- Stop the old Agent and back up the persistent files.
- If custom configuration, drivers, or licensing files exist only inside the old container, copy them out before removing it.
- Retain the existing host license, data, logs, and plugin folders; create and mount the additional 10.x folders shown in the new installation commands.
- For a new empty config folder, let 10.x initialize its defaults first, then reapply your necessary custom settings — and review any old absolute /agent/... paths before carrying configuration forward.
- Recreate the container using the 10.x image and the /opt/etlworks-agent/... mount destinations.
- Verify registration, drivers, premium licenses, and representative flows.
Do not run migrate-legacy, etlworks-upgrade.ps1, or a bridge update for a Docker Agent — those are native-installation procedures. And review custom drivers and configuration for Java 25 compatibility rather than assuming everything old carries over.
Remove the Docker Agent
docker stop --time 360 integrationagent
docker rm integrationagentThe host bind-mounted folders remain. Deleting them is a separate, deliberate action — and deleting the Agent definition in the application is separate again.
Troubleshooting
| Symptom | Check |
|---|---|
| Missing Agent license | The .alic file is present with its original filename, in the correctly mounted host license folder. |
| Connection refused or hostname errors | The license's application URL is reachable from the container's network context — check DNS, routing, proxy, and TLS configuration. --add-host is a fix only when that specific hostname must resolve to the Docker host. |
| Permission errors | Host folder access, Docker Desktop file-sharing settings, and filesystem restrictions on the mounted folders. |
| Dashboard unavailable | Dashboard enabled in configuration, listener address/port, and the published port mapping. |
| Memory failures | Heap settings (JAVA_OPTS), Docker memory limits, and native-memory headroom. |
| Startup failure after replacement | Container logs, mount destinations (especially /opt/etlworks-agent/... vs legacy /agent/...), configuration, and driver compatibility. |
Back up and restore Agent state
cd /opt/etlworks-agent
sudo ./etlworks-agent backup --archive /backups/agent-state.zip
./etlworks-agent restore --archive /backups/agent-state.zip --dry-run
sudo ./etlworks-agent restore --archive /backups/agent-state.zip --stop-service --skip-confirm- Default scope: configuration, license, data, existing plugins, and custom JDBC drivers (jdbc-custom). Logs are opt-in (--include-logs). The runtime payload, bundled JDBC, JVMs, downloads, and staging are excluded — a default backup is not a complete reinstall image; reinstall the matching release first, then restore state into it.
- By default, backup stops and restarts a running Agent for a consistent snapshot. --online backs up without stopping — faster, but files being written during the backup may be captured mid-change. --fail-on-locked-files fails instead of skipping unreadable files. --archive <path> names the destination (--output is an alias); without it, a timestamped ZIP is created under AGENT_HOME/backups. --skip-confirm permits overwriting an existing archive without prompting.
- Backups are ZIP archives with a manifest and checksums; restore validates all paths and checksums before changing anything (--dry-run previews), preserves the pre-restore state until completion, requires the Agent to be stopped — without --stop-service, restore refuses while the Agent is running — and does not start the Agent afterward; verify and start explicitly. The archive path can also be given positionally.
- Restore returns Agent state; runtime/target compatibility and any external-system side effects between backup and restore remain your responsibility.
Keep these explicit backups separate from the temporary transaction backups created by updates and migration — those are deleted automatically after success.
Installing connectors
Etlworks comes with hundreds of built-in connectors. Additional regular or premium connectors can be installed on the machine running the Agent.
Where connector files go on a modern Agent:
- New connectors: create a folder with any name (for example ext) under AGENT_HOME/jdbc and copy the files there. Folders absent from the bundled release are preserved across updates.
- Overriding a bundled driver: place your replacement files in AGENT_HOME/jdbc-custom using the same folder layout. Updates replace each bundled jdbc folder as a complete bundle, then re-apply your jdbc-custom overlay — a modified file placed directly inside a bundled folder does not survive an update.
- Keep vendor license files (.lic) next to their connector JARs; they follow the same preservation rules.
- Restart the Agent after installing connector files. In a multi-Agent deployment, install the connector and license on each Agent. In Docker, place additional/premium connector drivers in the mounted plugin folder and JDBC overrides in jdbc-custom — see premium connectors in Docker.
Installing Regular Connectors
Step 1. Obtain the connector: contact Etlworks support. It is provided as a .jar file and may include a license file or other required files.
Step 2. Create a folder (for example ext) under AGENT_HOME/jdbc — multiple connectors can share one folder — and copy all provided files into it. To override a bundled driver, use jdbc-custom instead, as described above.
Step 3. Restart the Integration Agent.
Installing Premium Connectors
Step 1. Obtain the connector: contact Etlworks support. It is provided as a .jar file along with a license key (or use a trial license).
Step 2. Copy the .jar into a folder under AGENT_HOME/jdbc (for example ext).
Step 3. Open a terminal and cd to that folder.
Step 4. Run the license activation command using the Agent's private Java runtime (any installed Java 8+ also works):
../../jvm/versions/<installed-version>/bin/java -jar connector.jar -license(Windows: ..\..\jvm\versions\<installed-version>\bin\java.exe -jar connector.jar -license. Check the installed version with ./etlworks-agent java status. The old ../../jre/bin/java path applies only to legacy installations.)
Step 5. Provide the required information when prompted: Name: support; Email: support@etlworks.com; License key: the key provided by Etlworks, or TRIAL for a trial license.
Step 6. Verify the license file was generated: its name matches the connector file with .jar replaced by .lic (for example cdata.jdbc.salesforce.jar → cdata.jdbc.salesforce.lic).
Step 7. Restart the Integration Agent.
Handle cases where the license file is not generated
Occasionally the activation fails to generate the license file (including trial licenses). The activation output then provides a download link:
Step 1. Open the link in any browser and follow the instructions to download the license file.
Step 2. Rename it to match the connector file name, replacing .jar with .lic.
Step 3. Copy the .lic file to the same folder as the connector.
Step 4. Restart the Integration Agent.
Remove an Agent
Four different removals — pick the right one:
- sudo etlworks-agent uninstall — removes the Agent service and runtime registration; application data is retained.
- sudo etlworks-agent updater uninstall — removes the updater service only.
- Docker removal — docker stop + docker rm; the host bind-mounted folders remain until you delete them deliberately.
- Deleting the Agent definition in the application (Agents window → Delete agent) — removes the registration and configuration in Etlworks; it does not uninstall anything on the host. Do both when decommissioning.
There is no Agent "--clean" wipe command; remove retained data directories manually and deliberately after confirming nothing needs them.
Monitor the Integration Agent
Each running Agent reports status to the host Etlworks instance on every ping (default once a minute; see Agent ping delay). The status of each Agent is displayed in the Agents window.
Available statuses
- Offline: the instance has no recent communication from the Agent. Usually the Agent is stopped — but a network, proxy, TLS, or registration problem produces the same indicator while the host process is still running. When Offline is unexpected, check the process on the host (./etlworks-agent status) and the Agent log before assuming a crash.
- Online: the Agent is running and communicating, and its scheduler is enabled.
- Online (suspended): the Agent is online but its scheduler is suspended — no flows run automatically; manual execution is still possible.
Keep the different signals distinct: the host process running (host-side status/health), recent application communication (Online/Offline in the UI), updater availability (Running/Stopped), resource state (warning/critical per the Monitoring thresholds), and successful flow execution (Flow Statistics) answer different questions — none implies the others.
Agent's Dashboard
The Agent's Dashboard displays statuses, the scheduler, JVM information, and running flows. To open it, click the status link in the Agents window. The dashboard shows the state recorded during the latest communication session.
Agent logs
There are two log files:
- Agent Log (logs/IntegrationAgent.log): the primary log, containing errors and warnings related to flow execution on the Agent.
- Updater Log (logs/UpdateService.log): user interactions with the updater — starting, restarting, and updating the Agent.
Download from the UI: on the Agent's Status screen click Agent log or Updater log, wait for the Download button, and click it. On the host: ./etlworks-agent logs --lines 200, logs --follow, or logs --file UpdateService.log.
Email notifications
Notifications can be sent by the Agent or by the Etlworks instance.
Configure the Agent to send email notifications
Configure the SMTP sender, then enable the notification events: On Success, On Fail, On Shutdown, On License Issue, On Status Change.
Configure the Etlworks instance to send email notifications
Step 1. Make sure the SMTP sender is configured under Configuration → Email.
Step 2. Click Configure flow notifications.
Step 3. Enable notifications and enter the email address(es) to receive them.
Suspend the Integration Agent
A suspended Agent keeps running but its scheduler executes no flows automatically; manual flow execution is still possible. Use it to debug Agent issues without stopping the process.
To suspend: select the Agent in the Agents window, enable Suspended (bottom of the screen or the card's suspend switch), and save. The status shows Online (suspended).
Note: Agent suspension is about the Agent's scheduler. It is a different mechanism from the full application's persistent flow-execution suspension (the Settings → Flows suspend used during instance maintenance); suspending one does not suspend the other.
Actions Queue
When you trigger an action — Run Flow, Stop Flow, Start/Stop/Restart Agent, update — the action is added to a queue and executed in order on the next communication session between the Agent (or its updater) and the Etlworks instance. An action can take up to a few minutes to run; queued is not completed.
View Actions
If there are actions in the queue, the View Actions button becomes green. Click it to see the queue. From there you can remove an action from the queue and check the estimated execution time of the next action.
Start, stop, and restart from the Etlworks UI
Remote start/stop/restart/update/reconfigure requires the updater to be installed and running on the Agent host. The Agent does not start automatically on boot — start it after installation and after host restarts, from the UI or the host CLI.
- Start: open the Agent's Status screen, click Start, and wait for the status to change to Online (allow up to ~120 seconds).
- Stop: Status screen → Stop; wait for Offline.
- Restart: Status screen → Restart; the status passes through Offline and back to Online. If an update ZIP is staged in the update folder, the restart installs it.
Programmatic and scheduled Agent restarts
A periodic restart can help in high-demand environments where long-running workloads accumulate memory or temporary files. Treat it as an operational mitigation, not a guaranteed cure — if an Agent needs regular restarts, investigate the underlying flows and memory sizing too.
Prerequisite: the updater must be active
Restart actions are executed by the updater; verify it is installed and running on the Agent's dashboard.
Restarting the Agent on demand via API
API Endpoint:
POST /rest/v1/agents/{agentId}/actions
Headers: Authorization: [your auth token]. Read more about the Authentication API.
Payload:
{
"action": "agent.restart"
}This queues a restart for the specified Agent. Like every queued action, it executes on the updater's next communication session — it is not immediate.
Scheduling regular Agent restarts via a scripting flow
To restart on a schedule, create a scripting flow that writes a command file with the instruction restart into the Agent's root; the updater detects it and restarts the Agent:
// Import Java classes for file handling
var File = Java.type("java.io.File");
var FileWriter = Java.type("java.io.FileWriter");
var BufferedWriter = Java.type("java.io.BufferedWriter");
// Get the data folder path
var dataFolder = new File(com.toolsverse.config.SystemConfig.instance().getDataFolderName());
// Get the parent directory (agent's root)
var agentsRoot = dataFolder.getParentFile();
// Define the file to create in agent's root with the name "command"
var commandFile = new File(agentsRoot, "command");
// Create and write to the file
var writer = new BufferedWriter(new FileWriter(commandFile));
writer.write("restart");
writer.close();
// Log the name of the created file
etlConfig.log("Command File created: " + commandFile.getAbsolutePath());Assign this flow to the Agent and schedule it at the desired cadence (for example, once a day during a quiet window — a restart interrupts nothing only when no flows are running).
Compatibility notes
- Modern application + legacy Agent: supported for existing functionality, but new runtime features do not reach the legacy Agent merely because the application has them. Plan the migration to use 10.x capabilities on the Agent.
- Update eligibility is checked against release identity, engine identity, installation format, and Java target together — the Java target or a version label alone proves nothing.
- JavaScript: existing JavaScript continues to run on Nashorn by default on modern builds; GraalJS is an explicit opt-in.
- Python: Java 8 builds use Jython 2.7; Java 25 builds use GraalPy 3. These are different Python implementations — review and test Python-based flows when moving an Agent between Java targets.
- Agent and application versions have been synchronized since Agent 5.9.0; the update workflow checks compatibility between the Agent and the application automatically.
Troubleshooting
- Agent shows Offline but the process runs — registration or connectivity: confirm the application URL in the downloaded registration is reachable from the Agent host (from inside the container for Docker), check TLS trust for proxied/private certificates, review the proxy settings, and read logs/IntegrationAgent.log for license or connection errors. If the instance URL changed, re-download the registration file.
- License errors at startup — the per-Agent .alic file is missing from license/, belongs to a different Agent definition, or references an unreachable URL. Enable the On License Issue notification to catch this early.
- Installation fails downloading Java — the host cannot reach the vendor HTTPS endpoint, CA trust is broken, or a checksum mismatch occurred (always a hard stop). Fix connectivity or use the offline --java-archive path. On Linux, Alpine/musl is rejected — use a glibc distribution.
- Linux install preflight fails — native installation requires root, a running systemd, and an existing service account; a non-root updater also requires polkit with JavaScript rules support. Install missing OS packages (curl/wget, unzip, tar, checksum tools) with your package manager.
- Remote actions never execute — the updater is stopped. Start it on the host: sudo ./etlworks-agent updater start; it starts automatically on subsequent boots.
- Update rejected as incompatible — the archive's Java target or installation format does not match (for example, a Java 8 ZIP on a Java 25 Agent, or a modern ZIP on a legacy Agent). Download the matching archive; a legacy Agent needs the migration, not an update.
- "Missing update ZIP" / "multiple ZIPs" — the update inbox must contain exactly one complete ZIP, or select one with --archive. Transfer with a temporary non-ZIP name and rename when complete.
- Windows file locks during update — the update retries locked moves briefly and otherwise rolls back safely; close Explorer windows, editors, or scanners holding Agent files open and retry. A failed update retains its ZIP and recovery material.
- Snowflake/Databricks reads fail with java.nio.Buffer.address errors — the Java 25 Arrow JVM option is missing; see the required option.
- Lifecycle commands blocked by a pending transaction — a JVM maintenance or update/migration/Docker journal is incomplete. Run the matching recover command (java recover, migration --recover) and do not delete retained journals or backups while recovery is pending.
- Backup completed with warnings — locked/unreadable files were skipped; rerun during a quiet window or with --fail-on-locked-files if you need all-or-nothing.
Legacy 9.x Agent operations
This entire section applies only to legacy 9.x Agents (the flat Java 8 layout installed by integrationagent.exe / install.sh). Modern 10.x Agents use the managed installation and the etlworks-agent CLI; none of the scripts below exist there. To move a legacy Agent to 10.x, use the migration.
Legacy: install on Windows
Step 1. In the Agents window select the Agent. Step 2. Click Download. Step 3. Select Windows OS and click Agent — the installer downloads as integrationagent.exe. Step 4. Run it. Step 5. Enter your Etlworks User ID (or email) and Etlworks instance URL. Step 6. Install the license. Step 7. Verify in the Services app that service IntegrationAgent (description "Etlworks LLC. Integration Agent") exists; if not, run install.cmd from an elevated prompt in the Agent directory.
Legacy: install on Linux
The Linux package downloads as integrationagent.tar.gz. Create a folder (for example /opt/agent), copy the archive there, then:
sudo tar -zxvf integrationagent.tar.gz
sudo ./install.shThen install the license.
Legacy: install a license and start the Agent
Step 1. Select the Agent in the Agents window. Step 2. Click Download. Step 3. Click License (the license is the same for Windows and Linux). Step 4. If the Agent is running, stop the service. Step 5. If they exist, delete AGENT_HOME/license/*.alic and AGENT_HOME/license/*.chk — do not delete integration-agent-online.lic. Step 6. Copy the new license file into AGENT_HOME/license. Step 7. Start the Agent service.
(On modern Agents the shared license files are bundled in the runtime, and this cleanup is unnecessary — see modern licensing.)
Legacy: service scripts
From an elevated prompt (Windows) or terminal (Linux) in the Agent directory:
- Agent (Windows): start.cmd, stop.cmd, install.cmd, remove.cmd, install-and-start.cmd.
- Agent (Linux): sudo ./start.sh, stop.sh, install.sh, install-and-start.sh.
- Update Service (Windows): update-service-start.cmd, update-service-stop.cmd, update-service-install.cmd, update-service-remove.cmd, update-service-install-and-start.cmd.
- Update Service (Linux): sudo ./update-service-start.sh, update-service-stop.sh, update-service-install.sh, update-service-install-and-start.sh.
Agents between 5.9.0-SNAPSHOT and 6.3.4-SNAPSHOT need the Update Service downloaded and installed separately (Download → Update service, as updateservice.exe on Windows or updateservice.tar.gz on Linux, installed/extracted into the same folder as the Agent). Installers 6.3.4-SNAPSHOT and later bundle it. The Windows update service appears as EtlworksUpdateService ("Etlworks LLC. Update Service").
Legacy: run as a standalone app
Windows: from an elevated prompt in the Agent directory, run integrationagent.exe standalone. Cross-platform: with system Java 1.8 as the default Java, run java -jar integrationagent.jar standalone. Standalone JVM memory is configured in AGENT_HOME/appstart.properties (app.vm.options=-Xms100m -Xmx8000m); stop the Agent, edit, start.
Legacy: change JRE and JVM memory parameters
Windows service: settings live in AGENT_HOME/install.cmd (JAVA_HOME, PR_JVM, PR_JVMOPTIONS; the JRE must be Java 8). Stop the Agent; set PR_JVMOPTIONS to a size in MB (8000), an -Xmx value (-Xmx8000m), or auto (70% of RAM); run remove.cmd, then install.cmd; start the Agent.
Linux service: edit MAX_MEM_OVERRIDE_VALUE in AGENT_HOME/install.sh (a size in MB or auto), re-run sudo ./install.sh, and start the Agent.
Legacy: troubleshoot Windows service installation (WMIC)
Applies to Agent 9.8.0 and earlier only — starting with 9.8.1 the Agent no longer depends on WMIC. If install.cmd reports 'wmic' is not recognized as an internal or external command: confirm wmic is missing (run wmic), install it via Settings → Apps → Optional Features → Add a feature → WMI Tools, restart if required, and run install.cmd again.