Overview

Etlworks provides multiple ways to monitor Flow execution in real time and review past runs.

You can:

Monitor running Flows
Review execution history and results
Inspect logs, errors, and metrics
View running tasks and scripts
Troubleshoot issues using SQL and debug information

All monitoring starts from the Flow Statistics dashboard.

Flow status

Each Flow execution has a status displayed in the Flows screen:

Running – Flow is currently executing
Success – completed successfully
Error – failed
Warning – completed with issues
Canceled – stopped manually

Click the Status link to open detailed execution information.

Flow Statistics dashboard

The Flow Statistics dashboard is the central place for monitoring and troubleshooting.

You can access it from:

1. Flows screen (Status column)

2. Schedules screen

3. Flow Builder

4. Agents screen

Execution details

Status
Duration
Execution ID
Trigger source (user, schedule, agent, listener)

Execution history

Displays a list of past executions:
Last 50 runs by default (can load more)
Shows who started the Flow and when
Click any entry to view full details

Direct links to executions

Each execution has a unique Execution ID.

You can open or share a specific execution using:

/#/app/flows/{flowId}/statistics/{executionId}

/#/app/schedules/{scheduleId}/statistics/{executionId}

Visual indicator for retries

Flow history visually indicates retries triggered by the scheduler or integration agent, including a parent process ID. This helps differentiate retries from regular executions.

Filtering execution history

You can filter Flow execution history using the filter panel. To open it, click the small filter icon next to the History label.

Filtering Options

Once the filter panel is open, you can narrow down the Flow execution history using the following options:

Date range (From / To)

From date: Shows executions starting from 00:00 (midnight) of the selected date, in the Admin or tenant timezone.
To date: Shows executions up to the end of the selected date, in the Admin or tenant timezone.

Together, these restrict results to a calendar date range.

Time range (From time / To time)

From time: Shows executions that happened at or after the specified time of day, in the Admin or tenant timezone. If more than one day of executions is displayed, this filter applies to each day individually.
To time: Shows executions that happened before the specified time of day, in the Admin or tenant timezone. If more than one day of executions is displayed, this filter applies to each day individually.
Both filters are independent of the From date / To date filters.

Status

Filter executions by status:

running
success
warning
error
canceled

Trigger type

Filter by how the execution was triggered:

schedule
on-demand
listener
agent

Started by

Identifies who or what initiated the execution:

If started manually → user login.
If started by schedule → schedule name.
If executed on an agent → agent name.

Description

Filter by the description associated with a Flow executed on an agent.

Metrics

Show executions whose Metrics or File Metrics contain the provided text.

Execution ID

Filter by a unique execution ID associated with a specific Flow run.

Quick filtering from charts

You can also click directly on a slice of the pie chart (right-hand panel) to quickly filter executions by status (e.g., show only Errors).

Metrics

Metrics provide insight into how data is processed. Starting with Etlworks 9.8.0 you can review them in two ways: Flat, the list of processed transformations, and Hierarchy, the runtime tree of the execution — see Flat and Hierarchy views.

Switch between Flat and Hierarchy views

The Metrics tab can present an execution in two ways. Flat is the familiar list of processed transformations. Hierarchy shows how the execution actually unfolded — the main flow, the nested flows it called, the steps inside them, loop iterations, retries, and error handlers — so you can see the runtime context of any step.

Flat metrics are excellent for reviewing individual transformations, record counts, timestamps, and results. What they do not show is where a step sat in a deeply nested flow. Hierarchy answers that: it tells you the parent flow, the grandparent flow, and the complete execution path of a failed step, which loop iteration and retry attempt it belonged to, and which conditional branches actually ran.

Hierarchy is especially useful for:

Finding the parent flow, grandparent flow, and complete execution path of a failed step.
Troubleshooting deeply nested workflows.
Understanding which conditional branches ran or were skipped.
Reviewing flows executed in loops, and identifying the iteration and retry attempt associated with an error.
Understanding parallel branches while keeping their parent-child relationships visible.
Viewing on-error handlers in the context where they were invoked.
Associating source-to-destination transformations with the flow that executed them.
Distinguishing multiple invocations of the same reusable subflow.

Availability. The Hierarchy view is available in Etlworks 9.8.0 or newer. Executions that run on an Integration Agent require an upgraded agent that supports hierarchical execution metrics. Older agents remain compatible with a newer Etlworks host — their executions simply continue to display the existing flat metrics.

Flat

Preserves the existing metrics presentation and groups the recorded data-processing metrics by flow.
Shows individual transformations, record counts, duration, status, high-water marks, and exceptions.
Remains the best view for comparing transformation-level processing statistics.
Supports CSV export.

Hierarchy

Displays the actual runtime tree of the flow execution.
Shows the main flow, nested flows, executed steps, loop invocations, retries, on-error flows, and the transformations associated with each flow.
Preserves parent-child relationships even when branches run conditionally, repeatedly, or in parallel.
Shows flow-specific icons, so you can tell at a glance what type of flow was executed.
Links saved flow rows directly to the Flow editor. The link opens the flow in a new browser tab.

When each view is available

Flat is the default when both flat metrics and hierarchical data are available.
If an execution has hierarchical data but no usable flat metrics, Etlworks opens Hierarchy automatically and disables Flat.
If an execution has no hierarchical data — an older execution, or one reported by an older Integration Agent — the Flat/Hierarchy selector is not displayed at all. Those executions continue to use the Flat view because hierarchical information was not recorded for those runs; all previously stored metrics remain fully supported.

Hierarchy columns

Status — the status icon at the start of each row.
Flow / Transformation — the executed flow, group, or transformation.
Step — the step within the parent flow.
Loop — the loop iteration, when the row belongs to a loop.
Attempt — the retry attempt number.
Started and Finished — timestamps.
Duration.
Extracted and Loaded — record counts.
Execution Path — the full path from the main flow to this invocation.

Drag the separators in the column headers to resize columns. This works in both the Flat and Hierarchy views.

Statuses and errors

Each row carries a status indicator:

Running — still executing.
Success — completed successfully.
Warning or ignored error — completed, but something was reported or an error was ignored.
Error — failed.
Canceled — stopped before completion.
Skipped — evaluated but not executed, for example a conditional branch whose condition was not met.

When an error has exception details, click the error status icon to view the exception.

Expanding and collapsing the hierarchy

Click the caret beside a row to expand or collapse that branch.
Click Expand All to expand the flow structure.
Click Collapse All to return to a compact overview.

Transformations are grouped under the flow that produced them and are collapsed by default. Expand the Transformations (N) row to see the individual transformations. Flow types that have no source-to-destination transformations — scripting flows, for example — do not show a Transformations group at all.

Loops and large numbers of iterations

Loops are summarized rather than listing every iteration:

Repeated invocations appear as a single Loop summary row. The summary reports the iteration count, the number of failed iterations when there are any, and any additional retry attempts.
Failed iterations are grouped separately, so they are easy to find.
Up to 100 iterations can be displayed directly.
Larger loops are divided into expandable ranges of 100 — for example Iterations 1–100 and Iterations 101–200. Expanding a range shows the individual iterations and their child steps.
Expand All keeps large loops summarized, so the dashboard never tries to render thousands of rows at once.
The Attempt column identifies retry attempts for an iteration.

Conditional, parallel, and error handling

Hierarchy reflects what actually happened at runtime, not just the static design of the nested flow:

Conditional branches that executed appear in their actual parent context.
A child that was evaluated but not run appears with Skipped status.
Parallel branches keep their parent-child relationships.
On-error flows appear under the invocation that triggered them.
Retried executions include the attempt number.
The Execution Path column gives the full path from the main flow to the selected invocation.

Example: locating a failed step

Main customer import
  → Validate input
  → Process regions — Loop
      → Iteration 17
          → Load customer records
              → Transformation: customers.csv to customers
              → Error

This immediately identifies the failed flow, its parent flows, the loop iteration, and the affected transformation — without reconstructing the context from the console log.

Transformation status

	Success	The transformation was successfully executed.
	Warning	The transformation was executed with a warning or not executed at all. Check the Warning column for more details.
	Error	The transformation was executed with an error. Click the error icon for more details.

Records Metrics

Shows:

Start and End timestamps
Records extracted
Records loaded
Transformation status: Success, Warning, Error
Last Recorded High Watermartk
Exception

File Metrics

Shows file operations such as:
Copy
Move
Delete
Rename

Nested Flows

Metrics are grouped by Flow when using nested Flows.

In the Hierarchy view the same execution is shown as a runtime tree instead of a grouped list, so you can see which flow called which, follow the complete Execution Path down to a failing step, and tell apart multiple invocations of the same reusable subflow. See Flat and Hierarchy views.

Metrics grouped by Flow for a nested Flow execution

Advanced metrics options

Recording all steps when executing nested flow

By default the flow only reports Records Metrics (metrics for processed source-to-destination ETL transformations) and File Metrics (proceeded files and folders). If you want the nested flow to report all executed steps, for example scripts, calling HTTP endpoints, etc., enable Track metrics for all types of flows for the main nested flow.

When this option is enabled the Records Metrics tab includes all flows, executed as a part of the main nested flow.

Relationship to the Hierarchy view. Track metrics for all types of flows controls which operations are included in the traditional Flat metrics — scripts, HTTP calls, and other non-ETL flow types. The Hierarchy view records the runtime structure of the nested flow execution, so it does not need this option merely to show the parent-child flow hierarchy.

Independently track metrics for each loop iteration

By default the flow aggregates Records Metrics across all loop iterations. To track metrics for each loop iteration independently enable Track metrics for loops.

When this option is enabled the dashboard displays metrics for each loop iteration:

Relationship to the Hierarchy view. Track metrics for loops controls whether traditional Flat record metrics are separated for each loop iteration. The Hierarchy view represents loop invocations in the execution tree independently of this option, and summarizes large loops into expandable ranges as described in Flat and Hierarchy views.

Filtering and Exporting Metrics to CSV

The Flow Stats Dashboard allows you to filter both Flow Metrics and File Metrics and export metrics to a CSV file.

Metrics can be filtered using the search field at the top of metrics panel. Filtering applies to all visible attributes, such as flow name, step name, file name, status, timestamps, duration, and record counts. The filter is applied immediately and updates the displayed results in real time.

The search field works in both the Flat and Hierarchy views. In Hierarchy it can match the flow name, step name, status, loop iteration, retry attempt, execution path, exception text, and transformation names and metrics.

For very large loops, Hierarchy shows the first 500 matching iterations and asks you to refine the search if there are more. A broad search that matches an entire loop leaves that loop summarized instead of expanding every iteration.

CSV export is available from the Flat metrics and from File Metrics. Hierarchy is an interactive troubleshooting view and is not exported directly — to export transformation metrics, switch to Flat and click Download CSV.

Metrics can be exported with or without an active filter. To export metrics, click the Download CSV button in the top-right corner of the dashboard.

Download CSV button in the metrics toolbar

If a filter is applied, the exported CSV file contains only the metrics that match the current filter. If no filter is applied, the CSV file contains all available metrics currently displayed in the selected tab.

The exported file name depends on the active tab:

From the Metrics tab, the file is named metrics-<auditId>.csv
From the File Metrics tab, the file is named file-metrics-<auditId>.csv

Monitor running Flows (real-time)

To monitor a Flow while it is running:

Open the Flows screen
Click the Running status
Open the Flow Statistics dashboard

From there you can:

View live metrics
Open logs
Monitor tasks

Flow logs

Flow logs provide detailed execution output.

View logs

Open the Flow Statistics dashboard
Click Console View

This opens a live (tail) view of the log.

Download logs

Open the Flow Statistics dashboard
Click Console Download

Read how to download Flow log using API.

Notes

Logs are available for running and completed Flows
Logs are stored in the file system
Requires Capture console log enabled

Monitor running tasks and scripts

Etlworks tracks currently executing operations such as:

SQL scripts JavaScript and Python code
Data extraction and loading
File operations
CDC and Streaming

Monitor tasks for a specific Flow

Open the Flow Statistics dashboard
Click View Running Tasks
Optionally enable Auto-refresh

Monitor tasks across all Flows

Open the Statistics screen
Expand Running flow tasks