Overview
The Execute dbt Platform Job flow triggers an existing dbt Platform deployment job through the dbt Platform API, polls it to completion, and reports the result as a normal Etlworks flow execution — with logs, artifacts, notifications, scheduling, and orchestration.
The flow is destination-free: it does not require an Etlworks warehouse connection, because the remote job owns its warehouse environment and credentials. Etlworks does not create or edit dbt Platform projects, environments, jobs, or schedules — it runs jobs you already manage in dbt Platform.
Available in Etlworks 9.9.9 and later. If you run dbt Core locally instead, see Execute dbt Project flow.
Create the flow
1. Create a dbt Platform connection
With your API host, account ID, and token.
2. In the Flows window click + and type dbt. Select Execute dbt Platform Job.
3. Select the connection in the required dbt Platform slot.
4. Configure the parameters below.
Parameters
- dbt Platform Job ID: numeric job to run. If empty, the Default Job ID from the selected connection is used.
- Override Platform Job Command: when disabled (default), dbt Platform executes the job exactly as configured. When enabled, Etlworks replaces the job commands with the structured command below.
- Command: used only with the override — build (default), run, test, seed, snapshot, compile, source freshness, docs generate, or run-operation.
- Macro / Macro Arguments: package-qualified macro name and optional JSON arguments, used only by run-operation.
- Target Schema Override: optional dbt Platform schema_override.
- Select / Exclude / Variables: used only when the command override is enabled.
- Threads: passed as the dbt Platform threads_override (default 4).
- Full Refresh / Fail Fast: flags for the overridden command.
- Poll Interval (ms): remote status polling interval, 1000–60000 (default 5000).
- Timeout (ms): when it expires, Etlworks requests remote cancellation (default 3600000).
- Capture Remote Logs: copies bounded dbt Platform run-step logs into the Etlworks execution log after completion (default on).
- Execute if Error, On Exception, Exception Mask, Disable manual flow execution: standard Etlworks flow behavior.
Polling, rate limits, and duplicate prevention
Etlworks polls the remote run with bounded requests and handles dbt Platform API rate limits and transient failures. A trigger request that might create duplicate work is never blindly retried — Etlworks marks each trigger with a cause and uses it to recover an uncertain trigger result, so one flow execution corresponds to at most one remote run.
Cancellation semantics
A timeout or user cancellation requests cancellation of the remote run. Because the run belongs to dbt Platform, Etlworks distinguishes three outcomes and records which one occurred:
- Cancellation confirmed — dbt Platform reported the run as cancelled.
- Cancellation requested — the request was accepted, but the terminal state was not yet observed.
- Uncertain — the remote state could not be determined.
The remote run URL and identifiers are preserved in every case, so you can always open the run in dbt Platform.
Results and artifacts
The execution appears in the standard dashboard with a dbt Results tab: normalized success, failure, or cancellation, models/tests/timing parsed from the run artifacts, the remote job ID, status, and a link to the remote run. Raw artifacts are retrieved and downloadable, subject to the same limits and retention as local executions.
Orchestration and automation
The flow participates in nested flows, Composer, loops, schedules, notifications, permissions, audit, the Run Flow API (with the dbt override object for job ID, polling, and command override), and the CLI. It can be assigned to an Integration Agent, which invokes and monitors the API — no local dbt installation is required on the agent for Platform jobs.