Overview
dbt's most powerful production techniques — state-based selection (state:modified), defer, and dbt retry — all depend on artifacts from a previous execution. The Execute dbt Project flow makes these first-class through the State, Defer, and Retry control: Etlworks retains the artifacts of past executions, validates compatibility, and stages the exact files dbt needs before it starts.
Selecting state
For normal commands, the State, Defer, and Retry control offers four state sources:
- No state (default) — run without --state.
- Specific successful execution — pick one retained execution with a manifest.json.
- Latest successful execution — always use the most recent compatible successful execution at run time.
- Uploaded manifest.json — upload a manifest to retain for this flow (available after the flow has been saved).
Check Defer to state to also pass --defer with the selected state manifest — unbuilt upstream models then resolve to the relations recorded in the state.
Cross-flow state: the Source dbt flow selector lets one dbt flow use the state of another. This is allowed only for flows you are authorized to read, and only when the two flows are compatible dbt flows.
What Etlworks validates
Before dbt starts, the selected state is authorized and validated: account and project identity, local/Git compatibility, repository and subdirectory, regular-file status, path containment, symlinks, size, SHA-256, manifest version and schema, and dbt project identity. The state files are staged privately for the execution and removed afterward. State that fails validation is rejected with a clear error rather than silently ignored.
Native dbt retry
Set Command to retry to use dbt's own retry command, which re-runs only the nodes that failed or were skipped in a previous execution. This is different from a generic Etlworks flow retry, which re-executes the whole flow.
With Command = retry, the control switches to Retry failed execution: you select one retained failed execution with a run_results.json. An execution is selectable only when its context is compatible with the current flow configuration — compatibility covers the project and commit, dependency state, dbt and adapter versions, warehouse connection, schema, select/exclude expressions, variables, environment and secret references, threads, flags, and macro context.
The prior manifest is staged automatically when the retry needs it. Retry provenance — which execution was retried, and the original command — is shown in the dbt Results tab.
Note: Generic Etlworks flow retries and loop retries remain normal re-executions and do not use dbt retry.
State in the API and CLI
The flow-execution API and the Etlworks CLI accept a state override for directly executed dbt flows, so automated pipelines can select state without editing the saved flow. All the same authorization and compatibility validation applies.
Availability
Available in Etlworks 9.9.9 and later, for local dbt Core executions on the host and on compatible Integration Agents.