Vertica is a column-store analytical database. It runs in the cloud or on-premises. Etlworks ships flow types optimized for Vertica's bulk-load path.
Which Vertica flow should I use?
| Flow | Use when |
|---|---|
| Any to Vertica (Database / File / Queue / Web service / Well-known API) | You need MERGE into Vertica, or to move large datasets with optional transformation. For plain INSERTs, the generic database ETL flow is often a better fit — Vertica's INSERT is fast. |
| Bulk load files into Vertica | The files already exist in local or cloud storage. No transformation needed. Auto-generates COPY; supports MERGE. |
| Stream CDC events into Vertica | You need real-time replication from a CDC-enabled source database. |
| Streaming with message queues | You need real-time ingestion from a message queue that supports streaming. |
What do I need before I start?
- A Vertica cluster reachable from your Etlworks instance. For on-prem Vertica, see Working with on-premise data.
- A Vertica user with INSERT on the target tables (and CREATE TABLE if Etlworks should auto-create tables).
- A staging location: local server storage, or an S3 / Azure Blob / GCS bucket that Vertica can read from.
Connect to Vertica
- Open the Connections window and click +.
- Type vertica in the search field.
- Select the Vertica connection and fill in the connection parameters. Enable Auto Commit. Full reference: configuring the Vertica connection.
Also create a stage connection: server storage (default), Amazon S3, Azure Blob, or Google Cloud Storage.
Where to go next
| Topic | Article |
|---|---|
| Extract, transform, and load data into Vertica | Extract, transform, and load data in Vertica |
| Bulk-load existing files | Bulk load files into Vertica |
| ELT — run transformation SQL directly in Vertica | ELT with Vertica |
| Reverse ETL — extract from Vertica into any destination | Reverse ETL with Vertica |
| Load many tables at once | Load multiple tables with a wildcard |
| Incremental load (HWM) | Incremental change replication using high watermark |
| Troubleshooting | Common issues when loading data into cloud data warehouses |