Snowflake is a cloud-native data warehouse. Etlworks ships several flow types optimized for loading and reading Snowflake at high performance.
Which Snowflake flow should I use?
| Flow | Use when |
|---|---|
| Any to Snowflake (Database / File / Queue / Web service / Well-known API) | You need to extract from any source, optionally transform, and load into Snowflake. |
| Bulk load files into Snowflake | The files already exist in an external Snowflake stage (S3, Azure Blob, GCS) or server storage. No transformation needed. Auto-generates COPY INTO; supports MERGE. |
| Stream CDC events into Snowflake | You need real-time replication from a CDC-enabled source database. |
| Streaming with message queues | You need real-time ingestion from a message queue that supports streaming. |
| COPY files into Snowflake (user-defined COPY INTO) | You want Etlworks to run your own COPY INTO command. Does not support automatic MERGE. |
What do I need before I start?
- An active Snowflake data warehouse.
- A Snowflake user with the necessary USAGE, INSERT, and (if you'll use MERGE or auto-create tables) CREATE TABLE / OWNERSHIP privileges on the target schema.
- A stage. Etlworks can create the named stage automatically, or you can point to an existing internal or external stage.
Connect to Snowflake
- Open the Connections window and click +.
- Type snowflake in the search field.
- Select the Snowflake connection and fill in the connection parameters. Full reference: configuring the Snowflake connection.
If you plan to load files from server storage rather than an external cloud stage, also create a server storage connection. For external stages, use one of Amazon S3, Azure Storage, or Google Cloud Storage.
Where to go next
| Topic | Article |
|---|---|
| Extract, transform, and load data into Snowflake | Extract, transform, and load data in Snowflake |
| Bulk-load existing files | Bulk load files into Snowflake |
| ELT — run transformation SQL directly in Snowflake | ELT with Snowflake |
| Reverse ETL — extract from Snowflake into any destination | Reverse ETL with Snowflake |
| Automatic stage creation | Automatic creation of the Snowflake stage |
| Data type mapping (JDBC ↔ Snowflake) | Data type mapping for Snowflake |
| Load many tables at once | Load multiple tables with a wildcard |
| Incremental load (HWM) | Incremental change replication using high watermark |
| Troubleshooting | Common issues when loading data into cloud data warehouses |