How to load data into Snowflake

Snowflake is a columnar database optimized for data retrieval operations, so loading data into it using DML statements (INSERT / UPDATE / DELETE) can be quite slow, especially for larger datasets.

While it is possible to load data in Snowflake using regular Flows, such as Database to database, it is highly recommended to use Snowflake-optimized Flow. 

A typical Snowflake Flow performs the following operations:

1. Extracts data from the source.
2. Creates CSV, JSON, Avro, or Parquet files.
3. Compresses files using the gzip algorithm.
4. Copies files into Snowflake area (local file system, Azure Blob, or Amazon S3).
5. Checks to see if the destination Snowflake table exists and if it does not create the table using metadata from the source.
6. Dynamically generates and executes the Snowflake COPY INTO command.
7. Cleans up the remaining files, if needed.

Read how to efficiently load large datasets into Snowflake with the Etlworks Integrator.