How to load data into Snowflake

Snowflake is a columnar database, optimized for data retrieval operations, so loading data into Snowflake using DML statements (INSERT/UPDATE/DELETE) can be quite slow, especially for larger datasets.

While it is possible to load data in Snowflake using regular flows, such as database-to-database, it is highly recommend to use Snowflake-optimized flow. 

A typical Snowflake flow performs the following operations:

1. Extracts data from the source.
2. Creates CSV, JSON. Avro, or Parquet files.
3. Compresses files using the gzip algorithm.
4. Copies files into Snowflake area (local file system, Azure Blob, or Amazon S3).
5. Checks to see if the destination Snowflake table exists, and if it does not - creates the table using metadata from the source.
6. Dynamically generates and executes the Snowflake COPY INTO command.
7. Cleans up the remaining files, if needed.

Read how to efficiently load large datasets into Snowflake with Etlworks.