Google BigQuery is a serverless, highly scalable, and cost-effective multi-cloud data warehouse designed for business agility. The Etlworks Integrator includes several pre-built Flows optimized for Google BigQuery.
Flows optimized for BigQuery
|Flow type||When to use|
||When you need to extract data from any source, transform it and load it into BigQuery.|
|Bulk load files in Google Cloud Storage into BigQuery||When you need to bulk-load files that already exist in Google Cloud Storage without applying any transformations. The flow automatically generates the COPY command and MERGEs data into the destination.|
|Stream CDC events into BigQuery||When you need to stream updates from the database which supports Change Data Capture (CDC) into BigQuery in real-time.|
|Stream messages from a queue into BigQuery||When you need to stream messages from the message queue which supports streaming into BigQuery in real time.|
ETL with BigQuery
Watch how to create ETL flows to load data in BigQuery
ELT with Google BigQuery
The Etlworks Integrator supports executing complex ELT scripts directly in Google BigQuery which greatly improves the performance and reliability of the data ingestion.
Reverse ETL with Google BigQuery
You can use any
Setup incremental replication using High Watermark (HWM)
Using HWM replication you can load only new and updated records into BigQuery.
Load multiple tables by a wildcard name
You can ETL data from multiple database objects (tables and views) into BigQuery by a wildcard name, without creating individual source-to-destination transformations.
Connect to Google BigQuery
Step 1. Open the
Connections window and click
Step 2. Type
bigqueryinto the search field.
Step 3. Select
Google Bigquery Connection and continue by defining the Connection parameters.
Read more about configuring BigQuery Connection.