Overview
Google BigQuery is a serverless, highly scalable, and cost-effective data warehouse designed for business agility. The Etlworks includes several pre-built Flows optimized for Google BigQuery.
Flows optimized for BigQuery
| Flow type | When to use |
|
When you need to extract data from any source, transform it and load it into BigQuery. |
| Bulk load files in Google Cloud Storage into BigQuery | When you need to bulk-load files that already exist in Google Cloud Storage without applying any transformations. The flow automatically generates the COPY command and MERGEs data into the destination. |
| Stream CDC events into BigQuery | When you need to stream updates from the database which supports Change Data Capture (CDC) into BigQuery in real-time. |
| Stream messages from a queue into BigQuery | When you need to stream messages from the message queue which supports streaming into BigQuery in real time. |
Videos
|
ETL with BigQuery Watch how to create ETL flows to load data in BigQuery |
Related resources
|
ELT with Google BigQuery The Etlworks supports executing complex ELT scripts directly in Google BigQuery which greatly improves the performance and reliability of the data ingestion. |
Reverse ETL with Google BigQuery You can use any
|
|
Setup incremental replication using High Watermark (HWM) Using HWM replication you can load only new and updated records into BigQuery. |
Load multiple tables by a wildcard name You can ETL data from multiple database objects (tables and views) into BigQuery by a wildcard name, without creating individual source-to-destination transformations. |
Connect to Google BigQuery
Step 1. Open the Connections window and click +.
Step 2. Type bigqueryinto the search field.
Step 3. Select Google Bigquery Connection and continue by defining the Connection parameters.
Read more about configuring BigQuery Connection.
Comments
0 comments
Please sign in to leave a comment.