Articles in this section

See more

Get started with databases

  • Updated
Follow

_TEMPLATES__etlworks.com_-_Get_started_banners.png

Etlworks works with almost all known relational databases.

What can you do with databases in Etlworks

Connecting to databases

Using a temporary (staging) database

Testing the Database Connection

Browsing database objects

Executing SQL queries in the Etlworks Explorer

ETL with relational databases

ETL tables matching a wildcard 

ETL transformation where the source is a relational database

ETL transformation where the destination is a relational database

Bulk load data into a database

Execute any SQL

ELT (extract-load-transform) with databases

Change Data Capture (CDC)

Create a pipeline to CDC data into a relational database

 Change replication using a high watermark

Change replication using CT (change tracking)

Open database connection when needed

Transaction management

Working with on-premise data

Performance tips when working with databases

Tips and tricks when working with databases

Automatic partitioning

 

 

 

Videos

ETL and CDC with databases

  1. Create a source Connection for the relational database
  2. Create a destination Connection for the relational database.
  3. Create a new Database to database Flow.
  4. Add a new source-to-destination transformation.
  5. Configure mapping.
  6. Test transformation.

Read the full article

ETL multiple tables by wildcard

  1. Create a source Connection for the relational database
  2. Create a destination Connection for the relational database.
  3. Create a new Database to database Flow.
  4. Add a new source-to-destination transformation.
  5. Specify the fully qualified wildcard source name in FROM. For example, public.*. Configure the destination name in TO using either only an asterisk character (*) or any part of the destination name with the asterisk character (public.*).

Read the full article

 

Related case studies

GEN-I case study

ETL data from multiple SQL Server tables into Snowflake using wildcard-based workflows, parallel partition processing, and high-watermark incremental loads.

 

GEN-I, a global leader in energy trading, needed a powerful data integration solution to efficiently ETL data from their SQL Server databases into Snowflake. The company sought a platform that could handle high-volume data with speed and accuracy while leveraging advanced ETL techniques to optimize performance and scalability across multiple tables.

Community Health Plan of Washington case study

ETL and CDC data from SQL Server and Oracle databases to Azure Synapse.

 

Community Health Plan of Washington (CHPW), a nonprofit providing health coverage for Washington residents, leverages Etlworks to efficiently ETL data from SQL Server and Oracle databases into their Azure Synapse data warehouse. Etlworks’ ability to handle frequent full database reloads with exceptional performance and reliability makes it a cornerstone of CHPW’s data integration strategy.

Intertek Alchemy case study

Stream CDC data from 1,500 MySQL databases into Snowflake in real-time.

Intertek Alchemy, a global leader in workforce training solutions, faced a monumental challenge: seamlessly streaming real-time Change Data Capture (CDC) events from over 1,500 MySQL databases into Snowflake. Despite exploring multiple data integration platforms, no solution on the market could meet their scalability and real-time performance requirements—until they discovered Etlworks.

More case studies

 



etlworks.com_-_free_trial_banners_2.png

Related articles

Comments

0 comments

Please sign in to leave a comment.