Get started with databases

_TEMPLATES__etlworks.com_-_Get_started_banners.png

Etlworks works with almost all known relational databases.

What can you do with databases in Etlworks

Connecting to databases	Using a temporary (staging) database	Testing the Database Connection
Browsing database objects	Executing SQL queries in the Etlworks Explorer	ETL with relational databases
ETL tables matching a wildcard	ETL transformation where the source is a relational database	ETL transformation where the destination is a relational database
Bulk load data into a database	Execute any SQL	ELT (extract-load-transform) with databases
Change Data Capture (CDC)	Create a pipeline to CDC data into a relational database	Change replication using a high watermark
Change replication using CT (change tracking)	Open database connection when needed	Transaction management
Working with on-premise data	Performance tips when working with databases	Tips and tricks when working with databases

	ETL and CDC with databases Create a source Connection for the relational database Create a destination Connection for the relational database. Create a new `Database to database` Flow. Add a new source-to-destination transformation. Configure mapping. Test transformation. Read the full article
	ETL multiple tables by wildcard Create a source Connection for the relational database Create a destination Connection for the relational database. Create a new `Database to database` Flow. Add a new source-to-destination transformation. Specify the fully qualified wildcard source name in `FROM`. For example, `public.`. Configure the destination name in `TO` using either only an asterisk character (``) or any part of the destination name with the asterisk character (`public.*`). Read the full article

Manufacturing company Data synchronization between the OLTP database and data warehouse Read the case study	"To incrementally load the data from the source into the destination, we will be using a feature called `Change Replication`. In order to work, the change replication requires each source table to have a so-called `High Watermark Field`. In our case, all source tables have a column called `timestamp`, which is a binary-encoded timestamp of when the record was last modified or inserted."
DevOps Load notifications from the Amazon SNS into the database Read the case study	"A DevOps team in an e-commerce company needs to save all email notifications emitted by the Amazon SNS service into the database for future analysis. Specifically, they want to know the rate of failures, what notifications have bounced, etc."