Get started with databases

_TEMPLATES__etlworks.com_-_Get_started_banners.png

The Etlworks Integrator works equally well with all relational and NoSQL databases, as long as there is a JDBC driver for the database and the database can be accessed from the server that the Etlworks Integrator is running on.

What can you do with databases in Etlworks Integrator

Connecting to databases	Using a temporary (staging) database	Testing the database Connection
Browsing database objects	Executing SQL queries in the Etlworks Explorer	Migrating data from one database to another
ETL tables matching a wildcard name from one schema or database to another	ETL transformation where the source is a relational database	ETL transformation where the destination is a relational database
Bulk load data into a database	Execute any SQL	ELT (extract-load-transform) with databases
Change Data Capture (CDC)	Create pipeline to CDC data into relational database	Change replication using high watermark
Transaction management	Open database connection when needed	Tips and tricks when working with databases
Working with on-premise data	Performance tips when working with databases

Connecting to databases	Connecting to the temporary (staging) database	Testing the database Connection
Browsing database objects	Executing SQL queries in the Etlworks Explorer	Migrating data from one database to another
ETL tables matching a wildcard name from one schema or database to another	ETL transformation where the source is a relational database	Bulk load data into a database
ETL transformation where the destination is a relational database	Execute any SQL	ELT (extract-load-transform) with databases
Change Data Capture (CDC)	Create pipeline to CDC data into relational database	Change replication using high watermark
Transaction management	Open database connection when needed	Tips and tricks when working with databases
Working with on-premise data	Performance tips when working with databases	Automatically create staging tables from any data source

Videos

	How to move data between databases Create a source Connection for the relational database Create a destination Connection for the relational database. Create a new `Database to database` Flow. Add a new source-to-destination transformation. Configure mapping. Test transformation. Read the full article
	How to move tables matching a wildcard name to another schema or database Create a source Connection for the relational database Create a destination Connection for the relational database. Create a new `Database to database` Flow. Add a new source-to-destination transformation. Specify the fully qualified wildcard source name in `FROM`. For example, `public.`. Configure the destination name in `TO` using either only an asterisk character (``) or any part of the destination name with the asterisk character (`public.*`). Read the full article

Related case studies

Manufacturing company Data synchronization between the OLTP database and data warehouse Read the case study	"To incrementally load the data from the source into the destination, we will be using a feature called `Change Replication`. In order to work, the change replication requires each source table to have a so-called `High Watermark Field`. In our case, all source tables have a column called `timestamp`, which is a binary-encoded timestamp of when the record was last modified or inserted."
DevOps Load notifications from the Amazon SNS into the database Read the case study	"A DevOps team in an e-commerce company needs to save all email notifications emitted by the Amazon SNS service into the database for future analysis. Specifically, they want to know the rate of failures, what notifications have bounced, etc."

Articles in this section

What can you do with databases in Etlworks Integrator

Videos

Related case studies

"A DevOps team in an e-commerce company needs to save all email notifications emitted by the Amazon SNS service into the database for future analysis. Specifically, they want to know the rate of failures, what notifications have bounced, etc."

Comments

Articles in this section

What can you do with databases in Etlworks Integrator

Videos

Related case studies

"A DevOps team in an e-commerce company needs to save all email notifications emitted by the Amazon SNS service into the database for future analysis. Specifically, they want to know the rate of failures, what notifications have bounced, etc."

Related articles