ETL Data Between Relational Databases

Overview

This flow extracts data from one relational database, optionally transforms it, and loads it into another relational database or a different schema within the same database.

Step-by-Step Instructions

Step 1. Go to Connections and create a connection for the source relational database. Test the connection.

Step 2. Create a connection for the destination relational database. Test the connection.

Step 3. Go to Flows and create a new Database to database flow.

Step 4. Add a new transformation. Select the FROM connection and table, and the TO connection and table.

Step 5. (Optional) Click Configure and configure a custom Source SQL if needed. Leave it blank to extract all data from the source table.

Step 6. (Optional) Configure per-field mapping if the source and destination schemas differ.

Step 7. Test the transformation.

Step 8. (Optional) Enable MERGE (UPSERT) if you want to update existing records.

Note: Enabling Predict Lookup Fields allows flow to automatically select keys for MERGE. Alternatively you can manually specify them using fully qualified table names and semicolon-separated pairs in the Lookup Fields:

Step 9. (Optional): Configure Change Replication to track and load only modified data

Step 10. (Optional): To fully refresh destination tables, add the following SQL to the Before SQL field:

truncate table {TABLE}

Step 11. Add as many pairs of source-to-destination transformations as needed.

Step 12. Save the Flow and execute it manually.

Step 13. Schedule the flow

• Go to Schedules

• Create a schedule to run the flow periodically

Next Steps

Want to move multiple tables using a wildcard? Learn how →