How to extract data from web service and load into database

This flow extracts data from a web service and loads into a relational database.

In Connections

Step 1. Create a source HTTP connection.

Step 2. Create a destination connection for the relational database. Test it.

Step 3. Create a format for the payload returned by web service, most likely JSON or XML.

In Explorer

Step 4. Link connection and format and test the response for the web service in the Explorer.

Step 5. It is most likely that the response is a nested JSON or XML so learn how to flatten the nested dataset.

In Flows

Step 6. Create a new web service to database flow.

Step 7. Add a new transformation and select from connection, format, and endpoint and to connection and table.

Step 8. Click the MAPPING button and configure the "flattening" for the nested JSON or XML. Most likely it is going to be a Source SQL.

Step 9. Test the transformation. Make sure it returns the flat data set.

Step 10. Add per-field mapping if needed.

Step 11. Configure MERGE (UPSERT) if needed.

Step 12. Save the flow and execute it manually.

In Schedules

Step 13. Schedule the flow to be executed periodically.