Creating and managing flows

About Data Integration Flows

Data integration flow is an ETL or automation scenario which can be executed manually, on a schedule, or triggered by calling an API endpoint.

Read about available flows.

Creating a flow

In most cases, before you can start creating flows, you need to create all the required connections, formats, and, in some cases, listeners.

Step 1. To create a new flow, go to the Flows page and click the + button. Then click the corresponding flow type in the opened box.

To quickly find the correct type, you can enter all or a part of the name in the search box. After this, only the types which include that string in their names will be displayed. To filter flows by category, you can click the corresponding flow category from the list to the left of flow types.

Step 2. After a flow type has been selected, continue by defining the flow's transformations. To add a new transformation, click the + button.

Transformations can be modified, deleted, and rearranged using drag-and-drop.

Most of the transformations require FROM and TO connections and formats, as well as the names of the FROM and TO objects. Objects could be file names, database tables, etc.

Step 3. For each transformation, you will need to define the mapping and parameters.

Start defining the mapping by clicking the MAPPING button in the grid which contains the transformation.

Continue in the opened box by creating a per-field mapping, either automatically (by clicking the Create Mapping button), or manually, by adding fields one-by-one using the + button. You can modify mappings at any time. You can also rearrange fields using drag-and-drop.

Step 4. After the flow has been created, you can save it by clicking the Save button.

Managing flows

All saved flows are displayed in a grid on the Flows page and can be quickly filtered by name and type. Flows can be modified, duplicated, or deleted directly from the grid.

Exporting and Importing Flows

In Integrator, you can export any flow to the file saved locally, and later import it from that same file. When you export the flow, Integrator also exports all related connections and formats. Export/Import is especially useful when you wish to migrate flows from one instance of Integrator to another, or copy flows between different accounts.

Export a flow

Go to the Flows page and click the Export Flow button. Then, select the flow by name in the opened box and click the Export button. The file with the extension .intflow will be downloaded to your local drive.

Import a flow

Go to the Flows page and click the Import Flow button. Then, click the Choose file button and select the file with the extension .intflow that you wish to import. A new copy of the flow and all corresponding connections and formats will be created in Integrator.