Using Memory Connection as a destination in any transformation

Overview

Almost all source-to-destination transformations can use a Memory Connection as a destination.

When the destination is a Memory Connection system, it extracts the dataset from the source and stores it in memory. The dataset can be used later to lookup data and update calculated fields, to parse the response from the web service, or for any other purpose which would typically require storing data in the temporary staging area.

Use in-memory dataset for lookup

Read how to use a lookup when the input a dataset:

• Lookup using an existing dataset and a SQL
• Lookup using an existing dataset and filter conditions

Use in-memory dataset to parse the response from the web service

Read how to use the in-memory dataset to parse the response from the web service.

Create a transformation with a destination set to Memory Connection 

Step 1. Create a Memory Connection.

Step 2. Create any extract-transform-load Flow.

Step 3. Add a source-to-destination transformation where the source is anything and the destination is Memory Connection created in step 1.

Step 4. Set the transformation name.

Step 5. Add other transformations as needed.

Access in-memory dataset from JavaScript

To access the in-memory dataset from the JavaScript within the source-to-destination transformation, use the following code:

var ds = scenario.getSources().get("CAPITALIZED_TRANSFORMATION_NAME").getDataSet();