How to Use Memory Connection to Work with the Response from a Web Service

Overview

In Etlworks, you can use a Memory Connection to work with the response from a web service. This is particularly useful in two common scenarios:

• Retrieving the unmodified response from the web service and saving it to a global variable.
• Parsing the response and performing actions based on the content, such as throwing an exception conditionally.

Retrieving and Saving the Unmodified Response from the Web Service

In this scenario, you retrieve the raw response from a web service and store it in a global variable for later use. This can be especially helpful if you want to log the response or use it in subsequent steps without altering it.

Process

Step 1. Create Memory Connection. 

Step 2. Create the source Format. It must the be same as format of the response returned by the web service, for example JSON.

Step 3: Use a Preprocessor in the source format to capture the raw response from the web service. The response can be in any supported format such as JSON, XML, or CSV. You can refer to the documentation on format preprocessors for more information on how to set up a preprocessor.

Example (JavaScript):

// store raw response into the global variable with 
// the name unique key
com.toolsverse.config.SystemConfig.instance().getProperties().
put('unique key', message);

// return unmodified response
value = message;

Step 4. Assuming that you have already created HTTP connection which will be used as source connection create a new flow using flow type web service to file.

Step 5. Create new source-to-destination transformation where:

• Source connection: HTTP connection
• Source format: format created in steps 2-3
• From: the endpoint
• Destination connection: memory connection created in step 1
• Destination format: format created in steps 2-3
• To: memory

Parsing the Response and Performing Conditional Actions

Example

In this use case, the goal is to call a web service, parse the response, and perform conditional actions based on the parsed data, such as throwing an exception if the flow finishes with an error.

Assume you need to call a flow using the API, and depending on the flow’s status, you either proceed or throw an exception if it executed with an error.

The response might look like this:

"flowId":123,
 "messageUuid":"UUID of the message", 
 "auditId":"ID of the record in audit table", 
 "status":4,
 "exception":"exception-stack-trace"
} 

The status can be one of the following:

• 0: just started.
• 1: Flow is still running.
• 2: Flow has been canceled.
• 3: Flow has not been executed.
• 4: Flow executed successfully. 
• 5: Flow executed with an error (exception). 

How it works

In this example, we will create a Flow that executes other Flows by name, reads the response, and throws an exception if the Flow called by name was executed with an error.

Step 1: Create the flow to be called (e.g., "the-transformation").

Step 2. Create a "caller" source-to-destination Flow, where the source is a web service, and the destination is a memory Connection. The source Connection is going to be an HTTP Connection pointed to the run Flow by name API (make sure that you are executing "the-transformation" Flow synchronously), and the source and destination Formats are going to be JSON. 

Step 3. Add a JavaScript code to the Flow created in step 2, which will be parsing the response from the web service. Click MAPPING and select Additional Transformations. Paste the following JavaScript in the After Extract field:

var status = dataSet.getFieldValue(dataSet.getRecord(0), 'status');
var exception = dataSet.getFieldValue(dataSet.getRecord(0), 'exception');

// error
if (status == 5) {
   throw 'There was an exception when executing the flow:' + exception; 
} 
// still running 
else if (status == 0 || status == 1) { 
   throw 'The flow is still running but expected to finish already'; 
} 
// canceled
else if (status == 2) { 
   throw 'The flow has been canceled'; 
} 
// has not been executed
else if (status == 3) { 
   throw 'The flow has not been executed'; 
}

Step 4. Save the caller Flow.

When the caller flow executes "the-transformation", it will parse the web service response. If the called flow ends with an error, the caller will throw an exception accordingly.