Preprocess source messages

Overview

In the source-to-destination transformation, the first step is extracting data from the source. When it happens, Etlworks reads either the source file or a response from the web service in memory before parsing and converting it to the internal data structure.

In many cases, it makes sense to modify the source document before transforming it, for example:

• When the source is empty or null.
• When the source is incorrectly formatted and requires preprocessing before Etlworks can read it as standard XML, JSON, or CSV.
• When the source would need to be modified to add extra information, for example, metadata. 

It can also be used to store the raw or transformed source message in a global or Flow variable. 

Create preprocessor

To create a preprocessor, use the following steps:

Step 1. Create an XML, JSON or CSV Format.

Step 2. When creating a Format, select preprocessor as the Transformation type.

Step 3. Enter JavaScript code into the Preprocessor field, which will modify the source document. You can reference the original source message using the message variable.

You can assign the new modified message to the value variable:

value = message.substring(12);

Or simply use the last line in the JavaScript code as a function which modifies the message:

message.substring(12);

You can use a mishmash of JavaScript and Java methods to modify the original message.

Store source message in a global variable

com.toolsverse.config.SystemConfig.instance().getProperties().
put('unique key', message);

value = message;

To access the original source message from anywhere in the JavaScript code:

var message = com.toolsverse.config.SystemConfig.instance().getProperties().
get("unique key");