Overview
The initial step in a source-to-destination transformation involves extracting data from the source. During this step, Etlworks reads the source file or the response from a web service into memory before parsing and converting it into an internal data structure.
In certain scenarios, it may be beneficial to modify the source document before proceeding with the transformation. Common use cases include:
Handling Empty or Null Sources
Preprocessing can address situations where the source document is empty or null, ensuring the transformation proceeds smoothly.
Correcting Formatting Issues
If the source document is incorrectly formatted, preprocessing can reformat it into a standard structure such as XML, JSON, or CSV, enabling Etlworks to process it correctly.
Adding Metadata or Extra Information
Modifications can include adding additional data, such as metadata, to enhance the transformation process.
Custom Transformations
In some cases, the source document may need transformations such as stringifying nested nodes in a JSON document to meet specific requirements.
Additional Use Cases
Preprocessing can also be used to store either the raw source message or the modified document in a global or Flow variable, providing flexibility for further operations or reference within the Flow.
Create preprocessor
To create a preprocessor, use the following steps:
Step 1. Create an XML, JSON or CSV Format.
Step 2. When creating a Format, select preprocessor as the Transformation type.
Step 3. Enter JavaScript code into the Preprocessor field, which will modify the source document. You can reference the original source message using the message variable.
You can assign the new modified message to the value variable:
value = message.substring(12);
Or simply use the last line in the JavaScript code as a function which modifies the message:
message.substring(12);You can use a mishmash of JavaScript and Java methods to modify the original message.
Store source message in a global variable
com.toolsverse.config.SystemConfig.instance().getProperties().
put('unique key', message);
value = message;
To access the original source message from anywhere in the JavaScript code:
var message = com.toolsverse.config.SystemConfig.instance().getProperties().
get("unique key");