Overview
Validation is a code transformation executed for each processed record in the source-to-destination transformation that can be used to reject records or the entire dataset. You can use JavaScript or Python to code the transformation.
Configure validation transformation
Validation, using JavaScript, is available under MAPPING
/ Additional Transformations
/ Validate
.
MAPPING
MAPPING/ Additional Transformations/ Validate
Examples
Generate an exception
The Flow execution will stop, and an exception will be generated.
// generate an exception when postal_code or phone are empty
// or postal_code is not a number
var postalCode = dataSet.getFieldValue(currentRow, 'postal_code');
var phone = dataSet.getFieldValue(currentRow, 'phone');
if (Utils.isNothing(postalCode) || Utils.isNothing(postalCode) ||
!Utils.isNumber(postalCode)) {
value = TaskResult.HALT;
} else {
value = TaskResult.CONTINUE;
}
Reject the dataset
The source-to-destination transformation will be skipped. No rows will be loaded into the destination.
// skip the transfromation when postal_code or phone are empty
// or postal_code is not a number
var postalCode = dataSet.getFieldValue(currentRow, 'postal_code');
var phone = dataSet.getFieldValue(currentRow, 'phone');
if (Utils.isNothing(postalCode) || Utils.isNothing(postalCode) ||
!Utils.isNumber(postalCode)) {
value = TaskResult.STOP;
} else {
value = TaskResult.CONTINUE;
}
Reject the current row
The current row will be skipped (not loaded in destination).
// skip the current row when postal_code or phone are empty
// or postal_code is not a number
var postalCode = dataSet.getFieldValue(currentRow, 'postal_code');
var phone = dataSet.getFieldValue(currentRow, 'phone');
if (Utils.isNothing(postalCode) || Utils.isNothing(postalCode) ||
!Utils.isNumber(postalCode)) {
value = TaskResult.REJECT;
} else {
value = TaskResult.CONTINUE;
}
Available variables
The following variables can be referenced by name from JavaScript and Python code:
Name | Class name / JavaDoc | Package |
---|---|---|
dataSet | com.toolsverse.etl.common.DataSet | com.toolsverse.etl.common |
currentRow | com.toolsverse.etl.common.DataSetRecord | com.toolsverse.etl.common |
etlConfig | com.toolsverse.etl.core.config.EtlConfig | com.toolsverse.etl.core.config |
scenario | com.toolsverse.etl.core.engine.Scenario | com.toolsverse.etl.core.engine |
row | current 0-based row number |
Comments
0 comments
Please sign in to leave a comment.