Filter data using simple expression

Overview 

Filter is a transformation executed for each processed record in the source-to-destination transformation that can be used to reject records. You can use JavaScript or SQL syntax. 

Use filter transformation

It is possible to filter out certain rows of the source dataset using simple expression.

Filter is available under Mappings/ Additional Transformations/ Filter. 

Important: A JavaScript Filter must return true (include record) or false (exclude record).

The field values can be referenced from JavaScript code, using the "field name".

SQL-like syntax for filter

If Use SQL like syntax for Filter is selected, you can use operands available in SQL, for example, <>, and, or, etc. Otherwise, only JavaScript operands, for example, !=, &&, and || are allowed.

An example of an SQL-like filter expression:

"Importer Name" <> null and "Importer Name".contains("test");

An example of JavaScript-like filter expression:

"Importer Name" != null && "Importer Name".contains("test");

Available variables

The following variables can be referenced by name from JavaScript and Python code: