When to use this Flow type
In Etlworks, it is possible to create a Flow to merge CSV or XML files.
Merge CSV files
Use this Flow type when you want to merge multiple CSV files into a single file.
This Flow can merge two or more CSV files with a different structure. The final CSV file will include a superset of all fields from all source files.
Step 1. Create FROM
and TO
Connections (it is recommended that you use the same Connection for both).
Only at this time is the Server Storage Connection type supported.
Step 2. Create the FROM
and TO
Formats, which must be CSV.
Step 3. Select the Merge CSV Files
Flow type from the list.
Step 4. Create a transformation where the source Connection (FROM
) points to the files to merge, and the destination Connection (TO
) to the location where the merged file should be created. The FROM
and TO
Formats must be created in step 2.
Step 5. Enter a wildcard file name, such as *.csv
, into the FROM
field and the name of the destination file into the TO
field.
Step 6. Continue by specifying the transformation parameters:
Dynamic Header
: if you are not sure whether all source files have the same header, set this parameter toYes
.Header
: the header to be added to the merged file. This parameter is ignored ifDynamic Header
is set toYes
.Footer
: the footer to be added to the merged file. This parameter is ignored ifDynamic Header
is set toYes
.On Exception
: by default, any error causes the halt of execution. Whenignore
is selected, the errors will be ignored and execution will continue. Read how to configure the Flow to ignore all or specific exceptions.Exception Mask
: you can specify what errors should be ignored and still halt the execution for all other errors. Enter part or all of the exceptioflown string. This field works only when theIgnore On Exception
option is selected.Execute if Error
: if this option is selected, Etlworks will execute this file operation when an error occurs.
Merge XML files
Use this Flow type when you want to merge multiple XML files into a single file.
Step 1. Create FROM
and TO
Connections.
Step 2. Select the Merge XML Files
Flow type from the list.
Step 3. Create a transformation where the source Connection (FROM
) points to the files to merge, and the destination Connection (TO
) to the location where the merged file should be created.
Step 4. Enter a wildcard file name, such as *.xml
, into the FROM
field and the name of the destination file into the TO
field.
Step 5. Continue by specifying the transformation parameters:
Path for the repeating segment in XML
: an XML path for the repeating segment, for example,Transaction/Entries
. This field is required.Include common segment
: if this property is enabled, the system will try to detect a common, non-repeating segment, and include it in the output file.Do not merge files that have already been merged
: if this option is enabled, Etlworks will not merge files that have already been merged.On Exception
: by default, any error causes the halt of execution. Whenignore
is selected, the errors will be ignored and execution will continue.Exception Mask
: you can specify what errors should be ignored and still halt the execution for all other errors. Enter part or all of the exception string. This field works only when theIgnore On Exception
option is selected.Execute if Error
: if this option is selected, Etlworks will execute this file operation when an error occurs.
Comments
0 comments
Please sign in to leave a comment.