Configuring flow to process files by a wildcard filename
Use the following technique if you want to extract, transform, and load multiple files matching the wildcard mask.
Step 1. Create a connection for the source files. Make sure the connection supports wildcard filenames.
Step 2. Create a destination connection.
Step 4. Click the
MAPPING button and select the Parameters tab. Make sure that Process all files is enabled (note that it is enabled by default).
Deleting processed files
Loaded files will remain in the source folder after processing unless you enable parameter Delete loaded source files under the MAPPING/Parameters. You can also enable Delete source files on error, which will cause the flow to delete files in the sources folder if there was an error during the load.
Processing files in the specific order
When processing the files by a wildcard filename, for example,
*.csv the flow first captures the list of files to process, then sorts the list. The list is sorted using the selected algorithm for the source connection.
- Disabled - default sorting for the connection, mostly likely by filename with ascending order
- oldest - oldest files first
- newest - newest files first
- ascending - by filename with ascending order
- descending - by filename with descending order
- largest - largest files first
- smallest - smallest files first