Performance tips when working with files and web services

Use server storage when possible

It is usually faster to work with files stored in server storage than remote files or cloud storage.

• Step 1. Create a Flow, which copies/moves files to server storage.

• Step 2. Create a Flow, which works with a file in server storage.

• Step 3. Create a nested Flow, which combines the Flows created in Steps 1 and 2 together.

Use streaming if possible

To enable streaming, go to MAPPING > Parameters and enable Stream Data.

Note that streaming might not work if your source or destination files are complex nested data objects. 

Split large files

If the source file is large, especially if it is an XML file, it makes sense to split it into multiple smaller files, and then process them one by one in a loop. Read more about splitting files.

Use parallel loops when iterating through a large number of identical tasks

Read about parallel loops.

Enable parallel file operations when processing files by a wildcard name

Read about enabling parallel file operations.

Zip multiple files before copying then 

If you can, Zip Files before copying/moving them to remote or cloud file storage.