Tips, Shortcuts, and Edge Cases
Helpful tricks and lesser-known features for working with flows, data, and configurations across different use cases.
Monitoring and Diagnostics
User Interface Tips
Transformations and Mapping Tricks
- Ignore transformation when source has no records
- Ignore transformation when source file does not exist
- Use CLOB Format to transform source messages
- Modify the source message in a raw Format before passing it to the transformation
- Use a field value as a filename
- Use templates to define fields
Flow Control and Retry Logic
- Daemon flow
- How to retry only failed or not yet executed steps
- How to execute Flow conditionally
- Parallel loops
- Log parameters passed to the flow
- Converting Flow variables to global variables
Tips for Working with databases
- Using a value returned by SQL query as a parameter
- Insert auto-generated column value when working with databases
- Insert images from files into a database table
- Using MERGE (UPSERT) when working with databases
- Handle table and column names with spaces and special characters when working with databases
- Read data from one dataset and load it into multiple database tables
Tips for Working with APIs
- Capture the response code and response headers from the last HTTP call
- Copy response from the HTTP request to a file
- Send a content of the file as a payload for an HTTP endpoint
- Configure auto-retry when working with web services
- Create a Payload for an HTTP Call Using a Template with Tokens
- How to Use Memory Connection to Work with the Response from a Web Service
Tips for working with files
- Work with local files
- Use server storage as a staging area
- Process Files in an ETL Flow Using Wildcard Filenames
- Process the file and then delete it
- Read and write files from/to specific folder using the same Connection
- Read from and write into different Excel worksheets using the same Excel Format