Tips for working with files
Best practices and lesser-known features for reading and writing files and using file system efficiently.
- Work with local files
- Use server storage as a staging area
- Process Files in an ETL Flow Using Wildcard Filenames
- Process the file and then delete it
- Read and write files from/to specific folder using the same Connection
- Read from and write into different Excel worksheets using the same Excel Format
- Read from and write into different worksheets using the same Google Sheets Connection
- Update existing Excel XLSX spreadsheet
- Group rows in Excel XLSX worksheet
- Read an Excel worksheet with a header, not in the first line
- Use an Excel worksheet index when reading Excel workbooks
- Filter rows in a flat file
- Append rows to the existing file
- Disable automatic setting of the field data type when working with files
- Work with very large files
- Merge CSV or XML files
- Read non-standard CSV files
- Preserve the exact data types when extracting data from a database and creating the CSV files
- Split a CSV file when creating it
- Add a custom header to a CSV file
- Enable base64 encoding of fields in CSV files
- Configure XML parser
- Read XML files version 2.x
- Archive files before copying them to a cloud storage
- Read gz and zip files in ETL transformations
- Add metadata when creating files in S3 and Google Storage
- Work with region-specific S3 buckets
- Work with Multi-Region S3 Buckets (MRAPs)
- Capture or Modify the File Name for Source-to-Destination Transformation
- Automatically Decompress ZIP and GZIP Files When Reading from a Connection