Automatically Decompress ZIP and GZIP Files When Reading from a Connection

Etlworks supports automatic decompression of ZIP and GZIP files when reading from any connection that works with files or file-like responses. This feature simplifies working with compressed data by eliminating the need to manually unzip files before processing them.

How It Works

The Expected Compression setting, available in most file-based and API connectors, allows you to configure how incoming resources should be handled:

• No Compression (default): Resources are read as-is.

• ZIP: Each file or response is treated as a single-entry ZIP archive and automatically decompressed.

• GZIP: Each file or response is treated as a GZIP-compressed item and automatically decompressed.

Once configured, Etlworks will attempt to decompress each resource using the selected format regardless of its file extension or content-type.

Where It Works

This setting is available in:

• All connections that work with files → See full list of file-based connectors

• HTTP API connector, when reading compressed responses from APIs. Read more.

Decompression is supported in both ETL flows and file-based flows, such as Copy and Move.

Important Notes

• This feature assumes that each compressed resource contains a single file or payload. It does not support ZIP archives with multiple files inside.

• If a resource is not actually compressed (despite the setting), Etlworks will still attempt to decompress it, which may result in an error.

• Use this feature only when you’re certain of the compression format being used on the source side.

When to Use

Use Expected Compression when:

• You receive GZIP-compressed logs, CSVs, or JSON files from an API or external storage.

• You want to streamline ingestion of zipped reports or data dumps without pre-processing.

• You’re reading from APIs that return compressed payloads in HTTP responses.

• You’re copying or moving files that arrive in compressed formats.