Working with Inbound Emails and Attachments in Etlworks

Overview

This guide explains how to extract, filter, transform, and delete emails and attachments using Inbound Email Connections in Etlworks. You’ll learn how to use emails as a source in ETL flows, copy attachments to files, read emails in bulk, and apply filters and post-processing options like deletion or marking messages as read.

ETL Emails and Attachments into a Destination 

You can use an Inbound Email Connection as the source in a source-to-destination transformation. The flow will extract data by reading emails or attachments, transform it, and load it into the configured destination.

Step 1. Create an Inbound Email Connection

This connection will serve as the source.

Step 2. Create a Destination Connection

Set up the destination where the transformed data will be loaded.

Step 3. Create the ETL flow

Create a ETL Flow where the source is an inbound email connection (enter email to in the Select Flow Type field):

Step 4. Configure Source and Destination

Set the FROM connection to the inbound email source created in Step 1.

Set the TO connection to the destination created in Step 2.

Use appropriate Formats for both source and destination.

Step 5. Filter Emails by Subject (optional)

Use the FROM field to define an email subject. Wildcards are supported.

If multiple emails match, Etlworks selects the first one based on its received timestamp.

Step 6. Configure Mappings and Parameters

• If the email contains attachments, the largest attachment will be used as the source.

• If there are no attachments, the email body will be used instead.

• To filter attachments by name, use the File property in the Inbound Email Connection (wildcards allowed).

• If the Dump Attachments To property is set, matching attachments will be saved to the specified folder and no transformation will take place.

Copy Email Content or Attachments to a File

You can use a Copy Files flow to move email content or attachments to a supported file-based destination.

Step 1. Create an Inbound Email Connection (source)

Step 2. Create a Connection to be used as the destination. Supported types of destination Connections are:

• Cloud storage
• File storage
• Redis
• HTTP

Step 3. Create a Copy Files Flow

Step 4. Configure Source and Destination

Set the FROM connection to the inbound email source.

Set the TO connection to the file-based destination.

Step 5. Filter Emails by Subject (optional)

Use the FROM field to filter emails by subject. Wildcards are supported.

The first matching email (by received time) will be used.

Read emails and Save Attachments in Bulk

Use the flow type Read emails, save messages and attachments to the folder when you need to quickly process a large number of emails from a slow inbound connection.

Use flow type Read emails, save messages and attachments to the folder when you need to read hundreds or thousands of emails as fast as possible from a relatively slow inbound email connection. 

• This flow is optimized for speed and can process hundreds or thousands of messages efficiently.

• Saved messages and attachments can then be processed using any file-to-destination flow.

NOTE: You can also use a regular ETL flow with an inbound email source, but it will be slower than the specialized bulk-read flow.

Filter inbound emails

You can filter which emails are processed using the Filter section of the Inbound Email Connection .

Available filters:

• Read Only New: If enabled, only unread messages will be retrieved.

• From: Filters emails by the sender. Partial matches allowed (e.g., etlworks.com).

• Subject: Filters emails by subject line. Partial matches allowed (e.g., important!).

• Folder: The name of the folder to read from (default is INBOX).

• File: Filters messages based on attached file names (wildcards allowed).

• Received After: Reads only messages received after the specified date (MM/DD/YYYY).

• Received Before: Reads only messages received before the specified date (MM/DD/YYYY).

Delete Emails After Processing

To automatically delete emails after they are processed:

• Enable the Delete After Reading property in the Inbound Email Connection.

  This option is disabled by default.

Mark Emails as Read After Processing

To automatically mark emails as read (seen) after they are processed:

• Enable the Mark as Seen property in the Inbound Email Connection.

  This option is also disabled by default.

Browse Inbound Emails in Etlworks Explorer

You can browse the content of an inbound mailbox configured with an Inbound Email Connection directly from the Etlworks Explorer interface.