ETL, browse, filter, and delete emails and attachments

Overview

An Inbound Email connection can be used as a source in a source-to-destination transformation. The flow will extract data by reading emails and attachments, transform it and load it into the destination.

ETL emails and attachments into the destination 

Step 1. Create an Inbound Email Connection, which will be used as the source.

Step 2. Create any destination connection.

Step 3. Start creating a Flow in the Flows window by clicking + and selecting any Flow type where the source is a file.

Step 4. Modify the Flow by selecting the Connection created in Step 1 as the source (FROM), and the Connection created in Step 2 as the destination (TO). Use appropriate Formats for the source and destination.

Step 5. Use the field FROM to define an email subject. 

Use wildcards if needed. If there are multiple matching emails, Etlworks will select the first one based on the timestamp marking when the email was received.

Step 6. Add Mappings and modify Parameters if needed.

• If an email contains an attachment, the contents of the attached file will be used as the source.
• If there are no attachments to the email, Etlworks will use the actual email content (text) as the source.
• If there are multiple attachments to the email, Etlworks will select the biggest one.
• It is possible to filter out attachments by specifying the property File for an Inbound Email Connection. Wildcards are allowed.
• If the property Dump Attachments To for an Inbound Email Connection is not blank, Etlworks will simply dump all matching attachments (if there are any) to the specified folder and will ignore the rest of the transformation.

Copy the contents of an email or attachment into a file

Step 1. Create an Inbound Email Connection, which will be used as the source.

Step 2. Create a Connection to be used as the destination. Supported types of destination Connections are:

• Cloud storage
• File storage
• Redis
• HTTP

Step 3. Start creating a Flow in the Flows window by clicking + and selecting the Copy Files Flow type.

Step 4. Modify the Flow by selecting the Connection created in Step 1 as the source (FROM), and the Connection created in Step 2 as the destination (TO).

Step 5. Use the field FROM to define the email's subject. 

Use wildcards if needed. If there are multiple matching emails, Etlworks will select the first one, based on the timestamp marking when the email was received.

Read emails, save attachments to the folder

Use flow type "Read emails, save messages and attachments to the folder" when you need to read hundreds or thousands of emails as fast as possible from a relatively slow inbound email connection. 

You can then process messages and attachments saved as files using any of the file to destination flow types.

You can still use a regular source-to-destination flow type, where the source is an inbound email connection to read the emails and attachments but it will be slower than a designated flow optimized for reading email messages.

Filter inbound emails

Use the Filter section when configuring an Inbound Email Connection to filter inbound emails.

Available filters:

• Read Only New: if enabled, only new (unread) messages will be retrieved.
• From: use this if you need to filter emails based on the From field. Partial matching is used. For example: etlworks.com will include all emails sent from the domain etlworks.com and will exclude everything else.
• Subject: use this if you need to filter emails based on the Subject field. Partial matching is used. For example: important! will include all emails which include the subject important! and will exclude everything else.
• Folder: the name of the folder to read messages from. The default is INBOX.
• File: the name of the file attached to the message. Wildcard file names are allowed. If this field is specified, the Etlworks will expect an attachment that matches the given name.
• Received After: the date in a MM/DD/YYYY Format. If a date is entered, Etlworks will read only the messages received after the specified date and will ignore everything else.
• Received Before: the date in a MM/DD/YYYY Format. If a date is entered, Etlworks will read only the messages received before the specified date and will ignore everything else.

Delete inbound emails after processing

You can configure a Connection to delete emails after processing them.

Simply enable the property Delete After Reading. This property is disabled by default.

Mark inbound emails as read (seen) after processing

You can configure a Connection to mark emails as read (seen) after processing them.

Simply enable the property Martk as Seen. This property is disabled by default.

Browse inbound emails

In Etlworks, you can browse inbound emails for the configured Inbound Email Connection in Etlworks Explorer.