Etlworks Integrator 2.3 has been released!
New
1. Added nine new connectors
Affected areas: connectors
- Microsoft Sharepoint connector
- OneDrive for business connector
- Google Drive connector
- Box connector
- Dropbox connector
- Smartsheet connector
- Magento connector
- OData connector
- Amazon Marketplace connector
2. New account usage notifications and account dashboard
Affected areas: notifications, account dashboard
Account usage notifications
We added the following new notifications to help customers stay within the limits set for the specific subscription plan.
- Etlworks Integrator daily overage notice (day of the month) - sent to the account owner when the number of records processed during the previous day exceeded the maximum number allowed for the subscription plan.
- Etlworks Integrator monthly overage notice (month) - sent to the account owner when the number of records processed during the previous month exceeded the maximum number allowed for the subscription plan.
New account dashboard
We added more important information to the account dashboard but kept it clean and easy to navigate.
- New look&feel
- New data points: the total number of records for the day, the current date in the timezone, configured for the account, the total number of the records for the selected month, the limits set for the account
- Ability to see stats for the specific month
- Ability to refresh real-time stats
- The dashboard now displays the overage information, see examples below:
Exceeding the daily maximum
Potential monthly overage
Exceeding the monthly maximum
3. Improved handling of the schema changes when loading data in Snowflake and Redshift
Affected areas: loading data in Snowflake and Redshift
The following options are available for both databases (two new):
- Ignore fields in the source that do not exist in the target table - if this option is enabled and the source has more fields than the target table, the system will automatically exclude extra fields. Options 1, 2 and 3 are mutually exclusive.
- NEW. Alter target table if the source has columns that the target table doesn't have - if this option is enabled and the source has different fields than the target table, the system will add extra fields to the target table. Options 1, 2 and 3 are mutually exclusive.
- NEW. Recreate target table if the source has columns that the target table doesn't have - if this option is enabled and the source has different fields than the target table, the system will automatically drop and recreate the target table. This option is not available when HWM change replication is enabled. Options 1, 2 and 3 are mutually exclusive.
- Insert null into the fields in the target table that do not exist in the source - if this option is enabled and the target table has different fields than the source, the system will automatically insert NULL values into these fields in the target.
4. Added support for PGP encryption
Affected areas: scripting, PGP encryption
In this update, we added the ability to encrypt and decrypt files and messages using PGP (Pretty Good Privacy) algorithms.
5. Added the ability to use a client-side SSL certificate with HTTP connection
Affected areas: HTTP connector
Etlworks HTTP connector now supports mutual authentication, which allows the user to present a certificate as well as the server.
6. Change Data Capture (CDC) flows now support wildcard table names
Affected areas: CDC flows, loading data in Snowflake and Redshift
Prior to this update, it was required to configure individual source-to-destination transformations for each pair of source-target tables.
In this update, we introduced the ability to use wildcard table names. If the wildcard table name is used the system will perform a CDC change replication for all tables in the source with a name that matches the wildcard pattern (example: test.dbo.prod_*
)
7. Improved handling of the errors in CDC flows
Affected areas: CDC flows
In this update, we introduced new option for the CDC connections. If any error occurred and the option Switch to Snapshot if Error is enabled, one the next run the system will automatically switch to the snapshot mode and attempt to reload the whole table.
8. Added new flow type for merging data with a template
Affected areas: merging data with template
In this update, we introduced a new flow type: Merge data with a template.
Use it when you want to merge global variables with the template and send the resulting payload to any of the following destination connections:
- File-based connections
- HTTP
- Social networks
- Outbound email
- NoSQL databases
Read more.
9. Added the ability to configure output template for the CLOB format
Affected areas: CLOB format
If the Output Template field is not empty, the {tokens} in the template will be merged with global variables to produce the output.
10. Added the ability to parse comments in the XML document
Affected areas: XML parser, XML Format
In this release, we introduced the ability to parse comments in the XML document.
Fixed
1. Fixed FTPS connector
Affected areas: FTPS connector
Prior to this update, in some rare cases, the FTPS connector was returning an empty list even when there were files in the folder. We fixed it by introducing a new configuration option "Execute PBSZ and PROT commands After Login", which is enabled by default.
2. Updated Salesforce connector
Affected areas: Salesforce connector
Salesforce updated some of the APIs so we updated our Salesforce connector.
3. Fixed issue with email attachments
Affected areas: sending emails
Prior to this release, the system was always sending an attachment even when the outbound email connection wasn't configured to have one. We fixed it in this release.
4. Fixed HTTP authentication when the authorization header parameter is user-defined
Affected areas: HTTP authentication when the authorization header parameter is user-defined
Previously, it wasn't possible to use tokens {user} and {password} in the user-defined authorization header. We fixed it in this release.
5. Fixed parsing of the x-www-form-urlencoded payload by the HTTP Listener
Affected areas: user-defined PUSH API, HTTP listener
Previously, if the payload was sent using x-www-form-urlencoded
content type the HTTP listener wasn't able to process it. We fixed it in this release.
Comments
0 comments
Please sign in to leave a comment.