This is a major release which in addition to the new features includes a lot of improvements, fixes, and changes under the hood. Self-hosted customers are encouraged to upgrade their instances.
Version
New
1. Improvements for log-based change replication (CDC)
Upgraded Debezium to 1.3
Affected areas: CDC flows, CDC connections
We upgraded the change data capture engine (Debezium) to the latest stable version 1.3. Previously Etlworks was using Debezium 1.2.
Oracle CDC connector now supports native LogMiner adapter
Affected areas: CDC flows, Oracle CDC connector
Debezium ingests change events from Oracle using the XStream API or directly by LogMiner.
In order to use the XStream API, you need to have a license for the GoldenGate product (though it is not required that GoldenGate itself is installed).
LogMiner is a built-in Oracle replication API that doesn't require an additional license to use.
New option to stop the long-running CDC flow
Affected areas: CDC flows, CDC connectors
Prior to this release, it was possible to configure the CDC connector to automatically stop pulling the data if there are no new CDC events after a configurable number of retries or a configurable number of minutes.
In this release, we added a new option to stop the CDC flow after the configurable number of minutes even if there are new CDC events coming from the database.
This option can be used to unconditionally shut down a long-running CDC flow.
2. Support for SQL Server Change Data Tracking (CT)
Affected areas: data replication, change data tracking
SQL Server Change Tracking, also known as CT, is a lightweight tracking mechanism, introduced the first time in SQL Server 2008, that can be used to track the DML changes performed in SQL Server database tables. SQL Change Tracking can be configured in all SQL Server editions, including the free Express edition.
In this update, we added the ability to use change data tracking for data replication.
3. The ability to use JavaScript to calculate high watermark value (HWM)
Affected areas: data replication, change replication using high watermark
Prior to this update, it was possible to use SQL to calculate the HWM value.
In this update, we added the ability to use JavaScript as well.
4. Flow parameters when executing flow manually
Affected areas: manually executing flows
Run flow dialog now allows adding Additional parameters (flow and global variables) for on-demand executions. Additional parameters remembered per flow, so you don't have to type them in every time
5. Improvements for the scheduler
Flow Parameters
Affected areas: scheduling flows
User suggestion: Allow setting flow variables in scheduler
It is now possible to configure the flow parameters that will be passed to the flow at run time. The parameters will be added as global and flow variables.
The ability to configure flows that shouldn't be running at the time of schedule getting triggered
Affected areas: scheduling flows
User suggestion: Scheduler queue
Non-listener schedules now have 'No running flows' list, which allows selecting a list of flows that shouldn't be running at the time of that schedule getting triggered
The ability to stop flow automatically if it is running longer than expected
Affected areas: scheduling flows
User suggestion: Max runtime value for scheduled job
If the flow is running longer than expected it is now possible to stop it automatically after a configurable period of time.
The ability to disable and enable the schedule using API
Affected areas: scheduling flows, schedules management API
6. The ability to search parameters and exceptions in the audit-trail log
Affected areas: audit-trail
User suggestion: Add the ability to search within audit log JSON
We added two new fields to the Audit Trail filter: Parameters and Exception.
7. HubSpot connector now supports OAuth2 authentication
Affected areas: HubSpot connector
We added the ability to 'Sign in with HubSpot' which simplified the authentication for HubSpot connector.
Usability and UX improvements
Flow statistics dashboard
Icons and labels
- History list now shows the schedule icon and name of the schedule that triggered the execution instead of always saying executed by 'scheduler'.
- On-demand executions will show user as always.
- Agent executions will show agent icon and name.
Same as in history list, you will now see the real cause of the execution with different icons and direct links to the flow itself and trigger schedule, or agent
Flows
Listener flows now clearly marked with listener icon as in other places
Nested flows now have a dropdown that shows all child flows and icons that show if that flow has defined condition or loop.
User suggestion: Add a nested flows icon to flow list
Schedule icon changed from bright green to filled in with check mark to reduce the amount of attention and to differentiate from successful execution icon and listener icon.
Edit (pencil icon) has been removed (on all other screens as well). Run flow icon will not be shown for listener flows (instead of being disabled).
Flows export and import icons were put into a single dropdown to be consistent with connections.
Flows export moved to its own modal dialog, same as in connections for performance reasons and consistency.
Schedules
Edit (pencil icon) and statistics removed from schedules list controls, since they are available from the list itself
Listener schedules in the list now show listener icon and 'next execution' is set to 'LIVE' instead of just saying Listener and Listening
Statistics
Statistics daily executions list now shows proper names of the trigger (Started by) and those are direct links as well
and can be extended to full screen.
Audit-trail
Audit trail records that had HTTP exception with no other information now will include that HTTP error response body
The duration column value was changed from long unreadable milliseconds to hh:mm:ss.mil format. Example: 853354 ms -> 00:14:13.354
Other UX improvements
Bottom controls in all editors now have a top border to clearly separate them from the rest of the screen
Rare login screen infinite loop blinking has been fixed
Breadcrumbs on all screens were updated. Each part of the breadcrumbs is a proper link now, so it is possible to use right-click and open on a new tab.
Comments
0 comments
Please sign in to leave a comment.