Maintenance Tasks

• Starter
• Business
• Enterprise
• On-Premise
• Add-on

Overview

Maintenance tasks are only available to the self-hosted customers and customers on Enterprise plans.

In Etlworks, it is possible to configure automatic, periodically executed agents, which perform maintenance tasks in the background. The following tasks are installed by default:

• Clean audit trail: this task deletes records in the Audit Trail table that are older than the configurable number of days. 
• Clean Flow execution statistics: this task deletes the events from the history of Flow runs that are older than the configurable number of days. 
• Clean messages: this task deletes inbound messages stored in the database that are older than the configurable number of days.  
• Aggregate flow statistics: this task aggregates flow executions date used to display Statistics dashboard. 
• Send API key age notifications: this task send email notification if API key hasn't been refreshed in more than specified days. 
• Auto enable/disable schedules: this task auto enables and disables schedules and agent schedules based on "Activate on date" and "Deactivate on date" settings. 
• Clean "running" state: this task clears flow status stuck in "running" state due to instance crash or reboot during flow execution. 
• Clean agent ping: this tasks clears agent ping records stored in the database that are older than the configurable number of days. 
• Clean agents cache files: this tasks removes agents cache files that weren't removed automatically. 
• Clean history records: this task removes old change history records. 
• Clean webhook events: this task removes webhook event stored in the database that are older than the configurable number of days. 
• Delete abandoned search index files: this task removes global search index files that weren't automatically deleted.
• Refresh Microsoft OAuth2 tokens: this task refreshes all Microsoft OAuth2 tokens in all connections to prevent them from expiring. 
• Refresh QuickBooks OAuth2 tokens: this task refreshes all QuickBooks OAuth2 tokens in all connections to prevent them from expiring. 
• Send Overage Report: this task automatically sends an overage report by email when a tenant exceeds their configured record limits.
• Clean flow log files: this tasks deleted flow execution log files than are older than a configured number of days.
• Clean app data files: This task automatically deletes files in the home folder ({app.data}) and its subfolders that are older than a configurable number of days. It is designed to help manage disk usage by removing outdated files. For details on configuring the retention period and exclusions, read more.
• Clean connection metadata files: This task automatically deletes metadata files associated with connections which are older than 7 days (configurable). Metadata files include objects (tables, views, endpoints, files) and columns associated with these objects. 

Configure maintenance tasks

To configure maintenance tasks, go to Settings/Maintenace.

Available commands

The following commands are available from the toolbar:

• Add: add a new task.
• Run: manually run a selected task.
• Delete: delete a selected task.
• Refresh: refresh the list of tasks.

The selected tasks can be configured, disabled, rescheduled, and set for manual execution only.

View history of executions

Maintenance task execution details can be viewed for the past 30 days for each individual scheduled task. Select desired task first and then click on View executions.

Clicking on the particular execution Status will reveal details as well as exception details in case task failed to properly execute.

Possible statuses are:

• Success - task successfully executed and no issues were encountered
• Warning - some of the subtasks failed during the execution
• Fail - task failed during the execution

Get execution notifications

In order to receive notification after maintenance task execution create webhook under Super Admin level and select one, or more Maintenance event types that should trigger that webhook. Webhook can be configured to just send an email, URL endpoint is not required.