Server Storage connector

About Server storage

Server Storage is file storage on any local storage device (hard drive, SSD, etc.) or a network device (NAS) attached to or mapped on the host machine running the Etlworks Integrator.

If you install the Etlworks Integrator on-premise, you can access files on any available network or local storage device. Otherwise, you can only access files in the Home folder and all subfolders of the Home folder.

Home folder

When you sign up for the Etlworks Integrator, we create a storage volume in our servers that is exclusively available to your account. In this documentation, we will be referencing it as your Home folder.

The home folder and all subfolders under this can be accessed using the Server Storage Connection type. The home folder can be referenced by using the token {app.data}.

When to use this connector

Use this connector to create Flows that work with files in the server storage.

Create a Connection

Step 1. In the Connections window, click + and type in server storage.

Step 2. Select Server Storage.

Step 3. Enter Connection parameters.

Connection parameters

• Directory: the directory(s) containing the files. This parameter can include subfolders. Example: /inbound/data. The parameter is optional. Note that Directory field is disabled for some of the Etlworks Integrator account types.
• Files: the actual file name or a wildcard file name, for example, *.csv. If the Directory field is disabled, you can specify the subfolders in the Files field. Example: /inbound/data/*.csv. In this case, the connection will be looking for files under home/inbound/data/.
• Add Suffix When Creating Files in Transformation: you can select one of the predefined suffixes for the files created using this Connection. For example, if you select uuid as a suffix and the original file name is dest.csv, the Etlworks Integrator will create files with the name dest_uuid.csv, where uuid is a globally unique identifier, such as 21EC2020-3AEA-4069-A2DD-08002B30309D.

This parameter works only when the file is created using source-to-destination-transformation. Read how to add a suffix to the files created when copying, moving, renaming, and zipping files.

• Enable Wildcard File Name: if there are multiple source files in a folder, you can specify an algorithm that will be used to select the next source file to be processed. For example, if Files is set to *.csv and oldest is selected, the Etlworks Integrator will always select the oldest file in the folder which matches the wildcard *.csv.
• Override Wildcard File Name set for Connection: if Wildcard File Names are allowed, Override Wildcard File Name set for Connection is enabled, and the file name entered in the FROM field of the transformation is a wildcard file name, the system will override the file name entered at the Connection level. The default behavior is to use the wildcard file name entered when the Connection is configured.

• Contains CDC events:When this parameter is enabled, the Etlworks Integrator adds standard wildcard templates for CDC files to the list of available sources in the FROM selector.

Encryption

When Server Storage Connection is used as a destination (TO) in the source-to-destination transformation, it is possible to configure the automatic encryption of the file created by the transformation using the PGP algorithm and public key uploaded to the secure key storage.

If the public key is available, all files created by the transformation will be automatically encrypted using the PGP algorithm and given key.  

Additionally, you can enable (default) or disable propertyASCII Armored. Encrypted files may be saved in a Format referred to asASCII Armored. This Format is an encrypted representation of a file consisting entirely of printable ASCII (or text-mode only) characters. Files in this Format contain no binary values, and therefore may be easily sent as part of e-mail messages and visually examined. Files saved in this Format are approximately 30% larger than their non-armored counterparts. When decrypted, both armored and non-armored files reproduce an identical original.

Read how to generate a pair of public/private keys.

Decryption

When Server Storage Connection is used as a source (FROM) in the source-to-destination transformation, it is possible to configure the automatic decryption of the encrypted source files using the PGP algorithm and private key uploaded to the secure key storage.

If the private key is available, all source files processed by the transformation will be automatically decrypted using the PGP algorithm and given key.  Note that the private key requires a password. 

Read how to generate a pair of public/private keys.