About Server storage
Server Storage is file storage on any local storage device (hard drive, SSD, etc.) or a network device (NAS) attached to or mapped on the host machine running the Etlworks Integrator.
If you install the Etlworks Integrator on-premise, you can access files on any available network or local storage device. Otherwise, you can only access files in the Home folder and all subfolders of the Home folder.
When you sign up for the Etlworks Integrator, we create a storage volume in our servers that is exclusively available to your account. In this documentation, we will be referencing it as your Home folder.
The home folder and all subfolders under this can be accessed using the Server Storage Connection type. The home folder can be referenced by using the token
When to use this connector
Use this connector to create Flows that work with files in the server storage.
Create a Connection
Step 1. In the
Connections window, click
+ and type in
Step 2. Select
Step 3. Enter Connection parameters.
Directory: the directory(s) containing the files. This parameter can include subfolders. Example:
/inbound/data. The parameter is optional. Note that
Directoryfield is disabled for some of the Etlworks Integrator account types.
Files: the actual file name or a wildcard file name, for example,
*.csv. If the
Directoryfield is disabled, you can specify the subfolders in the
/inbound/data/*.csv. In this case, the connection will be looking for files under
Add Suffix When Creating Files in Transformation: you can select one of the predefined suffixes for the files created using this Connection. For example, if you select
uuidas a suffix and the original file name is
dest.csv, the Etlworks Integrator will create files with the name
dest_uuid.csv, where uuid is a globally unique identifier, such as
Enable Wildcard File Name: if there are multiple source files in a folder, you can specify an algorithm that will be used to select the next source file to be processed. For example, if
Filesis set to
oldestis selected, the Etlworks Integrator will always select the oldest file in the folder which matches the wildcard
Override Wildcard File Name set for Connection: if
Wildcard File Namesare allowed,
Override Wildcard File Name set for Connectionis enabled, and the file name entered in the
FROMfield of the transformation is a wildcard file name, the system will override the file name entered at the Connection level. The default behavior is to use the wildcard file name entered when the Connection is configured.
Contains CDC events:When this parameter is enabled, the Etlworks Integrator adds standard wildcard templates for CDC files to the list of available sources in the FROM selector.
When Server Storage Connection is used as a destination (
TO) in the source-to-destination transformation, it is possible to configure the automatic encryption of the file created by the transformation using the PGP algorithm and public key uploaded to the secure key storage.
If the public key is available, all files created by the transformation will be automatically encrypted using the PGP algorithm and given key.
Additionally, you can enable (default) or disable property
ASCII Armored. Encrypted files may be saved in a Format referred to as
ASCII Armored. This Format is an encrypted representation of a file consisting entirely of printable ASCII (or text-mode only) characters. Files in this Format contain no binary values, and therefore may be easily sent as part of e-mail messages and visually examined. Files saved in this Format are approximately 30% larger than their non-armored counterparts. When decrypted, both armored and non-armored files reproduce an identical original.
When Server Storage Connection is used as a source (
FROM) in the source-to-destination transformation, it is possible to configure the automatic decryption of the encrypted source files using the PGP algorithm and private key uploaded to the secure key storage.
If the private key is available, all source files processed by the transformation will be automatically decrypted using the PGP algorithm and given key. Note that the private key requires a password.