When to use this connector
Use this connector to create Flows that work with files in Google Drive.
Create a Connection
To work with Google Drive, you will need to create a Connection. In the Connections window, click +, and type in google drive in the search field. Select the Google Drive Connection from the gallery.
Authentication
To connect to specific Google service, you will either be using the OAuth2 authentication (default) or the authentication with a Google Service Account.
OAuth2 authentication
To connect to the Google service using OAuth2 protocol, you will need to select OAuth2 as an Authentication type and click Sign in with Google. Read more about OAuth2 authentication with Google Services.
Authentication using Google Service account
To connect to the Google service using the service account, select the Service Account as an Authentication type, then share the folder with the service account email. The default service account email is etl-framework@api-project-201080624425.iam.gserviceaccount.com.
Important: To share a folder with a service account email, click SHARE, add the service account email to the list, and click Done.
Parameters for authenticating with Google Service Account
- Service Account Email: the drive must be shared with the service account email.
- Service Account: You can create your own service account in the Google API console, authorize it in the Google Analytics Admin and add it to this Connection. If nothing is entered in this field, the default account, linked to the email etl-framework@api-project-201080624425.iam.gserviceaccount.com will be used.
Connection parameters
- Shared Folder ID: a unique ID for the folder. For example, if the URL used to access a foldrer like the following https://drive.google.com/drive/u/0/folders/1DdCZgMuvn6bbIpzT6RIPmBZ-ZWZ8uZxd, the Folder ID is going to be 1DdCZgMuvn6bbIpzT6RIPmBZ-ZWZ8uZxd.
- Support MyDrive and Shared Drive: if this option is enabled (default) the connector supports MyDrives and Shared drives. When it is disabled it only supports MyDrives. Most of the drives are configured as MyDrives so you can try disabling it for more efficiency. Revert back to 'enabled' if the connector doesn't show any files in Explorer.
- Directory under shared folder: the actual directory with files. This parameter is optional.
- Files: the actual file name or a wildcard file name, for example, *.csv.
- Add Suffix When Creating Files in Transformation: you can select one of the predefined suffixes for the files created using this Connection. For example, if you select uuid as a suffix and the original file name is dest.csv, Etlworks will create files with the name dest_uuid.csv, where uuid is a globally unique identifier such as 21EC2020-3AEA-4069-A2DD-08002B30309D.
Note: This parameter works only when the file is created usingsource-to-destination-transformation. Read how to add a suffix to the files created when copying, moving, renaming, and ziping files.
-
File Processing Order: Specifies the order in which source files are processed when using wildcard patterns in ETL and file-based flows (e.g., copy, move, delete). The default setting is Oldest, meaning files are processed starting with the oldest by creation or modification time. Choose from various criteria such as file age, size, or name to determine the processing sequence:
- Disabled: wildcard processing is disabled,
- Oldest/Newest: Process files based on their creation or modification time, Ascending/Descending: Process files in alphabetical order, Largest/Smallest: Process files based on their size.
Decryption
When Google Drive Connection is used as a source (FROM) in the source-to-destination transformation, it is possible to configure the automatic decryption of the encrypted source files using the PGP algorithm and private key uploaded to the secure key storage.
If the private key is available, all source files processed by the transformation will be automatically decrypted using the PGP algorithm and given key. Note that the private key requires a password.
Read how to generate a pair of public/private keys.
Expected Compression
The Expected Compression setting allows you to specify the compression format expected when reading individual files from a connection. Supported options include No Compression, Zip, and GZip.
If set to Zip or GZip, Etlworks will automatically decompress each resource as it’s read. This setting assumes that each compressed file contains a single resource; it does not support archives with multiple files. Use this setting with caution, as the system will always attempt to decompress the resource based on the selected format, regardless of its file extension or actual content.