Articles in this section

See more

SharePoint storage connector

  • Updated
Follow

Overview

The SharePoint Storage Connector allows you to work with files stored in Microsoft SharePoint document libraries. It is designed for file-based operations such as reading, writing, copying, moving, and deleting files, and for using SharePoint as a source or destination in ETL and file workflows.

This connector focuses exclusively on files. If you need to access SharePoint Lists, Groups, Attachments, or Users, use the SharePoint API (Premium) connector instead.

When to Use This Connector

Use the SharePoint Storage Connector when your integration requires file-level access to SharePoint, including:

  • Processing files from SharePoint in ETL flows

  • Writing or uploading files to SharePoint document libraries

  • Copying, moving, renaming, or deleting files

  • Working with wildcard file patterns (for example, *.csv)

  • Automatically decrypting, decompressing, or archiving files

Creating a SharePoint Storage Connection

Step 1. Add a New Connection

  1. Open the Connections screen.

  2. Click + Create.

  3. Search for SharePoint.

  4. Select SharePoint Storage.

Step 2. Configure Connection Parameters

Required Parameters

Optional Parameters

  • SharePoint Site

    Optional site path.

    Example: MySite

  • Directory

    The folder path within the document library that contains files.

    Default: Shared Documents

  • Document Library Name

    Explicitly specifies the document library to use (for example, Documents, Shared Documents, Dokumente). If not set, Etlworks attempts to detect the library automatically.

    Recommended when:

    • The site has multiple document libraries

    • Automatic detection fails

Step 3. Authentication

OAuth Authentication

This method uses delegated permissions and acts on behalf of the signed-in user.

  • Works with SharePoint Online / Office 365

  • Required when 2FA or SSO is enabled for the Microsoft account

  • Access must be granted on behalf of a specific user

mceclip2.png

Steps:

  1. Select OAuth as the authentication method.

  2. Click Sign in with Microsoft.

  3. Complete the Microsoft authentication flow.

  4. Adjust permissions if needed.

mceclip3.png

Client Secret Authentication (Service Principal / App-Only)

Recommended for background jobs, automation, and service integrations.

This method uses application permissions and does not require user login.

Use this method when:

  • No interactive login is possible

  • The integration runs as a service

  • Tenant-wide or application-level permissions are required

Steps:

  1. Select Client secret as the Authentication method.
  2. Enter:

    • Client ID (Application ID from Azure App Registration)

    • Client Secret

    • Tenant ID

Ensure the Azure App Registration has:

    • Application permissions (not Delegated)

    • Admin consent granted

Username and Password Authentication

  • Supported only for environments where Microsoft still allows it

  • Disabled by Microsoft for most modern SharePoint environments

mceclip1.png

Note: OAuth or Client Secret authentication is strongly recommended.

File-Related Connection Options

Add Suffix When Creating Files in Transformation

Automatically appends a suffix to files created by source-to-destination transformations.

Example:

  • Original file: output.csv

  • Suffix: uuid

  • Result: output_uuid.csv

This option applies only to transformation output files.

For copy, move, rename, or archive operations, use file-management options instead.

File Processing Order

Controls the order in which files are processed when using wildcards.

Available options:

  • Disabled – wildcard processing disabled

  • Oldest / Newest – based on modification or creation time

  • Ascending / Descending – alphabetical order

  • Largest / Smallest – file size

Default: Oldest

Delete Behavior: Soft Delete vs Permanent Delete

By default, file deletion in SharePoint uses soft delete.

Soft Delete (Default)

  • Deleted files are moved to the SharePoint recycle bin

  • Files can be restored from SharePoint

  • Safer and recommended for most use cases

Permanent Delete (Optional)

Enable Permanent Delete to bypass the recycle bin:

  • Files are immediately and permanently deleted

  • Files cannot be restored

  • Use with caution

Decryption (PGP)

When used as a source in a source-to-destination transformation, the connector can automatically decrypt files encrypted with PGP.

  • Requires a private key stored in secure key storage

  • Private key password is required

  • All source files are decrypted automatically before processing

mceclip1.png

Read how to generate a pair of public/private keys.

Compression and Archiving

Expected Compression (Automatic Decompression)

Use Expected Compression when reading compressed files.

Supported values:

  • No Compression

  • Zip

  • GZip

If set to Zip or GZip:

  • Etlworks automatically decompresses the file as it is read

  • Each compressed file must contain a single resource

  • The system always attempts decompression regardless of file extension

Learn more about automatic decompression.

Automatic Archiving When Writing or Copying Files

Etlworks also supports automatic file archiving when writing or copying files.

  • Output files can be automatically archived as ZIP or GZIP

  • Applies to ETL operations and file copy workflows

This functionality is configured using Archive output files options.

Learn more about automatic archiving.

Related articles