The HTTP API connector in Etlworks supports a wide range of authentication methods used across modern APIs, including basic auth, token-based schemes, AWS Signature, and user-defined headers. This guide explains each supported method and how to configure it.

Supported Authentication Types

• Basic Authentication

• Basic Preemptive Authentication

• API Key Authentication

• Token-based (OAuth2) Authentication

• Browser-based OAuth2 (Used in Pre-configured Connectors)

• OAuth1 (One-step OAuth)

• AWS Signature Authentication

• Header-based Authentication

• SOAP-style Authentication

• JWT assertion

• Custom Authentication using Preprocessor

Basic Authentication

• Select basic in the Authentication field.

• Enter the username and password.

Credentials will be passed in the Authorization header using standard base64 encoding.

Basic Preemptive Authentication

• Select basic preemptive in the Authentication field.

• Enter the username and password.

Preemptive mode sends the credentials without waiting for a 401 challenge.

API Key Authentication

• Select none in the Authentication field.

• Enter the API key in the Password field.

• Use {password} token in the URL or headers to insert the key.

Example:

https://api.example.com/data?apiKey={password}

OAuth2 (Two-step Token-based Authentication)

OAuth2 is a widely used standard for token-based authentication. It allows secure access to APIs without passing usernames and passwords with every request.

How OAuth2 Works

Step 1: Request an Access Token

Etlworks sends a request to the Authentication URL using the provided User (client ID) and Password (client secret). The token endpoint typically returns a JSON object that includes the access token.

Step 2: Use the Token

The token is extracted from the response and automatically added to the Authorization header (or a custom header) in all subsequent API calls.

Configuration Options

• Authentication: Select token or oauth2.

• Authentication URL: The token endpoint (e.g., https://example.com/oauth/token).

• User: Client ID.

• Password: Client Secret.

• HTTP Method for Token Authentication: Usually POST.

• Access Token Attribute: JSON node containing the token (e.g., access_token or result.accessToken).

• Access Token Prefix: Prefix added to the token in the header (e.g., Bearer). Use none to omit the prefix.

• Authentication Request Payload: Optional JSON body for the token request. Use {user}, {password}, or {refresh_token} tokens.

• Authentication Request Headers: Optional custom headers.

• Authentication Header Name: Header to receive the token (default: Authorization).

• Authentication Request Content Type: Usually application/json or application/x-www-form-urlencoded.

Browser-based OAuth2 (Used in Pre-configured Connectors)

Browser-based OAuth2 uses the authorization code flow, which requires a user to interactively log in and grant permission.

For browser-based OAuth2, use a Pre-configured API Connectors instead of configuring OAuth2 manually.

OAuth1 (One-step OAuth)

• Select oauth1 in the Authentication field.

• Fill in:

  • User → Consumer Key

  • Password → Consumer Secret

  • Access Token

  • Access Secret

Optionally, select the OAuth1 Signature Method:

• HMAC-SHA1 (default, legacy)

• HMAC-SHA256 (recommended if supported by the API)

If required by the API, the OAuth1 realm can be provided by adding it explicitly as an Authorization header.

IMPORTANT: The realm value is not calculated and does not affect the signature.

AWS Signature Authentication (AWS v4)

• Select aws in the Authentication field.

• Enter Access Key in the User field.

• Enter Secret Key in the Password field.

• Optionally set:

  • AWS Region (default: us-east-1)

  • Service Name (e.g., s3, iam). If not set, Etlworks will attempt to extract it from the URL.

Header-based Authentication

Use this method when the API expects a non-standard authorization header.

• Select header in the Authentication field.

• Set:

  • Authentication Header Name: Custom header name (default: Authorization).

  • Content of the Authorization header: Token or header content (use {user}, {password} if needed).

• Use single quotes for string values: 'Bearer {password}' instead of double quotes.

SOAP-style Authentication

SOAP APIs often require signed payloads with tokens embedded in the XML.

The Preprocessor can be used to dynamically compute and set global variables, which are referenced in the SOAP envelope using {tokens}.

Step 1. Set a SOAP envelope with tokens in Payload

Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:ns1="http://www.vendor.com/soapendpoint/" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <SOAP-ENV:Header>
    <ns1:AuthenticationHeader>
      <vendorUserId>{requestUserId}</mktowsUserId>
      <vendorSignature>{requestSignature}</requestSignature>
      <vendorTimestamp>{requestTimestamp}</requestTimestamp>
    </ns1:AuthenticationHeader>
  </SOAP-ENV:Header>
  <SOAP-ENV:Body>
    <ns1:paramsGetMultipleCars>
      <leadSelector xsi:type="ns1:KeySelector">
        <keyType>MAKE</keyType>
        <keyValues>
          <stringItem>Ford</stringItem>
          <stringItem>Acura</stringItem>
        </keyValues>
      </leadSelector>
      <batchSize>100</batchSize>
    </ns1:paramsGetMultipleCars>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Step 2. Use Preprocessor to dynamically set values for tokens

Here is an example:

var javaImports = new JavaImporter(java.text, java.util, javax.crypto, 
javax.crypto.spec, org.apache.commons.codec.binary, com.toolsverse.config);

with (javaImports) {
   var props = SystemConfig.instance().getProperties();
   var vendorUserId = "user";
   var vendorSecretKey = "password";

   var df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ssZ");
   var text = df.format(new java.util.Date());
   var requestTimestamp = text.substring(0, 22) + ":" + text.substring(22);

   var encryptString = requestTimestamp + vendorUserId;
   var secretKey = new SecretKeySpec(vendorSecretKey.getBytes(), "HmacSHA1");
   var mac = Mac.getInstance("HmacSHA1");
   mac.init(secretKey);
   var rawHmac = mac.doFinal(encryptString.getBytes());
   var signature = new java.lang.String(org.apache.commons.codec.binary.Hex.encodeHex(rawHmac));

   props.put("requestSignature", signature);
   props.put("requestTimestamp", requestTimestamp);
   props.put("requestUserId", vendorUserId);
}

JWT Assertion Authentication

JWT Assertion Authentication is an OAuth 2.0 authentication mechanism where the client creates a signed JSON Web Token (JWT) and exchanges it for an access token. The signed JWT proves the identity of the application without sending the private key over the network. The JWT is sent to a token endpoint, the server validates the signature using the public key associated with the application, and if the assertion is valid the server returns a short-lived access token. Etlworks automatically generates the JWT, signs it using the configured private key, exchanges it for an access token, and then uses that token when executing API requests.

To configure JWT Assertion Authentication in an HTTP Connection, set Authentication to jwt assertion and configure the following parameters.

User or AccessKey

This is typically the OAuth client ID or application identifier provided by the API provider. In many APIs this value is also used as the JWT issuer. In the token request payload it is usually referenced as {user}.

Password or SecretKey

This is the OAuth client secret associated with the application. It is used when exchanging the JWT assertion for an access token. In the token request payload it can be referenced as {password}.

Authentication URL

The OAuth token endpoint where the signed JWT is exchanged for an access token. Etlworks sends the authentication request to this endpoint.

HTTP Method for Token and OAuth2 Authentication

The HTTP method used when requesting the token. Most APIs require POST.

Authentication Request Content Type

The content type of the authentication request payload. For OAuth token requests this is usually application/x-www-form-urlencoded.

Authentication Request Headers

Optional headers that should be sent when requesting the token. Many APIs do not require additional headers.

Authentication Request Payload

The body of the token request. This payload typically includes the grant type, client credentials, and the generated JWT assertion. Etlworks replaces placeholders such as {user}, {password}, and {jwt_assertion} at runtime.

Example payload:

grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&
client_id={user}&
client_secret={password}&
assertion={jwt_assertion}

JWT Issuer

The identifier of the entity that issued the JWT. This is usually the client ID of the application.

JWT Subject

The subject on whose behalf the JWT is issued. Depending on the API this may be a user ID, service account, or enterprise identifier.

JWT Audience

The intended recipient of the JWT. This is typically the OAuth token endpoint URL.

JWT Public Key ID

Identifier of the public key registered with the API provider. The server uses this value to locate the correct public key when verifying the JWT signature.

JWT Private Key (PEM)

The private key in PEM format used to sign the JWT. The corresponding public key must be registered with the API provider.

JWT Private Key Passphrase

If the private key is encrypted, provide the passphrase used to unlock it before signing the JWT.

JWT Expiration (seconds)

Optional expiration time for the generated JWT. If not specified, a default short-lived expiration is used.

JWT Header Claims

Optional custom claims added to the JWT header.

JWT Claims

Additional claims included in the JWT payload. Some APIs require vendor-specific claims.

Example: Box API

The Box platform supports JWT assertion authentication for server-to-server integrations. The following example shows how to configure an HTTP connection for the Box API.

Authentication parameters:

• Authentication: jwt assertion

• User or AccessKey: Box Client ID

• Password or SecretKey: Box Client Secret

Token acquisition parameters:

• Authentication URL:

  https://api.box.com/oauth2/token

• HTTP Method:

  POST

• Content Type:

  application/x-www-form-urlencoded

• Authentication Request Payload:

grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&
client_id={user}&
client_secret={password}&
assertion={jwt_assertion}

JWT assertion parameters:

• JWT Issuer: Box Client ID

• JWT Subject: Box User ID or Enterprise ID

• JWT Audience:

  https://api.box.com/oauth2/token

• JWT Public Key ID: Public key ID generated in the Box developer console

• JWT Private Key (PEM): Private key downloaded from the Box application configuration

• JWT Private Key Passphrase: Passphrase for the private key

• JWT Claims:

box_sub_type = user

With this configuration, Etlworks automatically generates the JWT assertion, signs it using the configured private key, exchanges it with the Box OAuth endpoint for an access token, and uses the returned token when executing Box API requests.

Custom Authentication Using Preprocessor

Use the Preprocessor to dynamically generate tokens or modify connection parameters before the request is sent. 

The {tokens} can be referenced in URL and headers when configuring connection.