Articles in this section

See more

Using Credential Manager APIs with Connections in Etlworks

  • Updated
Follow

Benefits of Using Credential Managers

Credential managers offer a secure and automated way to handle sensitive information such as usernames, passwords, API keys, and tokens, which are required to authenticate and authorize access to various services. They provide a centralized platform for securely storing, managing, and retrieving credentials, reducing the risks associated with hardcoding secrets or manually updating them in various applications. Here are the key benefits of using credential managers:

  • Enhanced Security

  • Simplified Secret Management

  • Reduced Human Error

  • Improved Compliance

  • Ease of Secret Rotation

  • Integration with Multiple Platforms

  • Scalability and Flexibility

Credential managers provide a, secure, and scalable way to manage sensitive information across applications. By using a credential manager, you can ensure that credentials are securely stored, regularly rotated, and dynamically retrieved without manual intervention. This reduces human error, enhances security, improves compliance, and simplifies credential management across complex environments.

Working with Credential Managers in Etlworks

In Etlworks, secure and dynamic management of credentials is essential when connecting to various services such as AWS Redshift, S3, RDS, and others. To streamline this process, all connections in Etlworks come with a feature: the Script to run before connect parameter. This feature allows users to enter JavaScript code to dynamically retrieve and set credentials using external credential management services like AWS Credentials Manager, STS, and others.

By leveraging this parameter, you can securely retrieve the necessary credentials at runtime instead of hardcoding them into the connection configuration. This ensures that your credentials are always up-to-date and securely managed.

Pre-installed Packages for AWS Security Manager and STS

Etlworks comes pre-configured with all the necessary packages required to integrate with AWS Security Manager and AWS STS. This allows you to easily retrieve credentials using AWS’s services without any additional setup.

If you are using other security management services and need specific packages or libraries, Etlworks supports adding these by request. For assistance with adding custom packages, contact support@etlworks.com.

How to Use “Script to Run Before Connect” for Credential Management

Accessing the Parameter

Each connection in Etlworks has a Script to run before connect field, where you can insert custom JavaScript code. This script runs every time the connection is established, allowing you to dynamically retrieve credentials.

Using AWS STS (Example with Redshift)

Below is an example of using AWS STS (Security Token Service) to assume a role and retrieve temporary credentials to access a Redshift cluster. This example demonstrates how to set up the script to securely connect to Redshift without hardcoding credentials:

var Region = Java.type('software.amazon.awssdk.regions.Region');
var RedshiftClient = 
Java.type('software.amazon.awssdk.services.redshift.RedshiftClient');
var GetClusterCredentialsRequest = 
Java.type('software.amazon.awssdk.services.redshift.model.GetClusterCredentialsRequest');
var StsClient = Java.type('software.amazon.awssdk.services.sts.StsClient');
var AssumeRoleRequest = 
Java.type('software.amazon.awssdk.services.sts.model.AssumeRoleRequest');
var StsAssumeRoleCredentialsProvider = 
Java.type('software.amazon.awssdk.services.sts.auth.StsAssumeRoleCredentialsProvider');

// Initialize STS client with a specific region
var stsClient = StsClient.builder().region(Region.US_EAST_1).build();
try {
   // Assume a role with the required permissions to access Redshift
   var assumeRoleRequest = AssumeRoleRequest.builder()
     .roleSessionName("etlworks")
     .roleArn("arn:aws:iam::111111111111:role/etlworks-redshift")
     .build();
   var stsAssumeRoleCredentialsProvider = StsAssumeRoleCredentialsProvider.builder()
     .stsClient(stsClient)
     .refreshRequest(assumeRoleRequest)
     .build();

   // Create Redshift client with temporary credentials
   var rsClient = RedshiftClient.builder().region(Region.US_EAST_1)
      .credentialsProvider(stsAssumeRoleCredentialsProvider).build();
   try {
      // Get cluster credentials
      var request = GetClusterCredentialsRequest.builder()
        .clusterIdentifier("redshift-warehouse-id")
        .dbName("prodwh")
        .dbUser("etlworks")
        .build();

      var response = rsClient.getClusterCredentials(request);

      // Set the retrieved credentials for the connection
      alias.setUserId("IAM:etlworks");
      alias.setPassword(response.dbPassword());
   } finally {
       rsClient.close();
   }
} finally {
   stsClient.close();
}

Explanation of the STS Example

STS Client: The script begins by setting up an STS Client using a specific region. This client will be used to assume an IAM role that has access to the Redshift cluster.

Assume Role: The script assumes the specified IAM role (roleArn) to obtain temporary credentials. These credentials are used to authenticate against Redshift.

Redshift Client: A Redshift client is created with the temporary credentials. Using this client, the script requests credentials for the specified Redshift cluster and database user.

Set Credentials: The retrieved credentials (user ID and password) are dynamically set for the connection using the alias.setUserId and alias.setPassword methods.

Using AWS Security Manager

var Region = Java.type('software.amazon.awssdk.regions.Region');
var SecretsManagerClient = 
Java.type('software.amazon.awssdk.services.secretsmanager.SecretsManagerClient');
var GetSecretValueRequest = 
Java.type('software.amazon.awssdk.services.secretsmanager.model.GetSecretValueRequest');
var ObjectMapper = Java.type('com.fasterxml.jackson.databind.ObjectMapper');

// Initialize Secrets Manager client
var secretsClient = SecretsManagerClient.builder().region(Region.US_EAST_1).build();

try {
   // Retrieve secret value from AWS Secrets Manager
   var secretName = "arn:aws:secretsmanager:us-east-1:111111111111:secret:redshift-credentials";
   var getSecretValueRequest = GetSecretValueRequest.builder().secretId(secretName).build();
   var secretResponse = secretsClient.getSecretValue(getSecretValueRequest);

   // Parse the secret JSON response to extract username and password
   var secretString = secretResponse.secretString();
   var objectMapper = new ObjectMapper();
   var secretMap = objectMapper.readValue(secretString, Java.type("java.util.Map"));

   var username = secretMap.get("username");
   var password = secretMap.get("password");

   // Set the retrieved credentials for the connection
   alias.setUserId(username);
   alias.setPassword(password);
} finally {
   secretsClient.close();
}

Explanation of the AWS Security Manager Example

AWS Secrets Manager Client: The script uses SecretsManagerClient to connect to AWS Secrets Manager and retrieve the secret, which contains the credentials (username and password) for Redshift.

Get Secret Value: The GetSecretValueRequest is used to fetch the secret value, which contains the credentials in JSON format.

Parse Secret JSON: The secret value is parsed using ObjectMapper from Jackson to extract the username and password from the JSON string.

Set Credentials: The retrieved username and password are set dynamically using the alias.setUserId and alias.setPassword methods.

This script dynamically retrieves credentials from AWS Secrets Manager, making it more secure and flexible, as credentials no longer need to be hardcoded or stored in configuration files.

Setting the Script in the Etlworks UI

The provided screenshot shows where the script is entered in the Etlworks UI:

Script to Run Before Connect: In the connection settings, you can paste the JavaScript code into this field. Every time the connection is initialized, the script will be executed, and the credentials will be retrieved dynamically.

Related articles