Install Etlworks on cloud Kubernetes: Amazon EKS, Azure AKS and Google GKE

Etlworks managed releases 10.0.14 and newer include installation profiles for Amazon EKS, Azure AKS and Google GKE Standard. This article covers the cloud-specific procedure: preparing the provider cluster, freezing a cloud plan, configuring and installing with that plan, network access and cloud troubleshooting.

Everything the cloud installation shares with the local profile — downloads, Cluster licensing, archive verification, the base configuration, first access, scaling, logs, backups and recovery — is in Install Etlworks on Kubernetes: downloads, cluster preparation, installation and recovery. Read that guide first; this article extends it.

In this article

• Supported cloud platforms and scope
• Check your release before starting
• Cloud platform profiles
• Requirements common to every cloud profile
• Freeze the target cluster and native add-ons into a cloud plan
• Amazon EKS
• Azure AKS
• Google GKE Standard
• Configure and install using the cloud plan
• Network access to the installed application
• Cloud storage identity, holds and retries
• Backups and recovery on cloud profiles
• Cloud troubleshooting

Supported cloud platforms and scope

The cloud profiles are fully tested and certified on Amazon EKS, Azure AKS and Google GKE Standard, and ship in the managed release archive for both Linux AMD64 and ARM64. The local profile remains available for self-managed clusters.

The cloud profiles use ordinary Linux node pools only. EKS Auto Mode and Fargate, AKS node auto-provisioning, GKE Autopilot, Windows nodes and Spot/preemptible nodes are excluded. The installer does not activate a node autoscaler, and nonempty-node consolidation and maintenance must respect the drain requirements in the main guide's node-scaling section.

Check your release before starting

The cloud profile modules ship inside the managed installer ZIP starting with Etlworks 10.0.14. If your downloaded release predates 10.0.14, these modules are absent from its archive. Obtain the matching release and architecture; do not edit an older release's descriptor, copy modules between releases or add an invented --cloud installation flag. The documentation and inventory bundled with your release are authoritative for what that release contains.

Cloud platform profiles

Select the profile with the platformProfile setting in the installation configuration. The profile selection is independent of the CPU/RAM resource profile named by your Cluster license: the license controls pod sizing and counts, the platform profile controls where and how the installation runs. Website and license-portal downloads do not require a provider selection.

Requirements common to every cloud profile

Bring an existing supported cluster and its explicit kubeconfig/context. The installer:

• Reuses the provider's native CNI and CSI. It never installs Calico over a cloud provider's network, creates a cloud cluster or VPC, enables cloud APIs, changes IAM or quotas, or enables a node autoscaler.
• Installs or reuses its pinned cert-manager and KEDA assets. Do not run a second KEDA stack or another HPA on the Etlworks workload.
• Creates namespaced application resources plus retained CSI StorageClasses and PersistentVolumeClaims. The namespace must be new and exclusive to this installation.

On the installation workstation you need Java 25, Python 3.12 or newer, Helm 3, kubectl and your provider's CLI, authenticated to the intended account, subscription or project with a scoped platform identity. Cloud credentials stay on the installer host: do not put cloud credentials in the Etlworks runtime Secrets or application properties.

Cluster admission rules must allow the installer's bounded storage-preparation Jobs and the member identity init container to run as UID 0 with only the CHOWN, FOWNER and DAC_OVERRIDE capabilities, without privileged mode, host mounts or API tokens. A globally enforced restricted Pod Security policy needs a reviewed exception for these helpers; the installer does not disable such a policy or grant blanket privileges.

Freeze the target cluster and native add-ons into a cloud plan

Before installation, run the plan writer shipped with the release. It reads the target cluster and writes a local cloud plan containing the provider resource ID, Kubernetes API endpoint, the kube-system namespace UID, explicit storage/network settings and the observed native add-on image references. It does not provision anything.

First list the actual native workloads, so the plan references real resources rather than conventional names:

kubectl --kubeconfig /absolute/path/kubeconfig --context CONTEXT \
  -n kube-system get daemonsets,deployments

Use actual kind/namespace/name references for the CNI and both CSI integrations, including visible node and controller components. If your platform uses custom add-on installation names, those replace the conventional names shown in the examples below. The plan writer requires a new output filename — it does not overwrite an existing plan. Keep the plan outside the verified release directory.

Amazon EKS

Prepare an EKS 1.34 cluster with ordinary EC2 managed node groups, network policy enabled in the VPC CNI add-on, EBS CSI, EFS CSI and the AWS Load Balancer Controller. Start with fixed nodes; do not use Auto Mode, Fargate or Spot. Prepare an encrypted EFS filesystem with private mount targets, security groups and a reviewed IAM policy permitting only the installation's access points. Native CSI/controller identities need their own narrow permissions. The installer identity must be able to read the named EKS cluster and the VPC-CNI add-on, as well as perform the Kubernetes installation.

Write a shared-parameters file using your EFS filesystem ID (the value below is an example):

{"fileSystemId":"fs-0123456789abcdef0"}

From the extracted release root, freeze the plan. Account, region, resource IDs, context, subnets and the source CIDR below are all examples — substitute your actual values. Choose subnets appropriate to the internal or public scheme and a real permitted client range:

python3 deployment/cloud-profiles/prepare_plan.py \
  --profile aws-eks-v1 --namespace etlworks \
  --kubeconfig /absolute/etlworks-private/eks.kubeconfig --context EKS_CONTEXT \
  --resource-id arn:aws:eks:us-east-1:123456789012:cluster/etlworks-test \
  --location us-east-1 --architecture amd64 \
  --shared-parameters ../etlworks-private/aws-shared.json \
  --source-range 203.0.113.10/32 --scheme internet-facing \
  --subnet subnet-0123456789abcdef0 --subnet subnet-0123456789abcdef1 \
  --addon daemonset/kube-system/aws-node \
  --addon deployment/kube-system/ebs-csi-controller \
  --addon daemonset/kube-system/ebs-csi-node \
  --addon deployment/kube-system/efs-csi-controller \
  --addon daemonset/kube-system/efs-csi-node \
  --addon deployment/kube-system/aws-load-balancer-controller \
  --output ../etlworks-private/cloud-plan.json

For ARM64 node groups use --architecture arm64 and the ARM64 installer. The default plan scheme is internal unless explicitly changed.

Installation creates a Service named CLUSTER_NAME-public: an NLB-backed TCP 443 Service to HAProxy using IP targets. Point application DNS at the advertised endpoint and establish frontend certificate trust separately. The Service does not expose lifecycle control ports.

Azure AKS

Prepare an AKS cluster with Linux VMSS node pools, enforced network policy, the managed Files and Disk CSI drivers and a Standard Load Balancer. This profile uses manually configured ordinary node pools, not node auto-provisioning. Prepare the storage account and network for Premium Azure Files NFS with encryption in transit, and use an eligible supported Linux node image. Important: do not disable encryption in transit to work around a mount failure. Subscription provider registration, identity/RBAC and sufficient regional/SKU quotas are platform tasks.

Write a reviewed shared-parameters file (example values):

{"storageAccount":"YOUR_STORAGE_ACCOUNT","resourceGroup":"YOUR_STORAGE_RESOURCE_GROUP"}

Set the add-on variables below to actual workload references from your cluster's kube-system inventory — they are not fixed Azure add-on names:

python3 deployment/cloud-profiles/prepare_plan.py \
  --profile azure-aks-v1 --namespace etlworks \
  --kubeconfig /absolute/etlworks-private/aks.kubeconfig --context AKS_CONTEXT \
  --resource-id /subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP/providers/Microsoft.ContainerService/managedClusters/CLUSTER_NAME \
  --location eastus --architecture amd64 \
  --shared-parameters ../etlworks-private/azure-shared.json \
  --source-range 10.0.0.0/8 --scheme internal \
  --addon "$AKS_CNI_REF" \
  --addon "$AKS_FILES_NODE_REF" \
  --addon "$AKS_DISK_NODE_REF" \
  --output ../etlworks-private/cloud-plan.json

Add other visible native controller references with repeated --addon. Use your actual Azure location and a real permitted client CIDR. Azure shared and backup claims default to at least 100 GiB each. The cloud Service sends TCP 443 through the Standard Load Balancer to HAProxy; DNS and trusted frontend TLS remain your responsibility.

Google GKE Standard

Prepare a GKE Standard cluster with ordinary Linux node pools, enforced network policy, the Filestore and PD CSI drivers, the required Google APIs and private service access. GKE/Filestore API enablement, IAM, the kubectl authentication plugin and quotas are platform tasks. Do not use Autopilot or preemptible/Spot nodes with this profile.

Write the shared-parameters file using your actual private network:

{"network":"projects/YOUR_PROJECT/global/networks/YOUR_PRIVATE_NETWORK"}

Set the add-on variables to actual inventory references:

python3 deployment/cloud-profiles/prepare_plan.py \
  --profile gcp-gke-standard-v1 --namespace etlworks \
  --kubeconfig /absolute/etlworks-private/gke.kubeconfig --context GKE_CONTEXT \
  --resource-id projects/YOUR_PROJECT/locations/us-central1/clusters/CLUSTER_NAME \
  --location us-central1 --architecture amd64 \
  --shared-parameters ../etlworks-private/gcp-shared.json \
  --source-range 10.0.0.0/8 --scheme internal \
  --addon "$GKE_CNI_REF" \
  --addon "$GKE_FILESTORE_NODE_REF" \
  --addon "$GKE_PD_NODE_REF" \
  --output ../etlworks-private/cloud-plan.json

A zonal cluster needs its actual zone in both the resource ID and --location. Add other visible native controllers with repeated --addon.

Important: the initial Basic Filestore defaults request 1 TiB each for application data and the separately mounted backup destination. Review quota and cost before installation. The mounted backup destination is not an off-cluster backup strategy. GKE identity and provisioned Redis disks request at least 10 GiB; these are storage minima, separate from licensed application RAM.

Storage security: this baseline uses private Filestore and provider network encryption. It does not provide NFS protocol-level encryption or NFS client authentication. If your security policy requires those properties, do not use this baseline; arrange a separately qualified storage configuration. The installer does not silently introduce Active Directory or Kerberos.

The GKE profile creates an internal or external passthrough TCP 443 load-balancer Service to HAProxy, according to the selected scheme.

Configure and install using the cloud plan

Keep the ordinary signed setup, hostname, initial administrator and explicit kubeconfig/context fields described in the main guide's configuration section. Replace the local storage object and add the selected platform profile. For AWS, for example:

"platformProfile": "aws-eks-v1",
"storage": {"cloudPlan": "/absolute/etlworks-private/cloud-plan.json"}

For AKS or GKE, use that profile's exact ID. Do not combine cloudPlan with workerA, workerB or retainedPlan. Use the same namespace and architecture as the frozen plan. Select PostgreSQL and Redis modes independently; for external services, follow the external-service requirements and the Secret checkpoint procedure in the main guide.

Run the usual release commands:

sh deployment/installer/etlworks-kubernetes.sh preflight \
  --config ../etlworks-private/install.json --json
sh deployment/installer/etlworks-kubernetes.sh install \
  --config ../etlworks-private/install.json --wait 30m --json
sh deployment/installer/etlworks-kubernetes.sh status \
  --config ../etlworks-private/install.json --json
sh deployment/installer/etlworks-kubernetes.sh resume \
  --config ../etlworks-private/install.json --wait 30m --json

Preflight reads the provider and Kubernetes identities, version and architecture, CNI policy, native CSI and the ready add-on image references. It does not change cloud account settings. Installation creates retained CSI classes and claims and bounded consumer Jobs; block volumes use placement-aware binding.

Network access to the installed application

Each cloud profile creates a Layer 4 TCP 443 Service to HAProxy, named CLUSTER_NAME-public, through the provider's load balancer: an NLB on EKS, the Standard Load Balancer on AKS and a passthrough load balancer on GKE. The scheme (internal or internet-facing/external) comes from the frozen plan; the default is internal.

Set the application DNS name to the Service's advertised endpoint and establish frontend certificate trust separately. Creating the Service does not configure DNS or issue a public certificate, and it does not expose installer or lifecycle control ports. The source range recorded in the plan restricts which client networks the load balancer admits.

Cloud storage identity, holds and retries

Private identity claims are exclusive to one pod. A member's logical identity and recorded volume binding survive normal CSI device-name changes; arbitrary replacement volumes are not adopted. The installer prepares fresh volume roots only — its short-lived helpers do not format or recursively change ownership of populated data. Known missing or changed volume identities cause a hold.

When storage is pending, inspect CSI events, PVC/PV bindings and the exact consumer Job, then resume the same operation. After fixing a failed helper's underlying cause, an explicit resume allows one new attempt per storage role for that invocation, with a maximum of eight retained attempts. Retry requires exact evidence that the previous worker terminated; a timeout or an unreachable possible writer is not that evidence. Failed Jobs remain available for inspection. Do not delete known claims, change context or rename the installation to bypass a hold.

Retained disks, access points and load balancers need an explicit cleanup plan after testing; deleting a pod is not infrastructure teardown.

Backups and recovery on cloud profiles

Backups and restore follow the main guide's backup verification and storage monitoring section. On cloud profiles, recovery additionally checks the recorded CSI storage identities. A same-installation restore is not an automatic clean-cluster disaster rebuild, and it is not a provider or service-mode conversion. A cloud-mounted backup claim is staging, not an independent recovery copy — copy the complete archive to an independent storage failure domain.

Cloud troubleshooting

Related articles

• Install Etlworks on Kubernetes: downloads, cluster preparation, installation and recovery
• Running Etlworks on-premise: installation, upgrade, maintenance
• Managing premium connectors