Create AWS EKS Cluster

Overview

This guide provides instructions for creating an AWS EKS (Elastic Kubernetes Service) cluster that will host your Observo Site deployment. The cluster is created using eksctl, a simple command-line utility for creating and managing EKS clusters.

Prerequisites

Software Requirements

• AWS CLI installed and configured with appropriate credentials

• eksctl installed on your local machine

  Install eksctl:

  # macOS
  brew tap weaveworks/tap
  brew install weaveworks/tap/eksctl
  
  # Linux
  curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
  sudo mv /tmp/eksctl /usr/local/bin

• kubectl installed (required for managing the cluster)

• AWS IAM permissions to create EKS clusters and related resources

AWS Permissions Required

Your AWS user/role needs permissions for:

• Creating EKS clusters

• Creating VPCs and networking components

• Creating EC2 instances (for node groups)

• Creating IAM roles and policies

• CloudWatch logging permissions

Cluster Configuration

Note: Review infrastructure requirements and sizing before creating the cluster — see the Site Sizing Calculation for VM and node sizing: Sizing

Sample EKS Cluster Configuration

Save the following configuration as cluster-create.yaml:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: observo-site-cluster  # change to a unique cluster name per customer/environment
  region: us-east-1          # AWS region to deploy the cluster (e.g. us-east-1, eu-west-1)
  version: "1.33"          # Kubernetes version (pick an EKS-supported version)

vpc:
  nat:
    gateway: Single          # Single | HighlyAvailable (Single is cheaper; HA requires more subnets/NATs)
  clusterEndpoints:
    publicAccess: true
    privateAccess: false

iam:
  withOIDC: true

addons:
  - name: aws-ebs-csi-driver  # required for dynamic EBS volumes; add other addons as needed

kubernetesNetworkConfig:
  ipFamily: ipv4
  serviceIPv4CIDR: 172.20.0.0/20  # Service CIDR; ensure it does not overlap with your VPC CIDRs

cloudWatch:
  clusterLogging:
    enableTypes: ["api", "audit"]  # enable Types: api, audit, authenticator, controllerManager, scheduler
    logRetentionInDays: 1             # retention in days; increase for compliance/auditing

managedNodeGroups:
  - name: mgr-nodes
    instanceType: t4g.xlarge # EC2 instance type. t4g.xlarge = ARM (Graviton). Use m5.xlarge for x86.
    desiredCapacity: 3        # starting node count; choose based on expected workload
    minSize: 3
    maxSize: 5               # autoscaler upper bound - set per cost/performance requirements
    volumeSize: 50           # root EBS volume size per node (GB)
    volumeType: gp3          # gp3 recommended; change to io1/io2 for high IOPS
    labels:
      role: observo-site
    tags:
      role: observo-site-node
    privateNetworking: false

availabilityZones: ["us-east-1a", "us-east-1b"]  # list AZs for multi-AZ resilience; add a third AZ for higher availability

Configuration Breakdown

Cluster Settings

• name: Your cluster name (can be customized)

• region: AWS region where the cluster will be deployed

• version: Kubernetes version (check EKS supported versions)

VPC Configuration

• nat.gateway: Single: Creates a NAT gateway for outbound internet access

• clusterEndpoints.publicAccess: true: Allows access from internet

• privateAccess: false: Disables private-only endpoint

IAM Configuration

• withOIDC: true: Enables OpenID Connect for service account integration

Addons

• aws-ebs-csi-driver: Enables EBS volume support (required for storage)

Networking

• serviceIPv4CIDR: IP range for Kubernetes services

• ipFamily: ipv4: Use IPv4 addressing

Node Group Configuration

• instanceType: EC2 instance type (t4g.xlarge uses ARM architecture)

• desiredCapacity: Target number of nodes

• minSize/maxSize: Autoscaling limits

• volumeSize: EBS volume size per node

• volumeType: EBS volume type (gp3 is recommended)

Availability Zones

• Specify which AZs to use for multi-AZ deployment

Customization Options

You can modify the configuration based on your requirements:

Change Region:

metadata:
  region: us-east-1  # Change to your preferred region

Change Instance Type:

managedNodeGroups:
  - instanceType: m5.xlarge  # Standard x86 instance

Adjust Cluster Sizing:

managedNodeGroups:
  - desiredCapacity: 5  # Increase node count
    minSize: 5
    maxSize: 10  # Allow more scaling
    volumeSize: 100  # Increase disk size

Use Specific Availability Zones:

availabilityZones: ["us-east-1a", "us-east-1b", "us-east-1c"]

Create the Cluster

1. Create the Cluster Configuration File

# Create or edit the configuration file
vi cluster-create.yaml
# Paste the configuration above and save (press 'i' to insert, paste, then ':wq' to save and quit)

2. Check if Cluster Already Exists

Before creating the cluster, verify that a cluster with the same name doesn't already exist:

aws eks list-clusters --region eu-west-1

Or check for a specific cluster name:

aws eks describe-cluster --name observo-site-cluster --region eu-west-1

If the cluster already exists, you will receive an error when trying to create it. You can either:

• Choose a different cluster name in the YAML configuration

• Delete the existing cluster first (see Cleanup section)

3. Validate Configuration

eksctl create cluster --dry-run --config-file=cluster-create.yaml

This will show what will be created without actually creating resources.

4. Create the Cluster

eksctl create cluster -f cluster-create.yaml

This command will:

• Create the EKS cluster

• Set up VPC, subnets, and NAT gateway

• Configure node groups

• Install addons (EBS CSI driver)

• Set up kubectl credentials

Expected Output:

[ℹ]  eksctl version 0.x.x
[ℹ]  using region eu-west-1
[ℹ]  setting availability zones to [eu-west-1a eu-west-1b]
[ℹ]  creating EKS cluster "observo-site-cluster" in "eu-west-1" region
...
[✓]  EKS cluster "observo-site-cluster" in "eu-west-1" region is ready

This process typically takes 15-20 minutes.

5. Verify Cluster Creation

# Check cluster status
kubectl cluster-info

# Verify nodes are ready
kubectl get nodes

# Should show something like:
# NAME                                           STATUS   ROLES    AGE   VERSION
# ip-192-168-1-100.eu-west-1.compute.internal   Ready    <none>   5m    v1.33.0

6. Configure kubectl (if needed)

aws eks update-kubeconfig --name observo-site-cluster --region eu-west-1

7. Ensure a Default StorageClass

Kubernetes should have a default StorageClass so PersistentVolumeClaims (PVCs) bind automatically.

1. Check existing storage classes and whether one is marked as default:

kubectl get storageclass

Example output when there is no default:

NAME   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
gp2    kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  2d7h

1. If no class is marked as default, set one (for example gp2):

kubectl patch storageclass gp2 -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'

1. Verify it is now the default:

kubectl get sc
# NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
# gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  2d7h

Ensure your cluster has a default StorageClass before proceeding with workloads that use PVCs.

Next Steps

Once your EKS cluster is created and verified:

1. Configure kubectl access

2. Review node capacity and adjust if needed

3. Continue with Helm-Based Deployment → to install Observo Site

4. Verify Load Balancer support for data pipeline requirements

Troubleshooting

Cluster Creation Fails

If cluster creation fails:

# Check AWS credentials
aws sts get-caller-identity

# Verify region availability
aws eks list-clusters --region eu-west-1

# Check IAM permissions

Cleanup

To delete the cluster (when no longer needed):

eksctl delete cluster -f cluster-create.yaml

Or by name:

eksctl delete cluster --name observo-site-cluster --region eu-west-1

This will delete all associated resources including VPC, NAT gateway, and node groups.

Additional Resources

• EKS User Guide

• eksctl Documentation

• EKS Best Practices