Create AWS EKS Cluster

Overview

This guide provides instructions for creating an AWS EKS (Elastic Kubernetes Service) cluster that will host your Observo Site deployment. The cluster is created using eksctl, a simple command-line utility for creating and managing EKS clusters.

Prerequisites

Software Requirements

AWS CLI installed and configured with appropriate credentials

eksctl installed on your local machine

Install eksctl:

# macOS
brew tap weaveworks/tap
brew install weaveworks/tap/eksctl

# Linux
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin

kubectl installed (required for managing the cluster)
AWS IAM permissions to create EKS clusters and related resources

AWS Permissions Required

Your AWS user/role needs permissions for:

Creating EKS clusters
Creating VPCs and networking components
Creating EC2 instances (for node groups)
Creating IAM roles and policies
CloudWatch logging permissions

Cluster Configuration

Sample EKS Cluster Configuration

Save the following configuration as cluster-create.yaml:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: observo-site-cluster
  region: eu-west-1
  version: "1.33"

vpc:
  nat:
    gateway: Single
  clusterEndpoints:
    publicAccess: true
    privateAccess: false

iam:
  withOIDC: true

addons:
  - name: aws-ebs-csi-driver

kubernetesNetworkConfig:
  ipFamily: ipv4
  serviceIPv4CIDR: 172.20.0.0/16

cloudWatch:
  clusterLogging:
    enableTypes: ["api", "audit"]
    logRetentionInDays: 1

managedNodeGroups:
  - name: mgr-nodes
    instanceType: t4g.xlarge # ARM
    desiredCapacity: 3
    minSize: 3
    maxSize: 5
    volumeSize: 50
    volumeType: gp3
    labels:
      role: observo-site
    tags:
      role: observo-site-node
    privateNetworking: false

availabilityZones: ["eu-west-1a", "eu-west-1b"]

Configuration Breakdown

Cluster Settings

name: Your cluster name (can be customized)
region: AWS region where the cluster will be deployed
version: Kubernetes version (check EKS supported versions)

VPC Configuration

nat.gateway: Single: Creates a NAT gateway for outbound internet access
clusterEndpoints.publicAccess: true: Allows access from internet
privateAccess: false: Disables private-only endpoint

IAM Configuration

withOIDC: true: Enables OpenID Connect for service account integration

Addons

aws-ebs-csi-driver: Enables EBS volume support (required for storage)

Networking

serviceIPv4CIDR: IP range for Kubernetes services
ipFamily: ipv4: Use IPv4 addressing

Node Group Configuration

instanceType: EC2 instance type (t4g.xlarge uses ARM architecture)
desiredCapacity: Target number of nodes
minSize/maxSize: Autoscaling limits
volumeSize: EBS volume size per node
volumeType: EBS volume type (gp3 is recommended)

Availability Zones

Specify which AZs to use for multi-AZ deployment

Customization Options

You can modify the configuration based on your requirements:

Change Region:

metadata:
  region: us-east-1  # Change to your preferred region

Change Instance Type:

managedNodeGroups:
  - instanceType: m5.xlarge  # Standard x86 instance

Adjust Cluster Sizing:

managedNodeGroups:
  - desiredCapacity: 5  # Increase node count
    minSize: 5
    maxSize: 10  # Allow more scaling
    volumeSize: 100  # Increase disk size

Use Specific Availability Zones:

availabilityZones: ["us-east-1a", "us-east-1b", "us-east-1c"]

Create the Cluster

1. Create the Cluster Configuration File

# Create or edit the configuration file
vi cluster-create.yaml
# Paste the configuration above and save (press 'i' to insert, paste, then ':wq' to save and quit)

2. Check if Cluster Already Exists

Before creating the cluster, verify that a cluster with the same name doesn't already exist:

aws eks list-clusters --region eu-west-1

Or check for a specific cluster name:

aws eks describe-cluster --name observo-site-cluster --region eu-west-1

If the cluster already exists, you will receive an error when trying to create it. You can either:

Choose a different cluster name in the YAML configuration
Delete the existing cluster first (see Cleanup section)

3. Validate Configuration

eksctl create cluster --dry-run --config-file=cluster-create.yaml

This will show what will be created without actually creating resources.

4. Create the Cluster

eksctl create cluster -f cluster-create.yaml

This command will:

Create the EKS cluster
Set up VPC, subnets, and NAT gateway
Configure node groups
Install addons (EBS CSI driver)
Set up kubectl credentials

Expected Output:

[ℹ]  eksctl version 0.x.x
[ℹ]  using region eu-west-1
[ℹ]  setting availability zones to [eu-west-1a eu-west-1b]
[ℹ]  creating EKS cluster "observo-site-cluster" in "eu-west-1" region
...
[✓]  EKS cluster "observo-site-cluster" in "eu-west-1" region is ready

This process typically takes 15-20 minutes.

5. Verify Cluster Creation

# Check cluster status
kubectl cluster-info

# Verify nodes are ready
kubectl get nodes

# Should show something like:
# NAME                                           STATUS   ROLES    AGE   VERSION
# ip-192-168-1-100.eu-west-1.compute.internal   Ready    <none>   5m    v1.33.0

6. Configure kubectl (if needed)

aws eks update-kubeconfig --name observo-site-cluster --region eu-west-1

7. Ensure a Default StorageClass

Kubernetes should have a default StorageClass so PersistentVolumeClaims (PVCs) bind automatically.

Check existing storage classes and whether one is marked as default:

kubectl get storageclass

Example output when there is no default:

NAME   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
gp2    kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  2d7h

If no class is marked as default, set one (for example gp2):

kubectl patch storageclass gp2 -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'

Verify it is now the default:

kubectl get sc
# NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
# gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  2d7h

Ensure your cluster has a default StorageClass before proceeding with workloads that use PVCs.

Next Steps

Once your EKS cluster is created and verified:

Configure kubectl access
Review node capacity and adjust if needed
Continue with Helm-Based Deployment → to install Observo Site
Verify Load Balancer support for data pipeline requirements

Troubleshooting

Cluster Creation Fails

If cluster creation fails:

# Check AWS credentials
aws sts get-caller-identity

# Verify region availability
aws eks list-clusters --region eu-west-1

# Check IAM permissions

Cleanup

To delete the cluster (when no longer needed):

eksctl delete cluster -f cluster-create.yaml

Or by name:

eksctl delete cluster --name observo-site-cluster --region eu-west-1

This will delete all associated resources including VPC, NAT gateway, and node groups.

Additional Resources

PreviousHelm Based Deployment NextHelm Values Reference

Last updated 29 days ago

Was this helpful?