Running Devcontainers in the Cloud

running-devcontainers-cloud.md

Running DevContainers in the Cloud

A research document detailing approaches to run Development Containers remotely on AWS and GCP, plus Kubernetes/EKS-based ephemeral workspaces.

---

Table of Contents

1. Introduction
2. AWS EC2-Based DevContainer Workspaces
3. GCP Compute Engine VM Workspaces
4. Feature Comparison & Trade‑offs
5. Best Practices
6. Recommendations & Next Steps
7. AWS EKS Ephemeral DevContainer Workspaces

---

Introduction

• DevContainers: Open specification for container-based development environments.
  Spec reference: https://containers.dev/spec/
• Why cloud?
  • On‑demand compute resources
  • Consistent environments
  • Team collaboration and resource sharing

---

AWS EC2-Based DevContainer Workspaces

Architecture

graph LR
  IDE[VS Code IDE]
  EC2[EC2 Instance]
  IDE -->|Remote SSH| EC2
  EC2 -->|Docker| DevContainer
  DevContainer -->|Git Clone| CodeRepo[(GitHub repo)]
  EC2 -->|VPC| Internet[(Internet)]

Loading

Prerequisites & IAM

• AWS CLI & aws configure (https://docs.aws.amazon.com/cli/)
• IAM Role with:
  • EC2: AmazonEC2FullAccess
  • SSM: AmazonSSMManagedInstanceCore
  • ECR: AmazonEC2ContainerRegistryReadOnly

Provisioning Steps

1. Launch EC2 (Amazon Linux 2) AMI.
2. Provide user data to install Docker & VS Code Server.
3. Configure Security Group:
   • SSH (22)
   • HTTPS (443) if using code server TLS
4. Connect via Remote SSH in VS Code.

Networking Setup

• VPC subnet (public IP) or private IP + NAT
• Bastion host pattern for secure access
• Security Groups & NACLs

Cost & Auto‑Shutdown

• AWS Instance Scheduler solution:
  https://aws.amazon.com/solutions/instance-scheduler/
• Lambda or SSM Automation to shut down idle instances

Security Considerations

• Session Manager: Avoid opening SSH port
• Patch Management: AWS SSM Patch Manager
• Encryption: EBS volumes with KMS

---

GCP Compute Engine VM Workspaces

Architecture

graph LR
  IDE[VS Code IDE]
  GCE[Compute Engine VM]
  IDE -->|Remote SSH| GCE
  GCE -->|Docker| DevContainer
  DevContainer -->|Cloud Storage| GCS[(GCS bucket)]
  GCE -->|VPC| Internet[(Internet)]

Loading

Prerequisites & IAM

• Google Cloud SDK & gcloud auth login
• Service Account with:
  • Compute Admin
  • Storage Admin

Provisioning Guide

1. gcloud compute instances create <name> \ --metadata-from-file startup-script=startup.sh
2. Firewall rules: allow TCP 22 (SSH)
3. Use gcsfuse to mount buckets if needed

Networking Setup

• VPC network & subnet configuration
• Cloud NAT for outbound
• Private Google Access for APIs

Cost & Auto‑Stop

• Preemptible VMs or auto‑stop on idle:
  https://cloud.google.com/compute/docs/instances/preemptible

Security Considerations

• OS Login for SSH via IAM
• Shielded VM
• VPC Service Controls
• Disk encryption with CMEK

---

Feature Comparison & Trade‑offs

Criterion	AWS EC2	GCP VM	AWS EKS Ephemeral
Startup Time	~1–2 min	~1–2 min	~30–60 sec
Cost Model	$/hour + EBS	$/hour + PD	Pod-based
Networking	VPC/Security Groups	VPC/Firewall Rules	Kubernetes CNI
IAM Integration	AWS IAM roles	GCP Service Accounts	IRSA + K8s RBAC
Auto‑shutdown	Scheduler/Lambda	Preemptible/Auto‑stop	Pod TTL

---

Best Practices

• Container Registry: Use private ECR/GCR
• Image Caching: Layered Docker builds
• Least‑Privilege IAM
• Shutdown Policies: Tag resources & enforce TTL

---

Recommendations & Next Steps

• Pilot: AWS EC2 approach for initial rollout
• Metrics:
  • Cold start vs. warm start time
  • Cost per user‑hour
  • Security audit outcomes
• Documentation: Onboarding playbook

---

AWS EKS Ephemeral DevContainer Workspaces

Architecture

graph LR
  IDE[VS Code IDE]
  EKS[EKS Cluster]
  IDE -->|Remote Kubernetes| EKS
  subgraph Namespace “workspace”
    Pod[Ephemeral Pod<br>(DevContainer image)]
    PVC[Ephemeral PVC]
    Pod -->|Mount| PVC
  end
  Pod -->|Git Clone| CodeRepo[(GitHub Repo)]
  EKS -->|Ingress/ALB| Internet

Loading

Prerequisites & IAM

• Provision EKS via eksctl or Terraform
• IAM Role for ServiceAccount with:
  • ECR access
  • SSM for session audit
  • SecretsManager

DevWorkspace CRD Field Details

apiVersion: workspace.devfile.io/v1
kind: DevWorkspace
metadata:
  name: dev-${USER}
  namespace: ${USER}
spec:
  template:
    components:
      - container:
          name: tools
          image: myregistry/devcontainer:latest
          mountSources: true
          env:
            - name: NODE_ENV
              value: development
          endpoints:
            - name: vscode
              targetPort: 8443
              exposure: public
      - volume:
          name: workspace-storage
          size: 5Gi
    commands:
      - id: init-project
        exec:
          component: tools
          commandLine: npm install

• metadata.name/namespace: unique per user
• template.components.container: image, mounts, env, endpoints
• volume: ephemeral storage
• commands: init, build, test steps

Provisioning Steps

1. Deploy DevWorkspace Operator:
   kubectl apply -f https://github.com/devfile/devworkspace-operator/releases/latest/download/install.yaml
2. Create namespace per user
3. Apply DevWorkspace CR instance (see above)
4. VS Code: Remote – Kubernetes attach via endpoint

Networking Setup

• AWS VPC CNI plugin for pod networking
• ALB Ingress Controller + TLS certificate
• NetworkPolicy per namespace

Cost & Auto‑Scaling

• Cluster Autoscaler on node groups or Fargate
• Pod TTL Controller to delete idle workspaces

Security Considerations

• Kubernetes RBAC scoped to namespaces
• Pod Security Standards (no privileged)
• Image signature verification (Cosign)
• Encrypted PVC via AWS KMS (EBS CSI driver)