Mage AI on Kubernetes/EKS using Helm

Uncategorized

Here is the consolidated Mage AI deployment process on Kubernetes/EKS using Helm, including the exact volume issue you hit and the working fix.

Mage AI on Kubernetes/EKS using Helm

Mage provides an official Helm chart through the mageai Helm repository. The official Helm docs show the basic flow: add the repo, update it, search the chart, and install mageai/mageai. They also document using extraVolumes and extraVolumeMounts for project storage, especially with PVC-backed storage on cloud Kubernetes. (docs.mage.ai)


1. Add Mage AI Helm repository

helm repo add mageai https://mage-ai.github.io/helm-charts
helm repo update

Check available charts:

helm search repo mageai

Example output:

NAME                  CHART VERSION   APP VERSION   DESCRIPTION
mageai/mageai         0.2.14          0.9.79        A Helm chart for Mage AI
mageai/mageai-ingress 0.0.1                         A Helm chart for deploying ingress for Mage AI

2. Create a dedicated namespace

Avoid installing directly into default.

kubectl create namespace mageai-dev

3. Important issue with default install

If you run this directly:

helm install mageai-dev mageai/mageai -n mageai-dev

the pod may fail with:

CreateContainerError

In your case, the exact error was:

failed to mkdir "/path/to/mage_project": mkdir /path: read-only file system

Why this happened

The chart was trying to use a placeholder/project path:

/path/to/mage_project

That path is not valid for your EKS setup. Kubernetes tried to create it, but the filesystem was read-only from the container runtime perspective.

So the image was fine:

Successfully pulled image "mageai/mageai:0.9.79"

The problem was volume/project path configuration, not image pull, not Helm repo, and not EKS networking.


4. Working test install using emptyDir

For testing, create this values file:

cat > mageai-dev-values.yaml <<'EOF'
service:
  type: ClusterIP

config:
  USER_CODE_PATH: /home/src/default_repo

extraVolumeMounts:
  - name: mage-fs
    mountPath: /home/src

extraVolumes:
  - name: mage-fs
    emptyDir: {}
EOF

Install fresh:

helm install mageai-dev mageai/mageai -n mageai-dev \
  -f mageai-dev-values.yaml

Or if release already exists:

helm upgrade mageai-dev mageai/mageai -n mageai-dev \
  -f mageai-dev-values.yaml

5. Verify deployment

kubectl get pods -n mageai-dev

Expected:

NAME                      READY   STATUS    RESTARTS   AGE
mageai-xxxxxxxxxx-xxxxx   1/1     Running   0          1m

Check service:

kubectl get svc -n mageai-dev

Expected:

NAME     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)
mageai   ClusterIP   172.20.xxx.xxx   <none>        6789/TCP

This means Mage AI is internal-only.


6. Access Mage AI from your laptop

Run this from your laptop terminal:

kubectl port-forward svc/mageai 6789:6789 -n mageai-dev

Keep that terminal open.

Then open:

http://localhost:6789

If port 6789 is already used on your laptop:

kubectl port-forward svc/mageai 6790:6789 -n mageai-dev

Then open:

http://localhost:6790

Complete clean install flow

Use this if starting from zero:

helm repo add mageai https://mage-ai.github.io/helm-charts
helm repo update

kubectl create namespace mageai-dev

cat > mageai-dev-values.yaml <<'EOF'
service:
  type: ClusterIP

config:
  USER_CODE_PATH: /home/src/default_repo

extraVolumeMounts:
  - name: mage-fs
    mountPath: /home/src

extraVolumes:
  - name: mage-fs
    emptyDir: {}
EOF

helm install mageai-dev mageai/mageai -n mageai-dev \
  -f mageai-dev-values.yaml

kubectl get pods -n mageai-dev
kubectl get svc -n mageai-dev

kubectl port-forward svc/mageai 6789:6789 -n mageai-dev

Open:

http://localhost:6789

Cleanup commands

Remove old failed/default install:

helm uninstall my-mageai -n default

Remove current dev install:

helm uninstall mageai-dev -n mageai-dev

Delete namespace:

kubectl delete namespace mageai-dev

Check all Mage resources:

helm list -A | grep mage
kubectl get all -A | grep mage
kubectl get svc -A | grep mage

Production storage note

The working fix used:

emptyDir: {}

That is fine for testing, but not production.

emptyDir is temporary pod storage. If the pod is deleted or rescheduled, Mage project files can disappear.

For production on EKS, use persistent storage such as:

AWS EFS → Kubernetes StorageClass → PVC → /home/src

Mage’s Helm docs show the production-style pattern of mounting project storage to /home/src using extraVolumeMounts and extraVolumes, with a PVC such as an EFS-backed ReadWriteMany volume. (docs.mage.ai)

Production-style values example:

service:
  type: ClusterIP

config:
  USER_CODE_PATH: /home/src/default_repo

extraVolumeMounts:
  - name: mage-fs
    mountPath: /home/src

extraVolumes:
  - name: mage-fs
    persistentVolumeClaim:
      claimName: mageai-efs-pvc

Then deploy:

helm upgrade --install mageai-dev mageai/mageai -n mageai-dev \
  -f mageai-prod-values.yaml

Final architecture from your working setup

Laptop Browser
    ↓
http://localhost:6789
    ↓
kubectl port-forward
    ↓
Kubernetes Service: mageai
Type: ClusterIP
Port: 6789
    ↓
Mage AI Pod
Container Port: 6789
    ↓
Mounted project path:
/home/src/default_repo

Your current successful setup is:

Mage AI app       ✅ Running
Service type      ✅ ClusterIP / internal-only
Access method     ✅ kubectl port-forward
Port              ✅ 6789
Storage           ⚠️ emptyDir / testing only
Volume issue      ✅ Fixed

Yes. Run these cleanup commands.

1. Stop port-forward

In the terminal where port-forward is running:

Ctrl + C

2. Check existing Mage Helm releases

helm list -A | grep mage

You may see:

my-mageai     default
mageai-dev    mageai-dev

3. Uninstall Mage releases

Remove old install from default:

helm uninstall my-mageai -n default

Remove fresh dev install:

helm uninstall mageai-dev -n mageai-dev

4. Delete Mage namespace

kubectl delete namespace mageai-dev

5. Verify cleanup

helm list -A | grep mage
kubectl get all -A | grep mage
kubectl get svc -A | grep mage

Expected: no output.

6. Confirm AWS LoadBalancer is removed

Because earlier installs created AWS LoadBalancers, check:

kubectl get svc -A | grep LoadBalancer

If no Mage service appears, Kubernetes has requested AWS to delete the ELB.

You can also check from AWS CLI:

aws elbv2 describe-load-balancers --region ap-northeast-1 \
  --query "LoadBalancers[?contains(LoadBalancerName, 'mage')].[LoadBalancerName,DNSName,State.Code]" \
  --output table

Full cleanup block

helm uninstall my-mageai -n default
helm uninstall mageai-dev -n mageai-dev

kubectl delete namespace mageai-dev

helm list -A | grep mage
kubectl get all -A | grep mage
kubectl get svc -A | grep mage
kubectl get svc -A | grep LoadBalancer

If helm uninstall my-mageai -n default says release not found, that is fine — it was already removed.

0 0 votes
Article Rating
Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x