Migrating helm setup to kustomize

KubernetesKustomizeHelm

I’ve been running a k3s cluster at home for a while now, managing everything with Helm charts and ArgoCD. It works fine but becomes a bit annoying if I want to introduce new CRD’s or constantly need to re-template stuff using my library charts.

My previous customer set everything up like this and that’s why I initially took that approach.

Why change?

My old setup had everything in a repo called homelab_helm-charts. Every app (a simple nginx deployment or a full monitoring stack) was wrapped in a Helm chart. That meant for something like my AdGuardhome instance, I had:

apps/adguardhome/
├── Chart.yaml
├── templates/
│   ├── deployment.yaml
│   ├── service.yaml
│   └── pvc.yaml
├── values.yaml
├── values-dev.yaml
└── values-prod.yaml

The templates were basically just Kubernetes resources with some heavy GO templating in them to manage everything through values files. I was just using it as a templating engine for YAML that was already almost-valid Kubernetes manifests.

The real issue showed up with the dev/prod split. I had three value files per app, and the actual differences between dev and prod were usually just an image tag and maybe a replica count. It felt like a lot of ceremony for not much payoff.

The other thing that bothered me was app discovery. Every app had to be manually listed in a values.yaml for the app-of-apps Helm chart to pick it up. -> Add a new app, remember to update the list. -> Forget to update the list, nothing deploys. Annoying.

Migrating to Kustomize

I created a new repo homelab_kube-apps with a Kustomize-based structure:

apps/
└── adguardhome/
    ├── base/
    │   ├── deployment.yaml
    │   ├── service.yaml
    │   ├── pvc.yaml
    │   └── kustomization.yaml
    └── overlays/
        ├── dev/
        │   └── kustomization.yaml
        └── prod/
            └── kustomization.yaml

The base/ folder contains plain Kubernetes manifests, so no templating and no placeholders. Just valid YAML you could kubectl apply directly. The overlays then patch whatever needs to differ between environments.

A prod overlay example:

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namePrefix: prod-
labels:
  - pairs: 
      env: production

resources:
  - ../../base
  - ingressroute.yaml

Switch to ApplicationSet

The other big win was replacing the manual app list with an ArgoCD ApplicationSet that just scans the directory structure:

---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: prod-apps
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: [email protected]:homelab/homelab_kube-apps.git
        revision: HEAD
        directories:
          - path: apps/*/overlays/prod
  template:
    metadata:
      name: 'prod-{{path[1]}}'
    spec:
      project: default
      source:
        repoURL: [email protected]:homelab/homelab_kube-apps.git
        targetRevision: HEAD
        path: '{{path}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: prod
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

Now adding a new app is just creating the folder structure. ArgoCD picks it up automatically on the next sync. No list to maintain.

The part I didn’t migrate: core components

This is where I made a deliberate decision to keep Helm. My core infrastructure components (Traefik, Cert-Manager, Longhorn, ArgoCD, …) are all upstream Helm charts where I’m just passing values to someone else’s chart.

For these, the wrapper chart pattern actually makes sense:

core-components/traefik/
├── extra-resources/
├── Chart.yaml
└── values.yaml

# core-components/traefik/Chart.yaml
---
apiVersion: v2
name: traefik
version: 1.0.2
dependencies:
  - name: traefik
    version: 39.0.6
    repository: https://helm.traefik.io/traefik

Some core components also have extra-resources (IngressRoutes, Middlewares, Vault secret, …) that the Helm chart doesn’t manage. These live in an extra-resources/ subfolder and get deployed as a second source in the ArgoCD Application:

...
sources:
  - path: core-components/traefik
    repoURL: [email protected]:homelab/homelab_kube-apps.git
    targetRevision: HEAD
    helm:
      valueFiles:
        - values.yaml
  - path: core-components/traefik/extra-resources
    repoURL: [email protected]:homelab/homelab_kube-apps.git
    targetRevision: HEAD
    directory:
      recurse: true
...

Lessons learned

  1. Kustomize patches are more explicit than value overrides, and that’s a good thing. With Helm I had to trace it back through the templates to understand what actually changed. With Kustomize patches, you’re directly editing a specific field on a specific resource. The intent is obvious.

  2. Don’t use Kustomize for upstream charts. I briefly considered converting core components to Kustomize using the helmCharts transformer, but it’s an extra layer of complexity for no real benefit. If you’re consuming an upstream Helm chart, just use Helm.

  3. The migration doesn’t have to be all-or-nothing. I ended up with a hybrid: Kustomize for my apps, Helm for core infrastructure. Both are managed by ArgoCD from the same repo. Using the right tool for each job ended up being cleaner than forcing consistency for its own sake.

The new repo is a lot easier to navigate, adding apps is faster, and I actually understand what’s deployed and where. The core components are unchanged in terms of how they work, just living in a new home.

Worth the few days of migration work.

← Back to blog