Packaging, GitOps and operators

A Kubernetes service is more than one Deployment. It is a set of manifests for each environment, plus the settings that differ between them. Packaging tools keep those manifests consistent. GitOps keeps the cluster in sync with a git repository. Operators extend the cluster with new kinds of objects that run their own runbooks.

The big idea

Think of a recipe with variations. One base recipe, with small changes for each kitchen, is easier to keep correct than separate recipes that drift apart. Kustomize and Helm are two ways to write that base recipe. GitOps is the rule that the kitchen's state must match the recipe card in version control.

Walking through it

• Kustomize is built into kubectl -k. It overlays a base with patches, such as different replica counts and ConfigMaps per environment. There is no templating language. You write YAML and get YAML out.
• Helm packages charts with Go templates, values files and a release lifecycle with install, upgrade and rollback. Most third-party software, such as ingress controllers and cert-manager, ships as a Helm chart. For your own service, teams pick Kustomize for simplicity, or Helm when they need a parameterized product chart.
• GitOps, with Argo CD or Flux, treats git as the desired state. A controller reconciles the cluster to match. You get an audit trail, rollback by git revert, and no long-lived cluster credentials in CI. CI still builds and pushes the image. GitOps syncs the manifests that pin it.

Pin images by digest in production, not by tag:

image: registry.example.com/checkout@sha256:4f1c...

A tag such as latest or v1.2.3 can be overwritten by a new push. A digest names one exact image, and it cannot change.

Operators are controllers for custom resources. A custom resource definition adds a new kind to the Kubernetes API, such as PostgresCluster or Kafka. An operator watches those objects and creates the StatefulSets, users and backups that make them real. Operators pay off when their automated runbook is well tested, as with cert-manager. They hurt when the custom resource hides failure modes you still have to understand at 3 a.m.

Running a primary database on Kubernetes is justified when you need the same control plane for many tenants, locality with the app, or regulatory constraints. Otherwise a managed database such as RDS or Cloud SQL usually means fewer pages and better backups.

Common mistakes

• Tagging images with mutable tags. Use digests in production manifests.
• Hand-editing manifests in the cluster. GitOps will revert the change, or the next apply will.
• Adopting an operator without reading its failure modes. Know what it does when a node dies.

Check yourself

1. Why pin images by digest instead of tag? A tag can be pushed again with new contents. A digest identifies exactly one image.
2. What does GitOps give you that kubectl apply from CI does not? An audit trail, rollback through git, and no long-lived cluster credentials in CI.

Key takeaways

• Kustomize overlays YAML, Helm templates it, and both work with GitOps.
• Pin production images by digest.
• Operators automate runbooks for custom resources. Understand what they automate before you rely on them.

Further learning

The cluster and infrastructure code are covered in Terraform, and the image side in containers. For the Argo CD reference, see below.