Kustomize your Kubernetes Deployments

When you start working with Kubernetes, you may get to a point where you’re shocked at how complex your YAML files have become. For a complex application consisting of different containers your YAML files will become very very long and it will become harder to change a single piece of configuration like the name of your application without breaking things. This is also known as the YAML hell.

A lot has already been written about how to work around this. Bash programmers write their own scripts and you may have already heard of the tool Helm Charts. I myself am not a very good Bash programmer and also I am not a friend of Helm Charts, because they only make the topic worse. The good news is that there is already an official solution called Kustomize. This declarative approach was originally a separate project which has become a part of Kubernetes since version 1.14. So there is no longer any reason to deal with endlessly long YAML files or Helm Charts if you just want to customize some details of your Kubernetes deployments. And you don not need to install any additional tools for this!

Note: Because of the very rapid development within the open source project Kubernetes, also good tutorials can quickly become obsolete. So be very careful about reading deployment tutorials written before May 2019!

In the following section I will give a brief an simple introduction about how to use Kustomize. You can find more details on the Kubernetes page. Also a good introduction about Kustomize can be found here.

Start as always….

Before you start with Kustomize, you need to have your original yaml files describing the resources you want to deploy into kubernetes. So you start as always with one or more long YAML files. This origin files are called the base. So you can put the origin files into a directory called /base . You write these files only once and you will never touch them for later customization!

Of course you can start your application with the origin configuration always from the /base directory

$ kubectl create -f ./my-app/base/

Customize your Application with kustomization.yaml

Now as you have defined the origin base resources you can easily customize this base by defining overlays. This is very similar to the layering of Docker containers. For each individual setup you create a new folder.

But first you need to create the file kustomization.yaml. This file simply defines all resources to be part of your base customization. In the following example my application consist of a 010-deployment.yaml and a 020-service.yaml file:

resources:
  - 010-deployment.yaml
  - 020-service.yaml

With the help of the file kustomization.yaml you can now apply your base configuration with:

$ kubectl apply --kustomize ./my-app/base

So far this results in the same outcome as using the option kubectl apply -f. OK – let’s start customizing….

Creating an Overlay

With Kustomize you can easily create an overlay with custom settings of a new application based on your base definition. For that you first create a new folder like /overlay/variant-1 and put again a kustomization.yaml file in here:

# Add a new namespace to all resources
namespace: my-custom-namespace

# The base directory
bases:
- ../../base

The kustomization.yaml file defines a new namespace for your new custom deployment and points into the base directory with the origin configuration. Now you have the following directory structure:

.
├── base
│   ├── 010-deployment.yaml
│   ├── 020-service.yaml
│   └── kustomization.yaml
└── overlay
    └── variant-1
        └── kustomization.yaml

To build your first overlay run:

$ kubectl create namespace my-custom-namespace
$ kubectl apply --kustomize ./my-app/overlay/variant-1

As you may expect this will result in the same deployment but within the new namespace ‘my-custom-namespace’.

Overwriting Settings

Now you can add any junk of yaml file into your overlay folder to overwrite settings of your base deployment. For this you just need to specify the path to a yaml file to be merged into your base deployment.

For example you want to change an environment variable to your deployment, add a new file named custom-env.yaml with the new setting:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
        - name: app
          env:
            - name: CUSTOM_ENV_VARIABLE
              value: some-value

Merge the new junk yaml file with the option patchesStategicMerge in your /overlay/variant-1/kustomization.yaml file:

# Add a new namespace to all resources
namespace: my-custom-namespace

# The base directory
bases:
- ../../base

# Patches
patchesStrategicMerge:
- custom-env.yaml

This will add the new ‘CUSTOM_ENV_VARIABLE’ with the value ‘some-value’ to your deployment of ‘my-app’.

Note: You need to specify always the full path of the resource setting you want to change.

Add New Resource Objects

Another way to customize your deployment is adding new resources. You can add additional deployment resources with the resource option in your kustomize.yaml file:

# Add a new namespace to all resources
namespace: my-custom-namespace

# The base directory
bases:
- ../../base

# Additional resources
resources:
- 090-ingress.yaml

# Patches
patchesStrategicMerge:
- custom-env.yaml

This example will add a new resource ‘090-ingress.yaml‘ into your base deployment.

That’s it! As you have seen it is very simple to use Kustomize as an overlay technique for defining resource objects in Kubernetes. With the examples above you can start to apply the concept of the build-in customization feature of kubernets. You will find more examples here.