dev-resources.site
for different kinds of informations.
☸️ Kubernetes: A Convenient Variable Substitution Mechanism for Kustomize
- 1. What is Kustomize?
- 2. Why variable substitution matters
- 3. Limitations and advantages of kustomize by default
- 4. Implementing custom variable substitution
- Wrapping up
- Further reading
1. What is Kustomize?
Kustomize is one of the prominent tools alongside Helm used for managing and configuring Kubernetes deployments across multiple environments. Unlike Helm, which utilizes templates, Kustomize simplifies management by primarily focusing on plain manifests without direct comparison to Helm’s approach.
2. Why variable substitution matters
Variable substitution becomes crucial when environments are ephemeral, as in the case of feature branches, or when configuration settings like DNS entries need to be determined dynamically at deployment time. Kustomize assumes a fixed environment list (e.g., dev/staging/prod), but practical Kubernetes usage often demands more flexibility.
3. Limitations and advantages of kustomize by default
The biggest advantage of Kustomize lies in its straightforward approach to managing manifests. It allows for easy manipulation of fields such as annotations, image names, and replicas across environments. However, Kustomize is strict about variable substitution, preferring a predefined set of options which can be limiting.
To circumvent these limitations, the following official method is often employed within a kustomization.yml file, after creating an environment-specific file:
resources:
- deployment.yml
- ingress.yml
- service.yml
- sa.yml
- role.yml
- role-binding.yml
generatorOptions:
disableNameSuffixHash: true
configMapGenerator:
- name: environment-variables
envs: [rendered/api-gateway.env]
behavior: create
replacements:
- source:
kind: ConfigMap
version: v1
name: environment-variables
fieldPath: data.API_GATEWAY_URL
targets:
- select:
kind: Ingress
name: api-gateway
fieldPaths:
- spec.rules.0.host
- spec.tls.0.hosts.0
- select:
kind: Deployment
name: api-gateway
fieldPaths:
- spec.template.spec.containers.0.env.[name=API_GATEWAY_URL].value
So much for such a simple need like variable substitution. People will even find Helm appealing at this step. Let's explain why and how to find a simpler solution.
4. Implementing custom variable substitution
While Kustomize apply workflow supports basic variable substitution using tools like envsubst on top of kustomize build, more complex scenarios require custom solutions. One more refined approach involves using renvsubst, to selectively replace variables based on specific criteria (e.g., K_* environment-specific or CI_* CI-specific variables).
We could then use in our pipelines, in ascending order of complexity :
- For simple use cases :
kustomize build . | envsubst - To avoid replacing some variables, define it in manifests as
${DOLLAR}MY_VARIABLEand then:kustomize build . | DOLLAR=$ envsubst
- Use
renvsubstto choose variables with prefixes:kustomize build . | renvsubst --prefix="K_" --prefix="CI_"
But all these solutions have some drawbacks. You have to pipe the kustomize build result before applying manifests; it breaks some convenient wrapping tools features like intelligent local updates using tilt.dev.
No compromise solution using renvsubst as a transformer
For a no compromise solution using a Kustomize transformer exec plugin using renvsubst, follow these steps.
Install renvsubst: Ensure
renvsubstis installed from here.set the
KUSTOMIZE_PLUGIN_HOMEvariable for kustomize to find your plugin, for exampleKUSTOMIZE_PLUGIN_HOME=$PWD/devops/k8s/scripts/Set up Transformer Script:
Create a transformer script (renvsubst) at $KUSTOMIZE_PLUGIN_HOME/transformers/renvsubst/renvsubst:
#!/bin/sh
RENVSUBST=${RENVSUBST_PATH:-$(which renvsubst)}
if [ -z "$RENVSUBST" ]; then
echo "Error: renvsubst command not found. Please install renvsubst from https://github.com/containeroo/renvsubst/releases or set RENVSUBST_PATH." >&2
exit 1
fi
$RENVSUBST --prefix="K_" --prefix="CI_" -i -
- Configure Custom Resource:
Define a custom resource (renvsubst.transformer.yml) next to your kustomization.yml using it:
apiVersion: transformers
kind: renvsubst
metadata:
name: renvsubst
- Integrate with Kustomize:
Update your kustomization.yml to include the custom resource:
resources:
- ...
transformers:
- renvsubst.transformer.yml
By following these steps, you can enhance Kustomize's capabilities to handle dynamic environments and complex configuration needs effectively.
Wrapping up
While Kustomize offers simplicity in managing Kubernetes manifests, its default limitations in variable substitution can be overcome through strategic customization. By leveraging custom transformers and renvsubst, Kubernetes deployments with customize can become more adaptable to diverse and evolving environments, ensuring smoother and more efficient operations.
Further reading
This article was enhanced with the assistance of an AI language model to ensure clarity and accuracy in the content, as English is not my native language.
Featured ones: