Conftest is a tool that lets you write tests against structure data like Kubernetes templates.
So, why should you use Conftest when you already established your policies with Azure Policy for Kubernetes?
As Azure Policy for Kubernetes uses Gatekeeper the OPA implementation for Kubernetes under the hood it uses Gatekeeper constraint templates written in Rego. Tests written for Conftest are also written in Rego. Therefore, you reuse the Rego part of the Gatekeeper constraint template for your Conftest test.
Providing Conftest tests to your developers makes live for them much easier. They may know how to write the Kubernetes templates to comply with all policies in place. But this might not be a guarantee for a successful deployment.
Without Conftest tests your developers need to check the replica set when a deployment fails ensuring a policy violation is the root cause for that.
This is cumbersome and not straight forward. And here are coming Conftest tests into play. Your developers include those tests into the application’s deployment pipeline and ensure that the Kubernetes template complies with the policies in place before deploying it to the Azure Kubernetes Service cluster.
Write Conftest tests
After that long introduction let us write a Conftest test. For instance, I have a custom policy deployed to my AKS cluster via Azure Policy for Kubernetes which checks that pods have disabled the automount of the service account token when the service account is the default one.
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: k8sdisableautomountserviceaccounttoken
spec:
crd:
spec:
names:
kind: K8sDisableAutomountServiceAccountToken
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8sdisableautomountserviceaccounttoken
missing(obj) = true {
not obj.automountServiceAccountToken == true
not obj.automountServiceAccountToken == false
obj.serviceAccount == "default"
}
check(obj) = true {
obj.automountServiceAccountToken
obj.serviceAccount == "default"
}
violation[{"msg": msg}] {
p := input_pod[_]
missing(p.spec)
msg := sprintf("automountServiceAccountToken field is missing for pod %v while using Service Account %v", [p.metadata.name, p.spec.serviceAccount])
}
violation[{"msg": msg, "details": {}}] {
p := input_pod[_]
check(p.spec)
msg := sprintf("Service Account token automount is not allowed for pod %v while using Service Account %v, spec.automountServiceAccountToken: %v", [p.metadata.name, p.spec.serviceAccount, p.spec.automountServiceAccountToken])
}
input_pod[p] {
p := input.review.object
}
-> https://github.com/neumanndaniel/kubernetes/blob/master/conftest/constraint-template.yaml
Most policies you write targeting pods as otherwise such policies written for deployments could be bypassed easily using a replica set or pod template for the deployment.
Our Conftest test needs to support deployments and cron jobs beside pods. Therefore, our test has three different input options covering all three kinds of Kubernetes objects.
input_pod[p] {
input.kind == "Deployment"
p := input.spec.template
}
input_pod[p] {
input.kind == "CronJob"
p := input.spec.jobTemplate.spec.template
}
input_pod[p] {
input.kind == "Pod"
p := input
}
The two violations we check for are mostly identical with the one from the Gatekeeper constraint template.
violation[{"msg": msg}] {
p := input_pod[_]
missing(p.spec)
msg := sprintf("automountServiceAccountToken field is missing for %v %v while using Service Account default", [input.kind, input.metadata.name])
}
violation[{"msg": msg, "details": {}}] {
p := input_pod[_]
check(p.spec)
msg := sprintf("Service Account token automount is not allowed for %v %v while using Service Account default, spec.automountServiceAccountToken: %v", [input.kind, input.metadata.name, p.spec.automountServiceAccountToken])
}
Minor adjustments to the message parts and the variables are required supporting the three kinds of Kubernetes objects mentioned before.
During a deployment Kubernetes automatically adds the serviceAccount field to the pod object with the value default if not defined otherwise. Hence, only one check and missing function is required for the Gatekeeper constraint template. For the Conftest test two additional check and missing functions are needed to check for a missing serviceAccount field in the Kubernetes templates we want to validate with Conftest.
missing(obj) {
not obj.automountServiceAccountToken == true
not obj.automountServiceAccountToken == false
missingServiceAccount(obj, "serviceAccount")
}
check(obj) {
obj.automountServiceAccountToken
missingServiceAccount(obj, "serviceAccount")
}
Those two functions calling the missingServiceAccount functions to determine a missing serviceAccount field.
missingServiceAccount(obj, field) {
not obj[field]
}
missingServiceAccount(obj, field) {
obj[field] == ""
}
And here is the full Conftest test.
package main
missingServiceAccount(obj, field) {
not obj[field]
}
missingServiceAccount(obj, field) {
obj[field] == ""
}
missing(obj) {
not obj.automountServiceAccountToken == true
not obj.automountServiceAccountToken == false
missingServiceAccount(obj, "serviceAccount")
}
missing(obj) {
not obj.automountServiceAccountToken == true
not obj.automountServiceAccountToken == false
obj.serviceAccount == "default"
}
check(obj) {
obj.automountServiceAccountToken
missingServiceAccount(obj, "serviceAccount")
}
check(obj) {
obj.automountServiceAccountToken
obj.serviceAccount == "default"
}
violation[{"msg": msg}] {
p := input_pod[_]
missing(p.spec)
msg := sprintf("automountServiceAccountToken field is missing for %v %v while using Service Account default", [input.kind, input.metadata.name])
}
violation[{"msg": msg, "details": {}}] {
p := input_pod[_]
check(p.spec)
msg := sprintf("Service Account token automount is not allowed for %v %v while using Service Account default, spec.automountServiceAccountToken: %v", [input.kind, input.metadata.name, p.spec.automountServiceAccountToken])
}
input_pod[p] {
input.kind == "Deployment"
p := input.spec.template
}
input_pod[p] {
input.kind == "CronJob"
p := input.spec.jobTemplate.spec.template
}
input_pod[p] {
input.kind == "Pod"
p := input
}
-> https://github.com/neumanndaniel/kubernetes/blob/master/conftest/test.rego
Use Conftest tests
Conftest tests are normally placed into a subfolder policy in your current working directory. But you can use the –policy option providing the path to where you store your tests at a central location.
Assuming we placed the test into the policy subfolder the following command tests our Kubernetes templates.
> conftest test <path to template>
I prepared three different Kubernetes templates all using the default service account with different settings violating or complying with the policy.
> conftest test deployment.yaml cronjob.yaml pod.yaml FAIL - cronjob.yaml - main - Service Account token automount is not allowed for CronJob go-webapp while using Service Account default, spec.automountServiceAccountToken: true FAIL - pod.yaml - main - automountServiceAccountToken field is missing for Pod go-webapp while using Service Account default 6 tests, 4 passed, 0 warnings, 2 failures, 0 exceptions
As seen in the test results the pod template misses the automountServiceAccountToken field, the cron job template has set the automountServiceAccountToken field to true, and the deployment template complies with the policy.
Summary
Conftest tests help you validating your Kubernetes templates against your Azure Policy for Kubernetes policies before the actual deployment happens to your AKS cluster. Especially for your developers those tests are tremendously helpful.
For more details about Conftest look into the documentation.
As always you find the Gatekeeper constraint template I am using as custom policy within Azure Policy for Kubernetes and the corresponding Conftest test in my GitHub repository.
-> https://github.com/neumanndaniel/kubernetes/tree/master/conftest