In June I already covered Azure Policy for Kubernetes in a blog post.
-> https://www.danielstechblog.io/using-azure-policy-for-kubernetes/
Back then Azure Policy for AKS was in public preview. At this year’s Microsoft Ignite Azure Policy for AKS went GA.
-> https://azure.microsoft.com/en-us/updates/ga-policy-addon-for-azure-kubernetes-service/
There have been some significant changes between the public preview and the GA version.
First, let us start with what Azure Policy for AKS is.
Introduction
Azure Policy for AKS is based on the Open Policy Agent implementation called Gatekeeper. When you enable the add-on on your AKS cluster two different components get installed. So, let us have a look at the Gatekeeper component first.
Azure deploys three pods in total into the gatekeeper-system namespace. Two replicas of the gatekeeper-controller ensuring high-availability compared to the one replica during the public preview.
One replica is only needed for the gatekeeper-audit part providing the audit functionality. It does not matter which enforcement action deny, the default one, or dryrun is defined in the constraint. The gatekeeper-audit pod always checks the cluster for violations.
Only when you set the enforcement action to deny the gatekeeper-controller gets active.
Whenever your deployment fulfills all policy conditions the request is allowed, and your application gets deployed. Otherwise, Gatekeeper denies the deployment as all request are forwarded by the validating admission controller to the gatekeeper-controller for validation. Except for resources in Kubernetes namespaces, you exempt from Gatekeeper.
The second component is the one for connecting Azure Policy with you Azure Kubernetes Cluster.
Azure deploys two pods in the kube-system namespace for it. Those pods are responsible for deploying constraint templates and constraints onto your AKS cluster based on the Azure Policy assignments you made.
Furthermore, the reporting functionality back to Azure Policy for detected violations is a core component. This allows you to check the compliance state of your AKS cluster in the Azure portal. At a glance you get all necessary compliance information about your entire AKS fleet.
Changes between public preview and GA
Let us have a look at the following table what has changed between public preview and GA.
| Public Preview | GA | |
| Gatekeeper deployment |
|
|
| Policy effects |
|
|
| Available built-in policies |
|
|
Policy effects
It is important to understand the different policy effects before starting with the rollout of Azure Policy for AKS onto the clusters.
Policies with the audit effect get deployed onto the AKS cluster. So, the Azure Policy pods rolling out the constraint templates and constraints for Gatekeeper. As the name says audit does only auditing. It sets the enforcement action on the Gatekeeper constraint to dryrun. Deployments that do not comply with the assigned policy are not denied, they succeed. But the violation against the policy is reported by Gatekeeper and transferred back to Azure Policy.
The audit effect lets you quickly rollout new policies without breaking your deployments.
On the other hand, the deny effect does the same as the audit effect for already existing deployments. Constraint templates and constraints get deployed onto your cluster, but the enforcement action is now set to deny.
Existing resources are checked against the policy for compliance. New deployments or pod restarts not complying with the policies get denied and will not be deployed.
Only enable the deny effect on policies you ran in audit mode before. Thus, ensuring you are not breaking your existing application deployments.
Setting disabled as effect on a policy assignment does not rollout constraint templates and constraints onto the cluster if it is a new assignment. On existing assignments, the disabled effect triggers the removal of constraint templates and constraints associated with the policy assignment.
> kubectl get constrainttemplates.templates.gatekeeper.sh NAME AGE k8sazureblockhostnamespace 44m k8sazurecontainernoprivilege 44m --------------------------------------------------------- > kubectl get constrainttemplates.templates.gatekeeper.sh NAME AGE k8sazurecontainernoprivilege 46m
Install Azure Policy for Azure Kubernetes Service
Installing the Azure Policy add-on on AKS is easy. You click on Enable add-on in the Policies section of your AKS cluster and you are done.
CLI and Infrastructure as Code are other options as well.
CLI:
> az aks enable-addons -g azst-aks-demo -n azst-aks-demo --addons azure-policy
ARM:
...
"addonProfiles": {
"azurepolicy": {
"enabled": true,
"config": {
"version": "v2"
}
}
},
...
Terraform:
...
addon_profile {
azure_policy {
enabled = true
}
}
...
Afterwards you assign the policies you want on your AKS clusters.
The rollout of new newly assigned policies might take a while before they appear on the cluster.
Every 10 minutes the gatekeeper-audit pod start an audit run and every 15 minutes the Azure Policy pod reports the compliance state back to Azure Policy.
So, do not wonder if it takes time updating the compliance state in Azure Policy.
Beside policies for Kubernetes resources, I highly recommend assigning the policy Azure Policy Add-on for Kubernetes service (AKS) should be installed and enabled on your clusters being certain that AKS clusters in your fleet have the add-on and are able to receive policies.
Looking at the following screenshots it is easy identifying non-compliant resources and mitigating the issues.
Mitigating can also mean adding a namespace exclusion as the deployment needs a specific configuration being fully functional.
Summary
Azure Policy for AKS has matured since covering it in my blog post back in June.
There is now a solid advanced set of built-in policies and this should cover many scenarios for most users.
But custom policy support is still missing and Azure Policy for AKS does not support an include namespace option as native Gatekeeper does.
I recommend Azure Policy for AKS for production usage. You get a lot out of the box with this managed solution.
When you rely on specific custom policy scenarios that are not covered by the built-in policies you should stay with Gatekeeper.
It is not a one-way street as you can onboard your constraint templates and constraints with minimal effort to Azure Policy for AKS when custom policies are hopefully allowed in the future.
Remember Azure Policy for AKS uses Gatekeeper under the hood.