Kubernetes network policies on Azure Kubernetes Service with Azure NPM

Microsoft provides an own network policy module to implement Kubernetes network policies with the Azure CNI plugin for acs-engine and AKS called Azure NPM.

-> https://github.com/Azure/azure-container-networking/tree/master/npm

The Azure NPM is available since quite some time for acs-engine and natively integrated, but not yet for AKS. If you want to use Azure NPM on Azure Kubernetes Service, you can do so. Just run the following command to deploy the Azure NPM daemonset on your AKS Cluster.

-> https://github.com/Azure/acs-engine/blob/master/parts/k8s/addons/kubernetesmasteraddons-azure-npm-daemonset.yaml

kubectl apply -f https://raw.githubusercontent.com/Azure/acs-engine/master/parts/k8s/addons/kubernetesmasteraddons-azure-npm-daemonset.yaml

Afterwards run

kubectl get pods -n kube-system --selector=k8s-app=azure-npm -o wide

to check the successful deployment.

The output should look like this.

NAME              READY   STATUS    RESTARTS   AGE   IP             NODE                       NOMINATED NODE
azure-npm-5f6mb   1/1     Running   0          1d    10.240.0.4     aks-agentpool-14987876-1   <none>
azure-npm-lf5w7   1/1     Running   0          1d    10.240.0.115   aks-agentpool-14987876-0   <none>

The Azure NPM works following the LIFO principle to establish fine grained network isolation between your pods.

-> https://github.com/Azure/azure-container-networking/pull/258

So, you can ensure that only specific pods forming the front-end can talk to a certain backend. Following the LIFO principle, you deploy a deny all inbound network policy to a certain namespace first to restrict all ingress traffic to all pods in this namespace.

Warning: Do not deploy a deny all inbound or outbound network policy to the kube-system namespace. This causes a serious operational impact on your Kubernetes cluster.

Then you deploy additional network policies to allow the traffic to your container applications as necessary but maintaining the network security/isolation between different container applications in the namespace. Furthermore, you establish with the deny all inbound network policy a first defense line that new container applications are not directly exposed and available for network traffic.

Let us dive into one example. I am running two simple web applications on my AKS cluster. One provides a static web page (aks.trafficmanager.net) and the other one (src.trafficmanager.net) shows you specific information about the web request targeting the web application.

aksazurenpm08

The sketch above shows how the web applications are published to the outside world.

aksazurenpm01aksazurenpm02

Before starting the lock down of the default namespace, we deploy a busybox container first.

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
  labels:
    app: busybox
spec:
  containers:
  - name: busybox
    image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
  restartPolicy: Always
kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/azure-npm/busybox.yaml

-> https://github.com/neumanndaniel/kubernetes/blob/master/azure-npm/busybox.yaml

We need the busybox container later in the example.

Next, we roll out the deny all inbound network policy to the default namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-inbound
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Ingress
kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/azure-npm/deny-all-inbound.yaml

-> https://github.com/neumanndaniel/kubernetes/blob/master/azure-npm/deny-all-inbound.yaml

When calling the web applications, they are not accessible anymore.

aksazurenpm03

So, we allow the traffic to the aks.trafficmanager.net web application from the Internet again with the following network policy.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-go-webapp
  namespace: default
spec:
  podSelector:
    matchLabels:
      app: go-webapp
  policyTypes:
  - Ingress
  ingress:
  - ports:
    - port: 8080
    from: []
kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/azure-npm/allow-go-webapp.yaml

-> https://github.com/neumanndaniel/kubernetes/blob/master/azure-npm/allow-go-webapp.yaml

As you can see in the template, Kubernetes network policies are label-based and with that totally dynamic and not dependent on the frequently changing pod IP addresses.

aksazurenpm04aksazurenpm05

The aks.trafficmanager.net web application is now accessible again, but not the src.trafficmanager.net. This one should only be accessible from the busybox container and not from the Internet or any other pods running on the AKS cluster.

In the next network policy template, you will see that the label-based selection also works in the from section of the network policy definition.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-src-ip
  namespace: default
spec:
  podSelector:
    matchLabels:
      app: src-ip-internal
  policyTypes:
  - Ingress
  ingress:
  - ports:
    - port: 8080
    from:
      - podSelector:
          matchLabels:
            app: busybox
kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/azure-npm/allow-src-ip.yaml

-> https://github.com/neumanndaniel/kubernetes/blob/master/azure-npm/allow-src-ip.yaml

So, the network policy is applied to all pods with the label matching app=src-ip-internal and only allows ingress traffic from pods matching the label app=busybox. Let us connect to the busybox container to test if it works.

kubectl exec -it busybox /bin/sh

In the container itself we run the following set of commands to test the connectivity.

wget --spider -S src.trafficmanager.net
wget --spider -S src-ip-internal
wget --spider -S 10.240.255.252
wget src-ip-internal
cat index.html

Have a look at the following screenshots showing the output of the previous commands.

aksazurenpm06aksazurenpm07

Calling src.trafficmanager.net does not work as expected, because it is the external address. Trying the Kubernetes service address src-ip-internal or the IP address of the Azure Internal Load Balancer works. So, the src.trafficmanager.net web application is only accessible from the busybox container inside the AKS cluster.

I hope you got an idea how network policies can be a beneficial security building block in your AKS cluster to better control the traffic flow to or from your container applications.

A good collection of Kubernetes network policy examples and use cases can be found under the following GitHub repository.

-> https://github.com/ahmetb/kubernetes-network-policy-recipes

Facebooktwittergoogle_pluslinkedinmail