Running Ambassador API gateway on Azure Kubernetes Service

Lately I was playing around with the Ambassador Kubernetes-native microservices API gateway as an ingress controller on Azure Kubernetes Service.

-> https://www.getambassador.io/

Ambassador is based on the popular L7 proxy Envoy by Lyft. Beside the API gateway capabilities, you can use Ambassador just as an ingress controller for publishing your container applications to the outside world.

-> https://www.getambassador.io/features/

The difference to other ingress controller or proxy implementations on Kubernetes is that Ambassador does not rely on the ingress object of Kubernetes. You configure Ambassador through annotations in the Kubernetes service object of your container application.

...
  annotations:
    getambassador.io/config: |
      ---
        apiVersion: ambassador/v1
        kind:  Mapping
        name:  src-ip
        prefix: /
        host: src.trafficmanager.net
        service: src-ip
...

Before we dig deeper into the configuration let us have a look at the deployment of Ambassador on an AKS cluster. On the Ambassador website you can find two getting started guides. One leveraging YAML templates and the other one a helm chart.

-> https://www.getambassador.io/user-guide/getting-started/
-> https://www.getambassador.io/user-guide/helm/

In my tests I used the YAML template for the Ambassador deployment and downloaded it on my local workstation as recommended.

-> https://getambassador.io/yaml/ambassador/ambassador-rbac.yaml

We will adjust the template before deploying Ambassador on an AKS cluster and coming back to this later.

The reason for the adjustments is the Ambassador service definition that sets the externalTrafficPolicy to Local instead of using the Kubernetes default Cluster. This preserves the client IP addresses and prevents an additional hop you can expect with externalTrafficPolicy set to Cluster.

If you want to know more about Kubernetes external traffic policies in detail, have a look at the following blog post.

-> https://www.asykim.com/blog/deep-dive-into-kubernetes-external-traffic-policies
-> https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip

When using externalTrafficPolicy set to Local we should specify a pod anti-affinity rule in the Ambassador template to ensure equal traffic distribution across all Ambassador pods.

...
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: service
                operator: In
                values:
                - ambassador
            topologyKey: kubernetes.io/hostname
...

The pod anti-affinity rule requiredDuringSchedulingIgnoredDuringExecution applies during the scheduling of the Ambassador pods and forces Kubernetes to deploy the pods on different agent nodes.

A more moderate pod anti-affinity rule is preferredDuringSchedulingIgnoredDuringExecution.

...
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: service
                  operator: In
                  values:
                  - ambassador
              topologyKey: kubernetes.io/hostname
...

This rule tells Kubernetes to deploy the pods on different agent nodes if possible, but not prevents in certain circumstances that two or more Ambassador pods run on the same agent node.

You can use the following templates to roll out Ambassador on an AKS cluster.

-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/ambassador-rbac.yaml
-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/ambassador-rbac-soft.yaml
-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/ambassador-svc.yaml

Just run the following kubectl commands.

kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/ambassador/ambassador-rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/ambassador/ambassador-svc.yaml

When the deployment was successful you should see a similar output on your AKS cluster.

[] > kubectl get pods -o wide -l service=ambassador
NAME                          READY   STATUS    RESTARTS   AGE     IP             NODE                                NOMINATED NODE
ambassador-7df789769b-hbltf   1/1     Running   0          2m26s   10.240.0.204   aks-nodepool1-14987876-vmss000001   <none>
ambassador-7df789769b-lx5hf   1/1     Running   4          6m3s    10.240.1.14    aks-nodepool1-14987876-vmss000002   <none>
ambassador-7df789769b-mx65k   1/1     Running   1          6m2s    10.240.1.138   aks-nodepool1-14987876-vmss000003   <none>
[] > kubectl get svc -o wide -l service=ambassador
NAME         TYPE           CLUSTER-IP   EXTERNAL-IP    PORT(S)        AGE     SELECTOR
ambassador   LoadBalancer   10.0.44.22   51.136.55.44   80:30314/TCP   5d15h   service=ambassador
[] > kubectl get svc -o wide -l service=ambassador-admin
NAME               TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE     SELECTOR
ambassador-admin   NodePort   10.0.173.115   <none>        8877:32422/TCP   5d21h   service=ambassador

As an example container application I am using the echoserver.

-> https://console.cloud.google.com/gcr/images/google-containers/GLOBAL/echoserver?gcrImageListsize=30

The echoserver responds with information about the HTTP request.

ambassador01

Let us have a look at the service object definition in the template file.

-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/src-ip-ambassador.yaml

apiVersion: v1
kind: Service
metadata:
  name: src-ip
  labels:
    app: src-ip
  annotations:
    getambassador.io/config: |
      ---
        apiVersion: ambassador/v1
        kind:  Mapping
        name:  src-ip
        prefix: /
        host: src.trafficmanager.net
        service: src-ip
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: src-ip

The first line of the Ambassador annotation defines the API version followed by the object definition. In this case it is a Mapping object for our redirect. Then we define the name of the Mapping object before we configure the redirect. As I am using an Azure Traffic Manager for my container application exclusively, I am specifying / as my prefix. The host contains the Traffic Manager URL. Last thing to do in the configuration is to specify the Kubernetes service that receives the traffic.

It is a simple redirect configuration and more configuration examples can be found on the Ambassador website if you want to test more advanced scenarios.

-> https://www.getambassador.io/reference/configuration

Facebooktwittergoogle_pluslinkedinmail