Lately I was playing around with the Ambassador Kubernetes-native microservices API gateway as an ingress controller on Azure Kubernetes Service.
-> https://www.getambassador.io/
Ambassador is based on the popular L7 proxy Envoy by Lyft. Beside the API gateway capabilities, you can use Ambassador just as an ingress controller for publishing your container applications to the outside world.
-> https://www.getambassador.io/features/
The difference to other ingress controller or proxy implementations on Kubernetes is that Ambassador does not rely on the ingress object of Kubernetes. You configure Ambassador through annotations in the Kubernetes service object of your container application.
...
annotations:
getambassador.io/config: |
---
apiVersion: ambassador/v1
kind: Mapping
name: src-ip
prefix: /
host: src.trafficmanager.net
service: src-ip
...
Before we dig deeper into the configuration let us have a look at the deployment of Ambassador on an AKS cluster. On the Ambassador website you can find two getting started guides. One leveraging YAML templates and the other one a helm chart.
-> https://www.getambassador.io/user-guide/getting-started/
-> https://www.getambassador.io/user-guide/helm/
In my tests I used the YAML template for the Ambassador deployment and downloaded it on my local workstation as recommended.
-> https://getambassador.io/yaml/ambassador/ambassador-rbac.yaml
We will adjust the template before deploying Ambassador on an AKS cluster and coming back to this later.
The reason for the adjustments is the Ambassador service definition that sets the externalTrafficPolicy to Local instead of using the Kubernetes default Cluster. This preserves the client IP addresses and prevents an additional hop you can expect with externalTrafficPolicy set to Cluster.
If you want to know more about Kubernetes external traffic policies in detail, have a look at the following blog post.
-> https://www.asykim.com/blog/deep-dive-into-kubernetes-external-traffic-policies
-> https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip
When using externalTrafficPolicy set to Local we should specify a pod anti-affinity rule in the Ambassador template to ensure equal traffic distribution across all Ambassador pods.
...
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: service
operator: In
values:
- ambassador
topologyKey: kubernetes.io/hostname
...
The pod anti-affinity rule requiredDuringSchedulingIgnoredDuringExecution applies during the scheduling of the Ambassador pods and forces Kubernetes to deploy the pods on different agent nodes.
A more moderate pod anti-affinity rule is preferredDuringSchedulingIgnoredDuringExecution.
...
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: service
operator: In
values:
- ambassador
topologyKey: kubernetes.io/hostname
...
This rule tells Kubernetes to deploy the pods on different agent nodes if possible, but not prevents in certain circumstances that two or more Ambassador pods run on the same agent node.
You can use the following templates to roll out Ambassador on an AKS cluster.
-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/ambassador-rbac.yaml
-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/ambassador-rbac-soft.yaml
-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/ambassador-svc.yaml
Just run the following kubectl commands.
kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/ambassador/ambassador-rbac.yaml kubectl apply -f https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/ambassador/ambassador-svc.yaml
When the deployment was successful you should see a similar output on your AKS cluster.
[] > kubectl get pods -o wide -l service=ambassador NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE ambassador-7df789769b-hbltf 1/1 Running 0 2m26s 10.240.0.204 aks-nodepool1-14987876-vmss000001 <none> ambassador-7df789769b-lx5hf 1/1 Running 4 6m3s 10.240.1.14 aks-nodepool1-14987876-vmss000002 <none> ambassador-7df789769b-mx65k 1/1 Running 1 6m2s 10.240.1.138 aks-nodepool1-14987876-vmss000003 <none> [] > kubectl get svc -o wide -l service=ambassador NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR ambassador LoadBalancer 10.0.44.22 51.136.55.44 80:30314/TCP 5d15h service=ambassador [] > kubectl get svc -o wide -l service=ambassador-admin NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR ambassador-admin NodePort 10.0.173.115 <none> 8877:32422/TCP 5d21h service=ambassador
As an example container application I am using the echoserver.
-> https://console.cloud.google.com/gcr/images/google-containers/GLOBAL/echoserver?gcrImageListsize=30
The echoserver responds with information about the HTTP request.
Let us have a look at the service object definition in the template file.
-> https://github.com/neumanndaniel/kubernetes/blob/master/ambassador/src-ip-ambassador.yaml
apiVersion: v1
kind: Service
metadata:
name: src-ip
labels:
app: src-ip
annotations:
getambassador.io/config: |
---
apiVersion: ambassador/v1
kind: Mapping
name: src-ip
prefix: /
host: src.trafficmanager.net
service: src-ip
spec:
ports:
- port: 80
targetPort: 8080
selector:
app: src-ip
The first line of the Ambassador annotation defines the API version followed by the object definition. In this case it is a Mapping object for our redirect. Then we define the name of the Mapping object before we configure the redirect. As I am using an Azure Traffic Manager for my container application exclusively, I am specifying / as my prefix. The host contains the Traffic Manager URL. Last thing to do in the configuration is to specify the Kubernetes service that receives the traffic.
It is a simple redirect configuration and more configuration examples can be found on the Ambassador website if you want to test more advanced scenarios.