Daniel's Tech Blog

Cloud Computing, Cloud Native & Kubernetes

Introducing breaking changes to the IoT Edge VK provider helm chart and deployment templates for Kubernetes

When you are running the IoT Edge VK provider deployed by the previous helm chart, you might have noticed the following log entry querying the virtualkubelet container logs.

kubectl logs hub0-iot-edge-connector-6b594c6444-g7mqz virtualkubelet
Flag --taint has been deprecated, Taint key should now be configured using the VK_TAINT_KEY environment variable

The process to define the taint for the virtual node, when using the Virtual Kubelet, has changed. The taint must now be defined via environment variables instead of a flag in the argument section.

This change and a couple of others have been introduced with PR 21 to the IoT Edge VK provider.

-> https://github.com/Azure/iot-edge-virtual-kubelet-provider
-> https://github.com/Azure/iot-edge-virtual-kubelet-provider/pull/21

deployment.yaml before:

...
- name: virtualkubelet
   image: "{{ .Values.vkimage.repository }}:{{ .Values.vkimage.tag }}"
   imagePullPolicy: {{ .Values.vkimage.pullPolicy }}
   env:
     - name: WEB_ENDPOINT_URL
       value: http://localhost:{{ .Values.edgeproviderimage.port }}
   command: ["virtual-kubelet"]
   args: ["--provider", "web", "--nodename", {{ default "web-provider" .Values.env.nodeName | quote }}, "--taint", {{ default "azure.com/iotedge" .Values.env.nodeTaint | quote }}]
...

deployment.yaml after:

...
- name: virtualkubelet
  image: "{{ .Values.vkimage.repository }}:{{ .Values.vkimage.tag }}"
  imagePullPolicy: {{ .Values.vkimage.pullPolicy }}
  env:
  - name: KUBELET_PORT
    value: "10250"
  - name: VKUBELET_TAINT_KEY
    value: {{ .Values.taint.key }}
  - name: VKUBELET_TAINT_VALUE
    value: {{ tpl .Values.taint.value $ }}
  - name: VKUBELET_TAINT_EFFECT
    value: {{ .Values.taint.effect }}
  - name: WEB_ENDPOINT_URL
    value: http://localhost:{{ .Values.edgeproviderimage.port }}
  command: ["virtual-kubelet"]
  args: [
    "--provider", "{{ required "provider is required" .Values.env.provider }}",
    "--namespace", "{{ .Values.env.monitoredNamespace }}",
    "--nodename", "{{ required "nodeName is required" .Values.env.nodeName }}",
    {{- if not .Values.taint.enabled }}
    "--disable-taint", "true",
    {{- end }}
    "--os", "{{ .Values.env.nodeOsType }}"
  ]
...

values.yaml before:

...
env:
  nodeName: iot-edge-connector-hub0
  nodeTaint:
...

values.yaml after:

...env:
  nodeName: iot-edge-connector-hub0
  nodeOsType: Linux
  monitoredNamespace: ""
  # 'provider' must be 'web'
  provider: web
taint:
  enabled: true
  key: virtual-kubelet.io/provider
  value: iotedge
...

Furthermore, a nodeSelector is introduced to ensure that the IoT Edge VK Provider lands on Linux nodes only.

deployment.yaml:

...
      nodeSelector:
        beta.kubernetes.io/os: linux

The last thing that changed is the switch to default rbac.install to true instead of the previous false. Looking at Kubernetes best practices your Kubernetes clusters should have RBAC enabled. So, per default the IoT Edge VK provider helm chart honors this.

IoT Edge VK provider deployment on a Kubernetes cluster with RBAC enabled:

helm install -n hub0 .

IoT Edge VK provider deployment on a Kubernetes cluster with RBAC disabled:

helm install -n hub0 --set rbac.install=false .

The taint changes are introducing breaking changes to your IoT Edge deployment templates.

Before:

...
  tolerations:
  - key: azure.com/iotedge
    effect: NoSchedule
...

After:

...
      tolerations:
      - key: virtual-kubelet.io/provider
        operator: Equal
        value: iotedge
        effect: NoSchedule
...

As you can see you must update your IoT Edge deployment templates, when using the update IoT Edge VK provider helm chart for installing or updating your IoT Edge VK provider deployment.


Posted

in

WordPress Cookie Notice by Real Cookie Banner