In May this year, Microsoft announced the general availability of the Azure Linux support in Azure Kubernetes Service.
-> https://azure.microsoft.com/en-us/updates/generally-available-azure-linux-support-in-aks/?WT.mc_id=AZ-MVP-5000119
-> https://techcommunity.microsoft.com/t5/linux-and-open-source-blog/introducing-the-azure-linux-container-host-for-aks/ba-p/3824101?WT.mc_id=AZ-MVP-5000119
Azure Linux is Microsoft’s Linux distribution of CBL-Mariner.
-> https://github.com/microsoft/CBL-Mariner
You can choose now between using Ubuntu or Azure Linux as the host operating system for your node pools in Azure Kubernetes Service, where Ubuntu is still the default.
Today, we quickly focus on running Fluent Bit on Azure Linux in Azure Kubernetes Service.
Situation
In its Azure Kubernetes Service documentation, Microsoft highlighted current limitations when using Azure Linux until September 19, 2023.
Some addons, extensions, and open-source integrations may not be supported yet on Azure Linux. Azure Monitor, Grafana, Helm, Key Vault, and Container Insights are supported.
-> https://learn.microsoft.com/en-us/azure/aks/use-azure-linux?WT.mc_id=AZ-MVP-5000119
-> https://github.com/MicrosoftDocs/azure-docs/commit/54b63b106c932c835f8bf0cc0bb612e774f4d251
That said, they did not explicitly mention Fluent Bit as supported. But Container Insights uses the following open-source tools under the hood: Fluentd, Fluent Bit, and Telegraf.
-> https://github.com/microsoft/Docker-Provider
Hence, Fluent Bit should work without any issues on Azure Linux.
Evidence
I am running an Azure Kubernetes Service cluster with Kubernetes in version 1.27.3 in my Azure subscription. The Azure Kubernetes Service cluster uses Azure Linux as the host operating system for its node pools.
❯ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME aks-default-33543087-vmss00000l Ready agent 7m24s v1.27.3 10.240.0.4 <none> CBL-Mariner/Linux 5.15.122.1-2.cm2 containerd://1.6.18 aks-default-33543087-vmss00000m Ready agent 7m v1.27.3 10.240.0.6 <none> CBL-Mariner/Linux 5.15.122.1-2.cm2 containerd://1.6.18 aks-default-33543087-vmss00000n Ready agent 7m9s v1.27.3 10.240.0.5 <none> CBL-Mariner/Linux 5.15.122.1-2.cm2 containerd://1.6.18
My next step is the deployment of Fluent Bit in version 2.1.9 to the cluster by running my deployment script.
./deploy-fluent-bit.sh <resource_group> <log_analytics_workspace>
You can find my example configuration on my GitHub repository.
-> https://github.com/neumanndaniel/kubernetes/tree/master/fluent-bit
After a few seconds, Fluent Bit is up and running on Azure Linux.
❯ kubectl get pods NAME READY STATUS RESTARTS AGE fluent-bit-7kdtn 1/1 Running 0 2m30s fluent-bit-brnmz 1/1 Running 0 2m38s fluent-bit-v9wjs 1/1 Running 0 2m22s ❯ kubectl logs fluent-bit-7kdtn Fluent Bit v2.1.9 * Copyright (C) 2015-2022 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io [2023/09/20 18:47:32] [ info] [fluent bit] version=2.1.9, commit=c625ad7ea4, pid=1 [2023/09/20 18:47:32] [ info] [storage] ver=1.4.0, type=memory+filesystem, sync=normal, checksum=off, max_chunks_up=128 [2023/09/20 18:47:32] [ info] [storage] backlog input plugin: storage_backlog.2 [2023/09/20 18:47:32] [ info] [cmetrics] version=0.6.3 [2023/09/20 18:47:32] [ info] [ctraces ] version=0.3.1 [2023/09/20 18:47:32] [ info] [input:storage_backlog:storage_backlog.2] initializing [2023/09/20 18:47:32] [ info] [input:storage_backlog:storage_backlog.2] storage_strategy='memory' (memory only) [2023/09/20 18:47:32] [ info] [input:storage_backlog:storage_backlog.2] queue memory limit: 9.5M [2023/09/20 18:47:32] [ info] [filter:kubernetes:logs_filter_1] https=1 host=10.240.0.6 port=10250 [2023/09/20 18:47:32] [ info] [filter:kubernetes:logs_filter_1] token updated [2023/09/20 18:47:32] [ info] [filter:kubernetes:logs_filter_1] local POD info OK [2023/09/20 18:47:32] [ info] [filter:kubernetes:logs_filter_1] testing connectivity with Kubelet... [2023/09/20 18:47:33] [ info] [filter:kubernetes:logs_filter_1] connectivity OK [2023/09/20 18:47:33] [ info] [filter:kubernetes:events_filter_1] https=1 host=10.240.0.6 port=10250 [2023/09/20 18:47:33] [ info] [filter:kubernetes:events_filter_1] token updated [2023/09/20 18:47:33] [ info] [filter:kubernetes:events_filter_1] local POD info OK [2023/09/20 18:47:33] [ info] [filter:kubernetes:events_filter_1] testing connectivity with Kubelet... [2023/09/20 18:47:33] [ info] [filter:kubernetes:events_filter_1] connectivity OK [2023/09/20 18:47:33] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020 [2023/09/20 18:47:33] [ info] [sp] stream processor started
A few minutes later, we see the ingested log data in Log Analytics, the final proof of a correct working Fluent Bit installation on Azure Linux in Azure Kubernetes Service.
Summary
Fluent Bit works out of the box on Azure Linux with the same configuration as on Ubuntu. When you are curious to learn more about Azure Linux, have a look at the following link.
-> https://learn.microsoft.com/en-us/azure/azure-linux/?WT.mc_id=AZ-MVP-5000119