Retrieve Kubernetes Pods IP addresses with Fluent Bit

In the recent 3.2.1 release, Fluent Bit added a long-awaited functionality that has been available for a long time in FluentD: the capability to extract the Kubernetes Pod IP address and enrich the log data with it.

Kubernetes (Filter)

• Retrieve kubernetes pod ip address if it is set in status.podip (#2783)

-> https://fluentbit.io/announcements/v3.2.1/
-> https://github.com/fluent/fluent-bit/issues/2301
-> https://github.com/fluent/fluent-bit/pull/2783

If you are using several filters like me that process the output of the Kubernetes filter you need to adjust those filters to benefit from this new functionality.

For instance, I am just using the nest and modify filter and only need one line Rename kubernetes_pod_ip PodIp to add the Kubernetes Pod IP address to the log data.

...
    [FILTER]
        Name         nest
        Alias        logs_filter_2
        Match        kubernetes.logs.*
        Operation    lift
        Nested_under kubernetes
        Add_prefix   kubernetes_

    [FILTER]
        Name   modify
        Alias  logs_filter_3
        Match  kubernetes.logs.*
        Add    Cluster                    ${CLUSTER}
        Add    Region                     ${REGION}
        Add    Environment                ${ENVIRONMENT}
        Add    NodeIp                     ${NODE_IP}
        Rename time                       TimeGenerated
        Rename message                    LogMessage
        Rename kubernetes_pod_name        PodName
        Rename kubernetes_namespace_name  PodNamespace
        Rename kubernetes_container_image ContainerImage
        Rename kubernetes_container_hash  ContainerImageDigest
        Rename kubernetes_docker_id       ContainerId
        Rename kubernetes_container_name  ContainerName
        Rename kubernetes_pod_id          PodId
        Rename kubernetes_pod_ip          PodIp
        Rename kubernetes_host            Computer
        Rename stream                     LogSource
        Remove logtag
...

After applying the configuration changes to the Fluent Bit deployment on my Azure Kubernetes Service cluster, it takes a few seconds for the new log data to have the Pod’s IP address attached to it.

The entire configuration example for the Azure Data Explorer and Fluent Bit configuration is available on my GitHub repository.

-> https://github.com/neumanndaniel/scripts/tree/main/Azure_Data_Explorer/Fluent_Bit_Kubernetes
-> https://github.com/neumanndaniel/kubernetes/tree/master/fluent-bit/azure-data-explorer
-> https://www.danielstechblog.io/sneak-peek-into-the-new-fluent-bit-azure-data-explorer-output-plugin-version/