Tag: Public Cloud
-
Use Fluent Bit for Kubernetes events gathering on Azure Kubernetes Service
For a while now Fluent Bit has a new input plugin that allows us to gather Kubernetes events, modify, and ingest them into the logging backend. -> https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes-events Today we look at how to configure and deploy Fluent Bit to gather Kubernetes events on an Azure Kubernetes Service cluster and ingest them into an Azure…
-
New Fluent Bit Azure Data Explorer output plugin version available
In the recent 3.2.2 release, the new Azure Data Explorer output plugin version is available. Azure_kusto (Output) fix multiple files tail issue and timeout issue (#8430) -> https://fluentbit.io/announcements/v3.2.2/ -> https://github.com/fluent/fluent-bit/pull/8430 The previous version had a couple of issues that have now been fixed. For instance, I was running into an unreliable authentication with the earlier…
-
Retrieve Kubernetes Pods IP addresses with Fluent Bit
In the recent 3.2.1 release, Fluent Bit added a long-awaited functionality that has been available for a long time in FluentD: the capability to extract the Kubernetes Pod IP address and enrich the log data with it. Kubernetes (Filter) Retrieve kubernetes pod ip address if it is set in status.podip (#2783) -> https://fluentbit.io/announcements/v3.2.1/ -> https://github.com/fluent/fluent-bit/issues/2301…
-
Designing the Azure Data Explorer table structure for Azure Diagnostic Logs or Defender for Cloud data ingestion
In my recent blog posts about Azure Data Explorer, I wrote about Activity Logs and Diagnostic Logs ingestion. -> https://www.danielstechblog.io/ingesting-azure-diagnostic-logs-into-azure-data-explorer/ -> https://www.danielstechblog.io/export-azure-kubernetes-service-control-plane-logs-to-azure-data-explorer/ Today, I would like to discuss how to design the Azure Data Explorer table for the Diagnostic Logs or Defender for Cloud log data ingestion. Depending on your preferences, you can choose between…
-
Export Azure Kubernetes Service control plane logs to Azure Data Explorer
In today’s blog post, we look at the Azure Kubernetes Service control plane logs and how to ingest them into Azure Data Explorer. Especially, the Kubernetes Audit (kube-audit) log. Azure Data Explorer – Ingestion Method Looking at the export options for the Azure Kubernetes Service control plane logs we can choose between an Azure Storage…
-
Sneak peek into the new Fluent Bit Azure Data Explorer output plugin version
In my last blog post, I wrote about the different options for using Azure Data Explorer as a logging backend for Fluent Bit. -> https://www.danielstechblog.io/using-azure-data-explorer-as-logging-backend-for-fluent-bit/ Especially, about my issues getting the Azure Data Explorer output plugin working and why this led to the decision to use the Kafka output plugin in combination with Azure Event…