Sending logs from the Azure Kubernetes Service master endpoint to Log Analytics

Azure provides native monitoring capabilities for an Azure Kubernetes Service cluster based on Azure Monitor, Azure Log Analytics and the Container Insights solution.

-> https://www.danielstechblog.io/monitor-azure-kubernetes-service-with-azure-monitor-container-health/

But this covers only the part for the agent nodes and the containers that are running on them. If you want to have the logs from the AKS master endpoint, then you must configure the Azure Monitor diagnostic logs first.

-> https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs

The Azure Monitor diagnostic logs are available for the following Kubernetes master node logs.

  • kube-apiserver
  • kube-controller-manager
  • kube-scheduler

Just jump into the resource group that contains your AKS cluster deployment and click on Diagnostic settings under the Monitoring section.

aksmasterlogs01

Next step is selecting the AKS cluster you want to have the master node logs from.

aksmasterlogs02

Click on “Turn on diagnostics” to get the configuration blade.

aksmasterlogs03

On the configuration blade itself you can choose to archive the master node logs into an Azure storage account, stream them to an Azure Event Hub or send them to a Log Analytics workspace. I am prefering the last option as an addition to the container health solution mentioned in the beginning for monitoring the agent nodes and containers. The last step here is to select the master node logs you want to receive.

aksmasterlogs04

After the configuration of the Azure Monitor diagnostic logs we can now enter the necessary search queries to retrieve the master node log entries from the Azure Log Analytics workspace for further inspection and troubleshooting support.

AzureDiagnostics
| where Category == "kube-apiserver"
| where log_s contains "deployments/go-webapp"
| where Resource == "AZST-AKS2"
| project log_s

aksmasterlogs05

AzureDiagnostics
| where Category == "kube-controller-manager"
| where Resource == "AZST-AKS2"
| project log_s

aksmasterlogs06

As you can see it takes only a few steps to get additional insights of your AKS cluster configuring the retrieval of the master node logs.

-> https://docs.microsoft.com/en-us/azure/aks/view-master-logs

Facebooktwittergoogle_pluslinkedinmail