Running an AKS cluster or containers on Azure does not eliminate the need to monitor your workloads. Looking at Azure it is simple to configure and deploy a monitoring solution for AKS. All what you require to do so are the following Azure components.
First, you need an Azure Log Analytics workspace and second, the container monitoring solution for Log Analytics. Log Analytics comes with a free tier SKU including max. 500 MB data upload daily and 7 days data retention, which should be sufficient enough to get started.
If you do not have a Log Analytics workspace with the container monitoring solution yet, you can deploy one with the following Azure Resource Manager template.
Launch the Azure Cloud Shell and execute the following Azure CLI command, where you only have to define the name for the Log Analytics workspace and the resource group you would like to deploy the workspace in.
az group deployment create --resource-group myResourceGroupName --template-uri https://raw.githubusercontent.com/neumanndaniel/armtemplates/master/operationsmanagement/containerMonitoringSolution.json --parameters workspaceName=myWorkspaceName --verbose
The next step would be following the Azure docs AKS tutorial with several manual steps to deploy the OMS agent onto the AKS agent nodes.
-> https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-monitor
You can follow these steps on your own or use the deployment scripts I have written for it. Depending on your preferences I am providing a bash script with Azure CLI or a PowerShell Core script with Azure CLI.
Bash:
#!/bin/bash omsWorkspaceName=$1 resourceGroupName=$2 gitHubTemplateUri='https://raw.githubusercontent.com/neumanndaniel/armtemplates/master/output/logAnalyticsWorkspace.json' gitHubLogAnalyticsAgentUri='https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/omsagent/oms-daemonset.yaml' #Get Log Analytics workspaceId and primary key, and deploy Log Analytics agent on the AKS cluster output=$(az group deployment create --resource-group $resourceGroupName --template-uri $gitHubTemplateUri --parameters workspaceName=$omsWorkspaceName --verbose) workspaceId=$(echo $output|jq -r .properties.outputs.workspaceId.value) primaryKey=$(echo $output|jq -r .properties.outputs.primaryKey.value) workspaceIdEncoded=$(echo $workspaceId|base64 --wrap=0) primaryKeyEncoded=$(echo $primaryKey|base64 --wrap=0) echo "apiVersion: v1 data: KEY: $primaryKeyEncoded WSID: $workspaceIdEncoded kind: Secret metadata: name: omsagent-secret namespace: default type: Opaque" > omsagent-secret.yaml kubectl apply -f ./omsagent-secret.yaml wget $gitHubLogAnalyticsAgentUri --output-document=oms-daemonset.yaml kubectl apply -f ./oms-daemonset.yaml
-> https://github.com/neumanndaniel/kubernetes/blob/master/omsagent/deployOmsAgentOnAks.sh
PowerShell Core:
#Parameters for AKS OMS deployment Param( [Parameter(Mandatory=$true,Position=1)] [string]$omsWorkspaceName, [Parameter(Mandatory=$true,Position=2)] [string]$resourceGroupName ) #Variables for AKS OMS deployment $gitHubTemplateUri='https://raw.githubusercontent.com/neumanndaniel/armtemplates/master/output/logAnalyticsWorkspace.json' $gitHubLogAnalyticsAgentUri='https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/omsagent/oms-daemonset.yaml' #Get Log Analytics workspaceId and primary key, and deploy Log Analytics agent on the AKS cluster $output=az group deployment create --resource-group $resourceGroupName --template-uri $gitHubTemplateUri --parameters workspaceName=$omsWorkspaceName --verbose|ConvertFrom-Json $workspaceId=$output.properties.outputs.workspaceId.value $primaryKey=$output.properties.outputs.primaryKey.value $workspaceIdEncoded=[Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($workspaceId)) $primaryKeyEncoded=[Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($primaryKey)) $yamlDefinition='apiVersion: v1 data: KEY: '+$primaryKeyEncoded+' WSID: '+$workspaceIdEncoded+' kind: Secret metadata: name: omsagent-secret namespace: default type: Opaque' Write-Output $yamlDefinition > omsagent-secret.yaml kubectl apply -f ./omsagent-secret.yaml Invoke-WebRequest $gitHubLogAnalyticsAgentUri -OutFile ./oms-daemonset.yaml kubectl apply -f ./oms-daemonset.yaml
-> https://github.com/neumanndaniel/kubernetes/blob/master/omsagent/deployOmsAgentOnAks.ps1
In this blog post I will only explain the PowerShell Core version, but both scripts are doing the same.
Open a new Azure Cloud Shell session or use the previous one and type in pwsh -noprofile and execute it to start the PowerShell Core. Next, download the PowerShell Core script, provide your Log Analytics workspace name, the resource group name and run it to kick off the deployment.
Invoke-WebRequest https://raw.githubusercontent.com/neumanndaniel/kubernetes/master/omsagent/deployOmsAgentOnAks.ps1 -OutFile ./deployOmsAgentOnAks.ps1 ./deployOmsAgentOnAks.ps1 -omsWorkspaceName myWorkspaceName -resourceGroupName myResourceGroupName -Verbose
The script makes use of an Azure Resource Manager template to get the workspace id and the workspace’s primary key, which will be required later.
-> https://github.com/neumanndaniel/armtemplates/blob/master/output/logAnalyticsWorkspace.json
After that it creates the Kubernetes secret with the workspace id and primary key for the OMS agent deployment in form of a YAML file. Why a YAML file and not directly? I will come back to this later. The last step is the OMS agent deployment through a YAML file. In the script I am referencing to the YAML definition file in my GitHub repository.
-> https://github.com/neumanndaniel/kubernetes/blob/master/omsagent/oms-daemonset.yaml
When all the things went successful, you should see the first data sets arriving in a couple of minutes.
Now the explanation of using a YAML file for the Kubernetes secret deployment. The written scripts should not be used only for the initial deployment, they should also cover updating the secret for the OMS agents and redeploying the OMS agents using the updated secret. That is the reason for it and why I am using kubectl apply instead of kubectl create.
Happy monitoring!