Today we focus on some performance optimizations for the Jenkins worker nodes on Azure instead of security as in the last blog post.
-> https://www.danielstechblog.io/secure-jenkins-worker-nodes-on-azure/
As you might know already each Azure VM gets an additional SSD-based temporary drive beside the OS disk drive. Per default the swap file is stored on this temporary drive, because it is ephemeral.
So, instead using an Azure VM size supporting premium storage (SSD) to get the required disk performance, you can use VM sizes only supporting standard storage (HDD) and leverage the temporary SSD drive for your builds.
The table below shows some VM sizes and their disk throughputs for the OS disk and temporary drive.
VM size OS disk IOPS OS disk throughput R/W in MBps Temporary drive IOPS Temporary drive throughput R/W in MBps Standard_A8_v2 500 60 8000 160 / 80 Standard_D8_v3 500 60 12000 187 / 93 Standard_D4_v2 500 60 24000 375 / 187 Standard_D4_v3 500 60 6000 93 / 46
You can achieve this in two ways specifying the Jenkins working directory or moving the entire HOME directory to the temporary drive.
Let us start with the first option specifying the Jenkins working directory.
As you see in the screenshot, I have set /mnt/jenkins
as the working directory.
Short side note: /mnt
is the mount point of the temporary drive for a Linux-based Azure VM.
Beside the working directory configuration, you add the following two commands at the beginning of the initialization script in the Azure worker node configuration section.
sudo mkdir /mnt/jenkins sudo chown agentadmin:agentadmin /mnt/jenkins
Otherwise your builds are failing with the following error message.
Started by user admin Running as SYSTEM Building remotely on jenkins-workerb27e60 (azure) in workspace /mnt/jenkins/workspace/Test Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-workerb27e60 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:957) at hudson.FilePath.act(FilePath.java:1070) at hudson.FilePath.act(FilePath.java:1059) at hudson.FilePath.mkdirs(FilePath.java:1244) at hudson.model.AbstractProject.checkout(AbstractProject.java:1202) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1818) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) java.nio.file.AccessDeniedException: /mnt/jenkins at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) at java.nio.file.Files.createDirectory(Files.java:674) at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) at java.nio.file.Files.createDirectories(Files.java:767) at hudson.FilePath.mkdirs(FilePath.java:3237) at hudson.FilePath.access$1300(FilePath.java:212) at hudson.FilePath$Mkdirs.invoke(FilePath.java:1252) at hudson.FilePath$Mkdirs.invoke(FilePath.java:1248) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3050) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE
For instance, the output of a simple job after we made the changes and a new Azure worker node was instantiated.
The second option does not require to specify the working directory.
You just need to add the following commands at the beginning of the initialization script section.
sudo mkdir /mnt/jenkins sudo chown agentadmin:agentadmin /mnt/jenkins cp /home/agentadmin/.bash* /mnt/jenkins HOME=/mnt/jenkins sudo sed -i 's+/home/agentadmin+/mnt/jenkins+g' /etc/passwd sudo mount --bind /mnt/jenkins /home/agentadmin
We ensure with those commands that tools get installed into the new HOME directory path during the worker node initialization. But only for those tools that targeting the HOME directory. Furthermore, every job that runs on this specific worker node makes use of the new HOME directory as intended.
The output of our simple job example shows us the correct configuration.