Before I begin to write about this topic, I want to clarify that the results are not an official statement by Microsoft.
The opinions expressed herein are my own personal opinions and do not represent my employer’s view in anyway.
Now we have clarified that, let us begin to talk about what ADE and SSE are in a short way. ADE stands for Azure Disk Encryption and is the volume-based encryption option for Azure IaaS VMs leveraging BitLocker or dm-crypt inside the operating system.
SSE stands for Storage Service Encryption and is the encryption option to enable encryption on storage account level. Both ADE and SSE are working with AES 256 bit.
My setup for the performance tests requires a look at the different performance values that the disk provides and what different VM sizes support.
Managed disk type | Throughput | IOPS |
S4|S6|S10|S20|S30 | 60 MBps | 500 |
P10 | 100 MBps | 500 |
P20 | 150 MBps | 2300 |
P30 | 200 MBps | 5000 |
Best fitting VM sizes for the tests are Standard_D4_v2 and Standard_DS4_v2. I had to select the Standard_DS4_v2 series to have support by the VM size for not limiting the throughput. Have a look at the details in the Azure documentation.
-> https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-general#dsv2-series
As well I am using for the standard managed disks the same VM size as for premium managed disks. So I am making sure this does not influence the test results using different VM sizes, even if it is possible for standard managed disks looking at the supported performance values.
For running the storage performance tests I am using the Microsoft PerfInsights tool.
-> https://www.microsoft.com/en-us/download/details.aspx?id=54915
But I had to modify the test settings in the PerfInsights_Settings.xml file to get the appropriate results for the throughput tests regarding the Azure documentation for testing premium storage based disks.
If you are using an application, which allows you to change the IO size, use this rule of thumb for the IO size in addition to other performance guidelines,
- Smaller IO size to get higher IOPS. For example, 8 KB for an OLTP application.
- Larger IO size to get higher Bandwidth/Throughput. For example, 1024 KB for a data warehouse application.
Here is an example on how you can calculate the IOPS and Throughput/Bandwidth for your application. Consider an application using a P30 disk. The maximum IOPS and Throughput/Bandwidth a P30 disk can achieve is 5000 IOPS and 200 MB per second respectively. Now, if your application requires the maximum IOPS from the P30 disk and you use a smaller IO size like 8 KB, the resulting Bandwidth you will be able to get is 40 MB per second. However, if your application requires the maximum Throughput/Bandwidth from P30 disk, and you use a larger IO size like 1024 KB, the resulting IOPS will be less, 200 IOPS.
Have a look at the following screenshots, the changes are marked yellow.
The changes are the IO size for running the throughput test and adding the throughout test for the OS disk as well.
Before we jump into the test results I will share the settings for the standard managed disks and premium managed disks test setup with you, so you can run the tests on your own.
Standard managed disks:
- Azure region: North Europe
- Azure VM size: Standard_D4_v2
- OS: Windows Server 2016
- 100% write
- 1 GB test file
- 30 seconds warm-up
- 90 seconds test duration
- 3 runs each for IOPS and throughput testing for each disk
- OS disk: S10 | NTFS 4 KB | read/write cache enabled
- Data disk: S30 | NTFS 64 KB | none
Premium managed disks:
- Azure region: North Europe
- Azure VM size: Standard_DS4_v2
- OS: Windows Server 2016
- 100% write
- 1 GB test file
- 30 seconds warm-up
- 90 seconds test duration
- 3 runs each for IOPS and throughput testing for each disk
- OS disk: P10 | NTFS 4 KB | read/write cache enabled
- Data disks: P10, P20, P30 | NTFS 64 KB | none
Test results – standard managed disks – ADE:
Standard_D4_v2 w/o ADE | Standard_D4_v2 w/ ADE | |
OS disk IOPS (500) | 494.24 (99%) IOPS | 494.83 (99%) IOPS |
OS disk throughput (60MB) | 60.00 (100%) MB/sec | 59.96 (99%) MB/sec |
Data disk IOPS (500) | 495.89 (99%) IOPS | 495.85 (99%) IOPS |
Data disk throughput (60MB) | 60.00 (100%) MB/sec | 60.00 (100%) MB/sec |
CPU average in % when creating 20 GB fixed VHD on data disk | 0.234% | 3.200 % |
Test results – standard managed disks – SSE:
Standard_D4_v2 w/o SSE | Standard_D4_v2 w/ SSE | |
OS disk IOPS (500) | 494.24 (99%) IOPS | 494.32 (99%) IOPS |
OS disk throughput (60MB) | 60.00 (100%) MB/sec | 59.99 (99%) MB/sec |
Data disk IOPS (500) | 495.89 (99%) IOPS | 496.13 (99%) IOPS |
Data disk throughput (60MB) | 60.00 (100%) MB/sec | 60.00 (100%) MB/sec |
CPU average in % when creating 20 GB fixed VHD on data disk | 0.234% | 0.215% |
Test results – premium managed disks – ADE:
Standard_DS4_v2 w/o ADE | Standard_DS4_v2 w/ ADE | |
P10 OS disk IOPS (500) | 508.15 (102%) IOPS | 508.24 (102%) IOPS |
P10 OS disk throughput (100MB) | 95.20 (95%) MB/sec | 70.15 (70%) MB/sec |
P10 data disk IOPS (500) | 509.94 (102%) IOPS | 509.97 (102%) IOPS |
P10 data disk throughput (100MB) | 97.28 (97%) MB/sec | 72.55 (73%) MB/sec |
P20 data disk IOPS (2300) | 2345.75 (102%) IOPS | 2345.83 (102%) IOPS |
P20 data disk throughput (150MB) | 145.90 (97%) MB/sec | 145.91 (97%) MB/sec |
P30 data disk IOPS (5000) | 5099.80 (102%) IOPS | 5099.58 (102%) IOPS |
P30 data disk throughput (200MB) | 192.07 (96%) MB/sec | 194.50 (97%) MB/sec |
CPU average in % when creating 20 GB fixed VHD on data disk | 0.272% | 2.934% |
Test results – premium managed disks – SSE:
Standard_DS4_v2 w/o SSE | Standard_DS4_v2 w/ SSE | |
P10 OS disk IOPS (500) | 508.15 (102%) IOPS | 498.29 (100%) IOPS |
P10 OS disk throughput (100MB) | 95.20 (95%) MB/sec | 97.20 (97%) MB/sec |
P10 data disk IOPS (500) | 509.94 (102%) IOPS | 509.97 (102%) IOPS |
P10 data disk throughput (100MB) | 97.28 (97%) MB/sec | 97.27 (97%) MB/sec |
P20 data disk IOPS (2300) | 2345.75 (102%) IOPS | 2342.84 (102%) IOPS |
P20 data disk throughput (150MB) | 145.90 (97%) MB/sec | 145.91 (97%) MB/sec |
P30 data disk IOPS (5000) | 5099.80 (102%) IOPS | 5100.67 (102%) IOPS |
P30 data disk throughput (200MB) | 192.07 (96%) MB/sec | 194.54 (97%) MB/sec |
CPU average in % when creating 20 GB fixed VHD on data disk | 0.272% | 0.221% |
Conclusion / Take aways:
Looking at the test results, without a surprise, using SSE does not have a performance impact on Azure IaaS VMs. Because SSE runs on the Azure platform itself and not in the VM, you have full performance without resigning a necessary security option.
The results for ADE differ a bit comparing VMs with standard managed disks and VMs with premium managed disks. Starting with VMs with standard managed disks and looking at the results, ADE adds up to 3% more CPU load. Further explanation should not be necessary, because we are using BitLocker in the VM itself. So the CPU has to deal with encrypting and decrypting data and that adds the additional CPU load. IOPS and throughput values of the disks are not effected by ADE.
Using ADE on VMs with premium managed disks also adds up to 3% more CPU load. IOPS and throughput values of the P20 and P30 disk sizes are not affected by ADE. Surprisingly the throughput values for P10 disks with ADE are significantly lower compared to the result without ADE. Looking at the results, the impact is nearly 30% and I have neither a clue nor an explanation why this happens with a P10 running a 100% write test. IOPS values are alright for the P10.
Those test results led me to run even more tests for the P10 with different settings for the write test. Comparing those results I can recommend for write intensive workloads that depend on throughput and not on IOPS to use a P20 or P30 disk. For IOPS intensive workloads you can also use a P10 beside the other ones. In general, looking at the results, a P10 works best for general workloads having 30% write and 70% read operations. For all other workloads with higher percentage of write operations it is better to use a P20 or P30, when throughput is required rather than IOPS.
I hope the blog post is helpful for you and you get the necessary information about how encryption impacts the performance of an Azure IaaS VM. Do not forget, the results are not an official statement by Microsoft.