Hardware Acceleration (AES-NI) Isn't being used on the Software version of XG v18

Hi everyone,


Currently the Software version of XG v18 (And v17.5.x) Isn't using AES-NI for It's hardware acceleration on OpenSSL and OpenVPN; Comparing two machines with the same CPU and RAM, but one running XG v18 MR-1 and another Arch Linux we can see a huge difference in throughput in anything related to encryption.

Both machines were using a Intel G5400.


I've verified and XG have all AES kernel modules loaded on it:


    SFVH_SO01_SFOS 18.0.1 MR-1# lsmod | grep "aes"
    aesni_intel           163840  1
    glue_helper            16384  1 aesni_intel
    aes_x86_64             20480  1 aesni_intel
    crypto_simd            16384  1 aesni_intel
    cryptd                 20480  2 aesni_intel,crypto_simd

    SFVH_SO01_SFOS 18.0.1 MR-1# cat /proc/crypto | grep -m1 -o "aes"
    aes

    SFVH_SO01_SFOS 18.0.1 MR-1# cat /proc/crypto | grep -m1 -o "aesni"
    aesni

    SFVH_SO01_SFOS 18.0.1 MR-1# cat /proc/cpuinfo | grep -m1 -o "aes"
    aes




Let's look at the difference between OpenSSL Throughput's now; We can use two commands, one to test with AES-NI hardware acceleration, and another explicit disabling AES-NI for the performance test.

AES-NI Enabled: "openssl speed -evp aes-128-cbc"

AES-NI Disabled: "OPENSSL_ia32cap="~0x200000200000000" openssl speed -evp aes-128-cbc"


First on XG:



    SFVH_SO01_SFOS 18.0.1 MR-1# openssl speed -evp aes-128-cbc
    Doing aes-128-cbc for 3s on 16 size blocks: 24786330 aes-128-cbc's in 2.92s
    Doing aes-128-cbc for 3s on 64 size blocks: 7583445 aes-128-cbc's in 2.94s
    Doing aes-128-cbc for 3s on 256 size blocks: 2013992 aes-128-cbc's in 2.92s
    Doing aes-128-cbc for 3s on 1024 size blocks: 509478 aes-128-cbc's in 2.96s
    Doing aes-128-cbc for 3s on 8192 size blocks: 64313 aes-128-cbc's in 2.96s
    OpenSSL 1.0.2r-fips  26 Feb 2019
    built on: reproducible build, date unspecified
    options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)  
    compiler: ccache_cc -m32 -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/target-x86_64_glibc/usr/include -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/target-x86_64_glibc/include -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/toolchain-x86_64_gcc-7.3.0_glibc/usr/include -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/toolchain-x86_64_gcc-7.3.0_glibc/include -znow -zrelro -DOPENSSL_NO_HEARTBEATS -DTERMIOS -fpic -Wa,--noexecstack -O3 -fomit-frame-pointer -Wall -fomit-frame-pointer -Wall -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/target-x86_64_glibc/usr/lib/fips-i386/include
    The 'numbers' are in 1000s of bytes per second processed.
    type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
    aes-128-cbc     135815.51k   165081.80k   176569.16k   176251.85k   177990.57k


    SFVH_SO01_SFOS 18.0.1 MR-1# OPENSSL_ia32cap="~0x200000200000000" openssl speed -evp aes-128-cbc
    Doing aes-128-cbc for 3s on 16 size blocks: 26476585 aes-128-cbc's in 2.96s
    Doing aes-128-cbc for 3s on 64 size blocks: 7672939 aes-128-cbc's in 2.92s
    Doing aes-128-cbc for 3s on 256 size blocks: 1936728 aes-128-cbc's in 2.92s
    Doing aes-128-cbc for 3s on 1024 size blocks: 511127 aes-128-cbc's in 2.94s
    Doing aes-128-cbc for 3s on 8192 size blocks: 64474 aes-128-cbc's in 2.92s
    OpenSSL 1.0.2r-fips  26 Feb 2019
    built on: reproducible build, date unspecified
    options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)
    compiler: ccache_cc -m32 -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/target-x86_64_glibc/usr/include -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/target-x86_64_glibc/include -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/toolchain-x86_64_gcc-7.3.0_glibc/usr/include -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/toolchain-x86_64_gcc-7.3.0_glibc/include -znow -zrelro -DOPENSSL_NO_HEARTBEATS -DTERMIOS -fpic -Wa,--noexecstack -O3 -fomit-frame-pointer -Wall -fomit-frame-pointer -Wall -I/srv/jenkins/workspace/OmC/CI_64/staging_dir/target-x86_64_glibc/usr/lib/fips-i386/include
    The 'numbers' are in 1000s of bytes per second processed.
    type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
    aes-128-cbc     143116.68k   168174.01k   169795.33k   178025.19k   180880.48k




Now on Arch Linux:



    $ openssl speed -evp aes-128-cbc
    Doing aes-128-cbc for 3s on 16 size blocks: 145734524 aes-128-cbc's in 2.78s
    Doing aes-128-cbc for 3s on 64 size blocks: 59272123 aes-128-cbc's in 2.74s
    Doing aes-128-cbc for 3s on 256 size blocks: 15320112 aes-128-cbc's in 2.74s
    Doing aes-128-cbc for 3s on 1024 size blocks: 3853467 aes-128-cbc's in 2.74s
    Doing aes-128-cbc for 3s on 8192 size blocks: 485119 aes-128-cbc's in 2.76s
    Doing aes-128-cbc for 3s on 16384 size blocks: 244213 aes-128-cbc's in 2.82s
    OpenSSL 1.1.1f  31 Mar 2020
    built on: Tue Mar 31 17:04:42 2020 UTC
    options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr)
    compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG -D_FORTIFY_SOURCE=2
    The 'numbers' are in 1000s of bytes per second processed.
    type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
    aes-128-cbc     838759.85k  1384458.35k  1431368.13k  1440127.81k  1439889.44k  1418860.21k


    $ OPENSSL_ia32cap="~0x200000200000000" openssl speed -evp aes-128-cbc
    Doing aes-128-cbc for 3s on 16 size blocks: 57147307 aes-128-cbc's in 2.98s
    Doing aes-128-cbc for 3s on 64 size blocks: 16715714 aes-128-cbc's in 2.99s
    Doing aes-128-cbc for 3s on 256 size blocks: 4296403 aes-128-cbc's in 2.98s
    Doing aes-128-cbc for 3s on 1024 size blocks: 1100749 aes-128-cbc's in 3.00s
    Doing aes-128-cbc for 3s on 8192 size blocks: 138102 aes-128-cbc's in 2.98s
    Doing aes-128-cbc for 3s on 16384 size blocks: 68775 aes-128-cbc's in 3.00s
    OpenSSL 1.1.1f  31 Mar 2020
    built on: Tue Mar 31 17:04:42 2020 UTC
    options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr)
    compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wa,--noexecstack -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG -D_FORTIFY_SOURCE=2
    The 'numbers' are in 1000s of bytes per second processed.
    type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
    aes-128-cbc     306831.18k   357794.55k   369086.97k   375722.33k   379641.47k   375603.20k




On Arch Linux we can see a huge difference between the speeds while using and not using AES-NI. But in XG we're stuck at the same speeds.

As stated on OpenSSL and OpenVPN Mailing Lists, "If the results or the two above commands are equal, then your openssl library does NOT use hardware crypto."


With OpenVPN, comparing --test-crypto between the two machines gives even "worse" results:

Arch Linux:


    $ time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-cbc
    Thu Apr 16 00:48:16 2020 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
    openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher   1.16s user 0.02s system 98% cpu 1.194 total

    3200/1.16 = 2,758 Mbit/s // Theorical Throughput | Real World Throughput is >860Mbit/s (Tested with Iperf3 but physically limited to 1G.)



XG v18:



    SFVH_SO01_SFOS 18.0.1 MR-1# time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-cbc
    real    0m 13.09s
    user    0m 12.90s
    sys     0m 0.04s


    3200 / 13.09 = 244 Mbit/s // Theorical Throughput | Real World Throughput is 220Mbit/s (Tested with Iperf3.)




I've had a talk with  last month about this, on his machine he got the same results, as AES-NI isn't being utilized.

Is there any reason for those results difference between both machines? Or this is a local issue on my installation?


Thanks!

  • In reply to FelixSteinbeis:

    Hi,

    thank you for that information.

    The software version does not support hardware-acceleration, the option is not in the system list of commands. The system commands do not seem to function very well, I will try a different OS machine.

    Ian

  • In reply to suzzyx:

    So, QAT indeed is used by IPsec,

    But what about SSL VPN (OpenVPN), OpenSSL, and for the new DPI Engine on v18? Any news on that?

     

    Thanks!

  • In reply to Vishal_R:

    Any news from the Dev team?

     

    Thanks!

  • In reply to Prism:

    Hello.

    As confirmed that the AES-NI is not supported on the software version and is only available on the following hardware appliance. 

    CR2500iNG_WP01 CR2500iNG_WP02 CR2500iNG_WP03 CR2500iNG-XP_WP01 CR1500iNG-XP_WP01 CR1000iNG-XP_WP01 SF01V_SO01 VCR01V_OE01 SG105_XN01 SG105_XN02 SG105w_XN01 SG105w_XN02 SG115_XN01 SG115_XN02 SG115w_XN01 SG115w_XN02 SG125_XN01 SG125_XN02 SG125w_XN01 SG125w_XN02 SG135_XN01 SG135_XN02 SG135w_XN01 SG135w_XN02 SG210_WP01 SG210_WP02 SG210_WP03 SG230_WP01 SG230_WP02 SG310_WP01 SG310_WP02 SG330_WP01 SG330_WP02 SG430_WP01 SG430_WP02 SG450_WP01 SG450_WP02 SG550_XN01 SG550_RL02 SG650_XN01 SG650_RL02 XG85_XN01 XG85w_XN01 XG105_XN02 XG105w_XN02 XG115_XN02 XG115w_XN02 XG115w-VCR_XN02 XG125_XN02 XG125a_XN02 XG135_XN02 XG125w_XN02 XG135w_XN02 XG85_AM02 XG85w_AM02 XG86_AM01 XG86w_AM01 XG105_XN03 XG105w_XN03 XG106_XN01 XG106w_XN01 XG115_XN03 XG115w_XN03 XG125_XN03 XG135_XN03 XG125w_XN03 XG135w_XN03 SG105_XN03 SG105w_XN03 SG115_XN03 SG115w_XN03 SG125_XN03 SG125w_XN03 SG135_XN03 SG135w_XN03 XG210_WP02 XG210_WP03 XG230_WP01 XG230_WP02 XG310_WP01 XG310_WP02 XG330_WP01 XG330_WP02 XG430_WP01 XG430_WP02 XG450_WP01 XG450_WP02 XG550_XN01 XG650_XN01 XG750_RL01 XG550_RL02 XG650_RL02 XG750_RL02
  • In reply to Aditya Patel:

    Thank you so much for the answer!

  • In reply to Aditya Patel:

    Hi Aditya,

    are there any plans to use this performance enhancing feature in future releases of XG?

    Ian

  • In reply to Aditya Patel:

    Hey Aditya your post with the list of supported platforms is not word wrapping in the post so we can't see it... can you repost that list?

     

    Thanks!

  • In reply to BrucekConvergent:

    Hi  

    Please find the requested information :

    Only below appliances use (AES-NI).

    CR2500iNG_WP01 CR2500iNG_WP02 CR2500iNG_WP03 CR2500iNG-XP_WP01 CR1500iNG-XP_WP01 CR1000iNG-XP_WP01 SF01V_SO01 VCR01V_OE01 SG105_XN01 SG105_XN02 SG105w_XN01 SG105w_XN02 SG115_XN01 SG115_XN02 SG115w_XN01 SG115w_XN02 SG125_XN01 SG125_XN02 SG125w_XN01 SG125w_XN02 SG135_XN01 SG135_XN02 SG135w_XN01 SG135w_XN02 SG210_WP01 SG210_WP02 SG210_WP03 SG230_WP01 SG230_WP02 SG310_WP01 SG310_WP02 SG330_WP01 SG330_WP02 SG430_WP01 SG430_WP02 SG450_WP01 SG450_WP02 SG550_XN01 SG550_RL02 SG650_XN01 SG650_RL02 XG85_XN01 XG85w_XN01 XG105_XN02 XG105w_XN02 XG115_XN02 XG115w_XN02 XG115w-VCR_XN02 XG125_XN02 XG125a_XN02 XG135_XN02 XG125w_XN02 XG135w_XN02 XG85_AM02 XG85w_AM02 XG86_AM01 XG86w_AM01 XG105_XN03 XG105w_XN03 XG106_XN01 XG106w_XN01 XG115_XN03 XG115w_XN03 XG125_XN03 XG135_XN03 XG125w_XN03 XG135w_XN03 SG105_XN03 SG105w_XN03 SG115_XN03 SG115w_XN03 SG125_XN03 SG125w_XN03 SG135_XN03 SG135w_XN03 XG210_WP02 XG210_WP03 XG230_WP01 XG230_WP02 XG310_WP01 XG310_WP02 XG330_WP01 XG330_WP02 XG430_WP01 XG430_WP02 XG450_WP01 XG450_WP02 XG550_XN01 XG650_XN01 XG750_RL01 XG550_RL02 XG650_RL02 XG750_RL02


  • In reply to rfcat_vk:

    Yesterday I opened a feature request regarding AES-NI support:

    https://ideas.sophos.com/forums/330219-xg-firewall/suggestions/40424218-hardware-acceleration-aes-ni-for-xg-software-ver

    You are all welcome to upvote this. Wink

  • In reply to TheBalmasque:

    Thank you for creating the feature request, voted for it.

    Ian

  • Hello all,

    I have received a response that this is already planned to be addressed in v18 MR3 so that it will utilise AES-NI on support CPUs in Software installations. Please take note that this is planned and not confirmed.

    If you'd like to keep an eye out for it in the release notes, the Jira ID is NC-59127.

    Emile

  • In reply to EmileBelcourt:

    Wow, great news. I hope this get confirmed by Sophos soon. Cool