Poor SSL VPN performance when using TCP

Shouldnt be the case, as i tested it with Sophos Connect 2.0 back in the days on multiple devices. 

Do you use Compression on SSLVPN? 

Did you try Sophos Connect 2.0 or the OpenVPN Client? 

Did you only try SMB? Can you try other protocols, as SMB can actually cause such problems (re transmissions). 

Likely caused by MTU Issues: https://forums.openvpn.net/viewtopic.php?t=25039

I really need this to get sorted out, otherwise we will stop deploying XG firewalls to our customers. Since it has nothing to do with SMB and SG Firewall is using the very same MTU size, it should be something else going on.

Could we try to get at least some related information.

As you can reproduce this on multiple appliances, i am curious, how those appliances could be related.

Maybe something like "all appliances are sitting behind another router from provider X", "All Appliances are installed with a ISO from Firmware X". 

Hello,

these appliances belong to different customers, so they have nothing in common but being XG firewalls. I thought this would be clear as i wrote that i tested them with different internet connections. The XG210 is sitting behind another router that is configured to pass all traffic to it, the others are connected directly, thus holding the WAN IP too. They have been installed on different times, with different images, but now sharing roughly the same version with 18.0 GA-Build379 being the oldest one, since i am keeping all appliances up-to-date. The XG210 i gave you access to was installed on 11th march this year with the then up-to-date image. The XG125 was installed years ago and my virtual applicance was just installed to test this, so just some days ago. Theoretically i have much more XG firewalls to test, but those are not connected to internet connections fast enough.

More interestingly: Did you try to reproduce this? I really can't imagine that this is was not reproducable at all.

I did another test with a fresh installed virtual XG. I used the latest software .iso from here: https://community.sophos.com/products/xg-firewall/b/blog/posts/xg-firewall-v18-mr1-build396
It is connected to my home connection (1000 Mbit/s Down, 53 Mbit/s Up) and holds my public WAN IP. No other routers are involved. I even didn't register or activate it.

Everything i configured was the following:

01. Configured default CA (for obvious reasons)
02. Created a firewall rule to allow all traffic from VPN zone to ANY
03. Created a SSL VPN remote access profile for use as Default Gateway
04. Created a testuser
05. Added testuser to group "Open Group" and selected the remote access profile created in step 3
06. Downloaded SSL VPN Client from Userportal and installed it on a machine hosted in a remote site
07. Connected VPN and went to https://fast.com for a speedtest. Result: 14 Mbit/s
08. Changed default SSL Settingsfrom TCP to UDP
09. Downloaded new config from userportal and added it to SSL VPN client installed in step 6
10. Connected VPN and went to https://fast.com for a speedtest. Result: 50 Mbit/s

There you have it folks. If anyone at sophos declares this as not reproducible, i can just shake my head and go back to UTM or another vendor. Maybe I'll keep doing your work and install a fresh UTM now to do the same test again, already knowing what is going to happen, because i used UTM for years with TCP SSL VPN.

This is 100% reproducible, I've just tested myself with 400/200 WAN connection, with TCP I can't go above 25Mbit/s while a single core of my XG spikes to 100%, and on UDP I can reach the full 200Mbit/s.

This has been happening since v17.5, I hope It's fixed soon.

I did another test with the VPN Client downloaded from a UTM appliance, but with the config from my test XG. Same results. I guess they are both the same anyway. I also did a test with the official OpenVPN beta client (openvpn-connect-3.1.3.713_signed). Same results, so this is not related to client software at all. My UTM reaches the full 53 Mbit/s with all clients i tested on TCP SSL VPN connections.

SFVH_SO01_SFOS 18.0.1 MR-1-Build396# openvpn --version
OpenVPN 2.3.6 i486-openwrt-linux-gnu

 

Jesus Christ, can any Developer at Sophos at least update OpenVPN for a more recent version? Seriously, 2.3.6 came in DECEMBER/2014, It didn't even had support for AES-GCM.

My link speed is 50/20.

I ran the test on my iPad which uses the http proxy without decrypt and scan and it returns 8mb/s where as if I use speedof.me I get 20mb/s

When I use the site in the thread on my mac mini I get 50mb/s and when I rune speedof.me I also get 50mb/s and speediest.net (TCP 8080) I get 49-50 mb/s.

I don't have a remote site to setup a VPN to run further tests.

So, that leaves the VPN software as a likely suspect.

Ian

Hi folks,

i can totally confirm the 16Mb/s limit with vpn ssl TCP, tried remotely from two gibabit fiber equiped sites...

OpenVPN Software is not the cause of this problem, since it is running fine with other appliances, even with Sophos own SG Firewall, which is running UTM by default. It is clearly XG Firmware related.

I am still looking into this in my personal time, because i am curious what is going on. 

As i am able to reproduce this limitation for one client, it seems like this limitation is not shared with multiple connections at the same time. 
So i can actually connect multiple TCP based SSLVPN Clients from different locations and get a high performance output. 

NET | tun0 1132% | pcki 4025 | pcko 10334 | sp 10 Mbps | si 1509 Kbps | so 113 Mbps | | coll 0 | mlti 0 | erri 0 | erro 0 | drpi 0 | drpo 0

Just one output of a XG, which sends 113 mbit/s through TCP SSLVPN. 

There were multiple clients connected to this XG. 

As you already created a Support Case ID, i would expect to follow up with the Support as for now. 

I am still looking into this in my personal time, because i am curious what is going on. 

As i am able to reproduce this limitation for one client, it seems like this limitation is not shared with multiple connections at the same time. 
So i can actually connect multiple TCP based SSLVPN Clients from different locations and get a high performance output. 

NET | tun0 1132% | pcki 4025 | pcko 10334 | sp 10 Mbps | si 1509 Kbps | so 113 Mbps | | coll 0 | mlti 0 | erri 0 | erro 0 | drpi 0 | drpo 0

Just one output of a XG, which sends 113 mbit/s through TCP SSLVPN. 

There were multiple clients connected to this XG. 

As you already created a Support Case ID, i would expect to follow up with the Support as for now. 

this can not be a coincidence that all of us experience the exact same limitations at precisely 16Mb with so many differences of situations and configurations 

I can confirm, its the same here on my XG115w

Just around 11MBit/5MBit (with TCP & compression) on a 5GBit/5GBit line.

I am curious, which Clients you are using? 

Windows Only or did you try different OS systems? 

MacOS, Linux, iOS, Android?

Because i could observe far better results on a Linux (Openvpn) based system. 

Same configuration file -  Same Download. 

Lets not forget how this works: 

OpenVPN is getting a config file (OVPN). In this file there are general configuration settings to connect to the OpenVPN Server.

The openVPN Server (XG/SG) will push other information after the configuration is established. The reason, you do not have to regenerate and publish the config file every time, there is a change in the config on server end. (For Example new route or something like that). 

And please, everybody here with a valid Sophos Support Subscription - Open a Case. 

This needs to be tracked. 

With a markeshare of nearly 90% Windows is the platform to use of course. Which End User is using Linux at all?! Maybe test something relevant.

LuCar Toni said:

Windows Only or did you try different OS systems? 

I've used the OpenVPN Client that is available on the user portal in a Windows machine. Got pretty much the same results than everyone here.

LuCar Toni said:

Because i could observe far better results on a Linux (Openvpn) based system. 

This is correct, after this I've tried out on Linux machine with the latest version of OpenVPN, I've got 120Mbit/s over a single connection with TCP. (Still a lot slower than UDP.)

I've also tried the new Sophos Connect 2.0 EAP on windows, I've imported the .ovpn file on it and got similar results. (90Mbit/s).

Will there be any news on SC 2.0 EAP soon? It's not possible to deploy right now because of the captcha on the user portal.

Thanks!

Sophos Connect can actually import the OVPN file in 2.0. So you can simply import this file and it will work. The automatic policy attachment does not work, if you access the WAN User Portal. 

I've also tried the new Sophos Connect 2.0 EAP on windows, I've imported the .ovpn file on it and got similar results. (90Mbit/s).

That is odd - So you are saying, on Windows with "OpenVPN TCP" on Sophos Connect, the speed is "much higher" than the reported 10-20 mbit/s? 

As most people here report a cap of 2 mb/s (16 mbit/s), you are archiving a better result. 

Nothing of this explains why TCP SSL VPN is working without problems with UTM. It's the same OpenVPN client.

That is odd - So you are saying, on Windows with "OpenVPN TCP" on Sophos Connect, the speed is "much higher" than the reported 10-20 mbit/s? 

Sophos Connect can actually import the OVPN file in 2.0. So you can simply import this file and it will work.

That's exactly what I did!, here's some results. (I'm current limit to ~60Mbit/s on bad wireless.) (CPU usage on the client has less than 10%.)

Thanks!

I am not here to explain the issue, as i am still in the process of finding the root cause of this.

As i am trying to figure out, where this issue could sit, the first step is, to find the component, which is affected and which is not affected. 

If you know, there is only one platform affected, you can look deeper into the config pushed to this particular OS. If all OS are affected, it is likely caused by the server platform. 

As Linux seems to be able to handle this quite nicely, i am assuming, this could be caused by something pushed by XG to the Windows Clients and causing this. As openVPN interacts differently with the OS, there could be the Rootcause. 

There are different OS specific values like buffer caches and kernel buffers, which could likely cause this. 

Prism 

On the same client, as you posted your Sophos Connect results, deinstall the SC and install the OpenVPN Client, provided by XG, will you get the same results or not? 

On the same client, as you posted your Sophos Connect results, deinstall the SC and install the OpenVPN Client, provided by XG, will you get the same results or not? 

Stuck on 16Mbit/s as everyone in here.