Site-to-Site IPSEC Extremely Slow

Hi Chris,

Take SSH to XG and go to option 5.> 3. Advance Shell. Type:

cd /log

tail -f ips.log

Upload a file and monitor if something is dropped in the IPS logs. If there is no drop try changing the MSS value on the LAN and WAN interfaces to a lesser value. 

Awaiting response.

Thanks

Hi Sachin,

I checked the logs as you asked while running a file transfer, and nothing was being dropped in the IPS log.

Regarding the MSS setting, I tried lowering it from 1460 to 1300 on both ends (HQ and Branch) for the LAN interface and the WAN interface.  It did not make a difference.

Any other ideas why the IPSEC VPN speeds are so slow?  This is fairly business-critical as no one can transfer any files between the branch office or HQ with the current speeds.  It is about 10-12 times slower than it should be.  I have no restrictions or throttling in place on either the LAN to VPN or VPN to LAN policies on either end.

Hi Sachin,

The devices were connected directly to the XG interface when I initially set this up and it did not make a difference.  Now, I have a brand new L2 switch in the middle that all devices connect to and the switch is plugged into port 1 on my XG devices, but that's the only device in the picture.  It's only L2 so it doesn't do any throttling or anything.  Because I had this problem even when directly connected, I'm tempted to believe that's not an issue.  Additional, speeds are very fast all over except through the VPN tunnel.

I also restarted VPN services on both ends.  No difference.

I ran a ping test and traceroute through the tunnel and it is very fast... average is around 15-20ms:

Reply from 10.0.0.65: bytes=32 time=14ms TTL=253
Reply from 10.0.0.65: bytes=32 time=16ms TTL=253
Reply from 10.0.0.65: bytes=32 time=16ms TTL=253
Reply from 10.0.0.65: bytes=32 time=18ms TTL=253
Reply from 10.0.0.65: bytes=32 time=23ms TTL=253
Reply from 10.0.0.65: bytes=32 time=18ms TTL=253

Tracing route to 10.0.0.65 over a maximum of 30 hops

1 <1 ms <1 ms <1 ms 10.1.0.1
2 * * * Request timed out.
3 18 ms 20 ms 20 ms 10.0.0.65

I will try rebuilding the IPSEC policy on both ends... I'm not sure what else to do here.  The speeds are over 10x slower than they should be and I can't figure out why... Sophos support has not been helpful and just keep asking if it's an IPS issue which it is not.

Here's a SS of my VPN policy... as you can see, no traffic throttling, no IPS, no content filtering, nothing... 

This is the other end of the VPN... the branch office.  

I was able to completely fix this by turning off IPSEC VPN and setting up an SSL Site to Site VPN on both sides.  It took 1 minute to setup, and immediately worked flawlessly.  I have 10 Mbps upload and through the VPN tunnel I am getting 9.7Mbps.  Amazing.

So, this begs the question: Why is IPSEC VPN so terribly slow for me?  I have tried EVERYTHING... turning off compression, turning off PFS, going with the absolute most basic security protocols (DES, MD5, etc.).  It did not make a difference at all.  The max I could get with IPSEC VPN is 0.72 Mbps.  I cannot fathom why the difference is so large between the two VPNs... if SSL VPN was not an option I would have been in trouble. 

Hi Chris,

Wonderful! Provide us some time to recreate the instance and get back to you. You can also provide me the ticket# to look into the case if any progress is paused.

Thanks

Hi Sachin,

Thank you.  The ticket number is 6009528.

Thanks,

Chris

Just to be clear:

- Both offices are within 30ft of each other and have a latency of 15ms

- One office has an XG125, the other has an XG105.

- On the XG105 end, a L2 switch connects to port 1 and all devices plug into the L2 switch

- On the XG125 end, a L2 switch connects to port 1 and all devices plug into the L2 switch

- This has been tested with a workstation directly connected to port 1 instead of the switch; same results

- I have tested different MSS on both sides for the LAN and WAN interfaces.  I have not tweaked MTU.

- I have tried disabling compression and PFS

- I have tried setting up the following phase 1 AND phase 2 negotiations and the speeds were slow in each one:

     - DES / MD5

     - 3DES / MD5

     - AES128 / MD5

     - AES128 / SHA1

     - AES128 / SHA256

     - AES256 / SHA256

- I have tried most of the DH groups.

And both offices get 120 Mbps down / 10 Mbps up.

Speeds through IPSEC tunnel:

Read: 0.72Mbps
Write: 0.70Mbps

Speeds through SSL VPN tunnel:

Read: 9.8Mbps
Write: 9.8Mbps

Hey Sachin,

I am experiencing the same issue as Chris.  However I am having a hard time setting up an SSL Site to Site because the documentation is pretty brutal...

I have had 2 sophos xg firewalls for about 6 months and they have been miserable speed wise.  

Our main site is an XG135 with a voice server and 100/50mbps bandwidth.  Our satellite office across the street has an xg 115 with VoIP phones and 100/50mbps bandwidth.

Latency is all over the map and file transfers over the VPN are miserable.  Everything to the internet is working perfectly.

Can you point me in the direction of help?

Update: Sophos is still working on my ticket.  It sounds like they were able to replicate the horrible speed issue with using IPSEC Site to Site tunnels, but have not been able to find a workaround yet.

Update: Sophos is still working on my ticket.  It sounds like they were able to replicate the horrible speed issue with using IPSEC Site to Site tunnels, but have not been able to find a workaround yet.

ChrisWestmacott said:

Update: Sophos is still working on my ticket.  It sounds like they were able to replicate the horrible speed issue with using IPSEC Site to Site tunnels, but have not been able to find a workaround yet.

 

Hi ChrisWestmacott,

Any update from Sophos  ? :)

Hi

there are any updates from the support?

Thank you.

Regards.

I am also curious as to what support has to say on this.  What are some speeds that others are getting from Site to Site Ipsec tunnels and SSL VPN Tunnels?  

Hi Thomas,

Please DM us the case ID so we may check the issue, we would require our support to investigate this issue.

Our issue turned out to be HA MTU based. We're using a simple router to handle VDSL for a HA configuration. We had to reduce the MTU on the Sophos WAN ports that connect to that router. We'd moved to Site to Site SSL VPNs that worked around the performance issue, but now they're back on IPSEC they're working fine.

Hello,

I have posted a few weeks ago about IPSec speed being slower than expected (240 Mbps) on 1Gbps link.  My conclusion was that Sophos isn't using the AES-NI encryption chip when it is available, but no one could confirm this.  My tests were between software Sophos XG residing on VMs under ESXi 6.7, and also a XG 125 rev3 appliance.  I tested the same architecture between 2 pfSense VMs and with AES-NI active and I could get around 1Gbps (out of 5.5 Gbps of max bandwitdh on the virtual switch), and around 320 Mbps when AES-NI was deactivated.

Now I am testing the same IPSec performance between an XG 105 and 115 (bot rev3) in a private lab with 1Gbps link, and the max throughput I can get with the standard HQ/Branch policy is ~100Mbps for an Site-To-Site IPSec VPN.  Again, I am very concerned about Sophos not offloading to the AES-NI encryption cpu.