Site-to-Site IPSEC Extremely Slow

I have an IPSEC tunnel established between two sites that are within 30ft of each other (the buildings are next door).  Both sites get 100Mbps down / 10 Mbps up.  I setup an IPSEC tunnel between both sites using the default configuration of DefaultHeadOffice and DefaultBranchOffice in the IPSEC settings.  I have policies allowing LAN to VPN and VPN to LAN.  Everything is all pretty basic.

Once I setup the tunnel, I tried to do a simple file transfer of one 20MB file between a branch workstation and a server at Head Office.  It transferred the file at a speed of 0.7Mbps.  Considering both sites get 10Mbps upload, and given some overhead for the VPN tunnel, I would expect the speeds to be at least 7 or 8 Mbps, not 0.7....  Does anyone else have any experiences of insanely slow site-to-site IPSEC tunnels or have any recommendations?

The Head Office has an XG125 and remote office has an XG105 running MR2.  Both are at 50% memory usage and between 0-10% CPU usage.

  • Hi Chris,

    Take SSH to XG and go to option 5.> 3. Advance Shell. Type:

    cd /log

    tail -f ips.log

    Upload a file and monitor if something is dropped in the IPS logs. If there is no drop try changing the MSS value on the LAN and WAN interfaces to a lesser value. 

    Awaiting response.

    Thanks

  • In reply to sachingurung:

    Hi Sachin,

    I checked the logs as you asked while running a file transfer, and nothing was being dropped in the IPS log.

    Regarding the MSS setting, I tried lowering it from 1460 to 1300 on both ends (HQ and Branch) for the LAN interface and the WAN interface.  It did not make a difference.


    Any other ideas why the IPSEC VPN speeds are so slow?  This is fairly business-critical as no one can transfer any files between the branch office or HQ with the current speeds.  It is about 10-12 times slower than it should be.  I have no restrictions or throttling in place on either the LAN to VPN or VPN to LAN policies on either end.

  • In reply to ChrisWestmacott:

    Rebooted both firewalls... still incredibly slow.

    I can try building a new IPSEC policy tomorrow I guess, but the current one I am using is not fancy (HQ is using DefaultHeadOffice profile and BO is using DefaultBranchOffice profile).  Very frustrated by these performance issues.

  • In reply to ChrisWestmacott:

    Also upgraded both firewalls to MR-3.  No change in performance.

  • In reply to ChrisWestmacott:

    Hi Chris,

    Connect a device directly to XG interface on both the ends and configure an IPSec Policy. Check what is the throughput you receive with this architecture. 

    Also, take SSH to XG and go to option 6. VPN Management > 2. Restart VPN services.

    Thanks

  • In reply to sachingurung:

    Hi Sachin,

    The devices were connected directly to the XG interface when I initially set this up and it did not make a difference.  Now, I have a brand new L2 switch in the middle that all devices connect to and the switch is plugged into port 1 on my XG devices, but that's the only device in the picture.  It's only L2 so it doesn't do any throttling or anything.  Because I had this problem even when directly connected, I'm tempted to believe that's not an issue.  Additional, speeds are very fast all over except through the VPN tunnel.

    I also restarted VPN services on both ends.  No difference.

    I ran a ping test and traceroute through the tunnel and it is very fast... average is around 15-20ms:

    Reply from 10.0.0.65: bytes=32 time=14ms TTL=253
    Reply from 10.0.0.65: bytes=32 time=16ms TTL=253
    Reply from 10.0.0.65: bytes=32 time=16ms TTL=253
    Reply from 10.0.0.65: bytes=32 time=18ms TTL=253
    Reply from 10.0.0.65: bytes=32 time=23ms TTL=253
    Reply from 10.0.0.65: bytes=32 time=18ms TTL=253

    Tracing route to 10.0.0.65 over a maximum of 30 hops

    1 <1 ms <1 ms <1 ms 10.1.0.1
    2 * * * Request timed out.
    3 18 ms 20 ms 20 ms 10.0.0.65

    I will try rebuilding the IPSEC policy on both ends... I'm not sure what else to do here.  The speeds are over 10x slower than they should be and I can't figure out why... Sophos support has not been helpful and just keep asking if it's an IPS issue which it is not.

  • In reply to ChrisWestmacott:

    Here's a SS of my VPN policy... as you can see, no traffic throttling, no IPS, no content filtering, nothing... 

  • In reply to ChrisWestmacott:

    This is the other end of the VPN... the branch office.  

  • In reply to ChrisWestmacott:

    I was able to completely fix this by turning off IPSEC VPN and setting up an SSL Site to Site VPN on both sides.  It took 1 minute to setup, and immediately worked flawlessly.  I have 10 Mbps upload and through the VPN tunnel I am getting 9.7Mbps.  Amazing.

    So, this begs the question: Why is IPSEC VPN so terribly slow for me?  I have tried EVERYTHING... turning off compression, turning off PFS, going with the absolute most basic security protocols (DES, MD5, etc.).  It did not make a difference at all.  The max I could get with IPSEC VPN is 0.72 Mbps.  I cannot fathom why the difference is so large between the two VPNs... if SSL VPN was not an option I would have been in trouble. 

  • In reply to ChrisWestmacott:

    Hi Chris,

    Wonderful! Provide us some time to recreate the instance and get back to you. You can also provide me the ticket# to look into the case if any progress is paused.

    Thanks

  • In reply to sachingurung:

    Hi Sachin,


    Thank you.  The ticket number is 6009528.


    Thanks,


    Chris

  • In reply to ChrisWestmacott:

    Just to be clear:

    - Both offices are within 30ft of each other and have a latency of 15ms

    - One office has an XG125, the other has an XG105.

    - On the XG105 end, a L2 switch connects to port 1 and all devices plug into the L2 switch

    - On the XG125 end, a L2 switch connects to port 1 and all devices plug into the L2 switch

    - This has been tested with a workstation directly connected to port 1 instead of the switch; same results

    - I have tested different MSS on both sides for the LAN and WAN interfaces.  I have not tweaked MTU.

    - I have tried disabling compression and PFS

    - I have tried setting up the following phase 1 AND phase 2 negotiations and the speeds were slow in each one:

         - DES / MD5

         - 3DES / MD5

         - AES128 / MD5

         - AES128 / SHA1

         - AES128 / SHA256

         - AES256 / SHA256

    - I have tried most of the DH groups.

  • In reply to ChrisWestmacott:

    And both offices get 120 Mbps down / 10 Mbps up.

    Speeds through IPSEC tunnel:

    Read: 0.72Mbps
    Write: 0.70Mbps

    Speeds through SSL VPN tunnel:

    Read: 9.8Mbps
    Write: 9.8Mbps

  • In reply to sachingurung:

    Hey Sachin,

    I am experiencing the same issue as Chris.  However I am having a hard time setting up an SSL Site to Site because the documentation is pretty brutal...

    I have had 2 sophos xg firewalls for about 6 months and they have been miserable speed wise.  


    Our main site is an XG135 with a voice server and 100/50mbps bandwidth.  Our satellite office across the street has an xg 115 with VoIP phones and 100/50mbps bandwidth.

    Latency is all over the map and file transfers over the VPN are miserable.  Everything to the internet is working perfectly.

    Can you point me in the direction of help?

  • In reply to JimBoyle:

    Update: Sophos is still working on my ticket.  It sounds like they were able to replicate the horrible speed issue with using IPSEC Site to Site tunnels, but have not been able to find a workaround yet.