This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

IPsec Site-to-Site VPN passes no traffic after updating to 9.406-3

Greetings, folks.

"Successfully" updated 1 x SG230 and 2 x SG115 to 9.406-3 this morning only to get reports that the site-to-site VPN was not working. Sure enough, no traffic is being passed across the IPsec VPN tunnels. The log files were showing errors suggestive of configuration mismatches, e.g, "max number of retransmissions (2) reached STATE_QUICK_I1. No acceptable response to our first Quick Mode message: perhaps peer likes no proposal". I tore down and re-created the remote gateways and IPsec connections.

After redoing the VPN configurations, the tunnels show connected and Live Log shows no errors whatsoever. The connections get their usual 'refreshes', everything is green and active. Alas, absolutely 0 bytes of data go back and forth. Accessing server shares, DNS, AD, etc., is not happening.


The client is being very good about things so far, but I need to fix this and I simply have no idea of where to start. The configuration is correct, the firewall rules are in place to allow traffic on all services and yet 0 traffic. All from one reboot to the next.

Ideas?



This thread was automatically locked due to age.
  • From which version did you upgrade? Version before 9.405? If so, you may suffer from the MTU DHCP bug where your provider sends a way too low MTU for your WAN connection (usually 1500).


    Managing several Sophos firewalls both at work and at some home locations, dedicated to continuously improve IT-security and feeling well helping others with their IT-security challenges.

    • apijnappels said:

      From which version did you upgrade? Version before 9.405? If so, you may suffer from the MTU DHCP bug where your provider sends a way too low MTU for your WAN connection (usually 1500).

      We upgraded from 9.504. As well, the units are on fixed IPs with MTU @ 1454 (the usual for Japan PPPoE connections). That WAN interfaces themselves work just fine. Only the VPN side of the connections are borked.

    • This might seem obvious, but you're not running into a firewall issue now are you? 

      • rsenio said:

        This might seem obvious, but you're not running into a firewall issue now are you? 

        One would sincerely hope not. The automatic firewall rules that were in place prior to the upgrade were working fine. After tearing down and rebuilding all the gateways and IPsec connections, check the rules were in place, same scenario. The firewall Live Log shows no packets being dropped for the s2s-vpns.

        I got nothin'.

      • This just goes from bad to worse.

        One of the s2s-vpns has been working sporadically for no known reason. It suddenly just quit completely and now the 'hq' firewall  is showing this in the live log:

        sendto on ppp0 to xxx.xxx.xxx.xxx:500 failed in main_outI1. Errno 1: Operation not permitted

        That tells me that the firewall is now blocking VPN traffic, yet the bloody HQ firewall has not seen a configuration change or a reboot.

        I love Sophos firewalls, but this firmware release seems to have been half-baked at best. I don't even have any hair to pull out in the frustration I'm experiencing here.

      • Hi Trane, 

        Your problem sounds very similar to mine (https://community.sophos.com/products/unified-threat-management/f/58/t/79896). Down to the fact that it does sporadically decide to start working. I do have an active case with Sophos which has been going for a few weeks now. Last i heard it has apparently been 'escalated'. 

        In my case we were  able to determine that it is actually only one side that has decided to not send anything down the tunnel (which makes it look like nothing at all is happening in the tunnel). I used espdump  (guide here: https://community.sophos.com/kb/en-US/115702) to monitor the traffic and can clearly see activity on and from the remote side, but nothing on the HQ side. 

        Is the same thing happening on yours?

        I know this doesn't offer a solution to your problem, but i'm thinking if it does turn out to be very similar issues it kind of does point to being a bug in the firmware. And maybe someone at Sophos will take note and speed things along a bit. 

        • Hi, Patrick.

          Well, well, well. That certainly seems to be a similar, if not identical, problem. Unfortunately, I don't have any easy means of running espdump as I administer this network from half-way across Japan. I've now got a support ticket open with the Japanese reseller and will escalate it to Sophos proper should it remain unresolved for any length of time. For the one branch office, I resorted to implementing network logon PPTP VPN on each of the desktops. It's a small office, so not really a bother.

          I do hope it gets sorted quickly, though. Thanks for your input!

          • Guys, have you tried restoring the backup done automatically before you applied these latest Up2Dates?  If that didn't straighten things out, does a reboot?

            Cheers - Bob

             
            Sophos UTM Community Moderator
            Sophos Certified Architect - UTM
            Sophos Certified Engineer - XG
            Gold Solution Partner since 2005
            MediaSoft, Inc. USA
            • Hi, Bob.

              Yeah, the very first thing I did was restore the configuration in case things got corrupted during the upgrade. Unfortunately, it made no difference.

              Cheers!

              • In that case, I think I'd push Sophos to escalate beyond the 2nd-level.

                Cheers - Bob

                 
                Sophos UTM Community Moderator
                Sophos Certified Architect - UTM
                Sophos Certified Engineer - XG
                Gold Solution Partner since 2005
                MediaSoft, Inc. USA
                • Thanks. So far, their response is that this can be resolved by 'normal troubleshooting'. I don't share their opinion. [li] :D

                  • I would tell them that you're not satisfied with their response and that you'd like to speak with the manager that's preventing the case from being escalated.  I'm a really nice guy, but ...

                    Cheers - Bob

                     
                    Sophos UTM Community Moderator
                    Sophos Certified Architect - UTM
                    Sophos Certified Engineer - XG
                    Gold Solution Partner since 2005
                    MediaSoft, Inc. USA
                    • Hi, Bob.

                      In the end, the reseller was in contact with Sophos support and the mutual conclusion was that reimaging the systems with older firmware will be required. That's unfortunate, as I am half-way across the country. I will now need to discuss the matter with the client.

                      Cheers and have a great weekend.

                      trane

                      • Hi Trane,

                        Can you please provide me the case# with support? I will personally look into it and provide you the information. Community members can always notify me when they have a case with support. 

                        Thanks

                        Sachin Gurung
                        Team Lead | Sophos Technical Support
                        Knowledge Base  |  @SophosSupport  |  Video tutorials
                        Remember to like a post.  If a post (on a question thread) solves your question use the 'This helped me' link.

                        • Thanks for the kind offer. I succeeded just a few hours ago in finally achieving active tunnels on both SG115s. The solution is annoyingly simple, but should not have been necessary. There seem to be a lot of overlapping semi-problems that got in the way of arriving at a timely solution. The client is currently happy that the tunnels are up, but they are now considering replacing the whole lot with a different vendor's product. Shots fired. Bang Bang.

                          In the end, the SG230 was the culprit. Creating the tunnels seemed to work fine, but the logs were peppered with various errors. The final solution involved:

                          • Tearing down ALL tunnels and their components, including destroying all network definitions;
                          • Creating new VPNs with a custom policy (to be compatible with another vendor's products);
                          • New VPN tunnels were created using MANUAL firewall rules.

                          The process was repeated on the SG115s, but with the difference that automatic firewall rules were fine.

                          For whatever reason, the SG230 now refuses to pass VPN traffic when automatic firewall rules are implemented. This was true even after re-imaging the firewall with 9.357 and restoring the configuration. What worked before, did not work now. I've spent almost 7 full workdays struggling with this issue and it includes flying halfway across the country and 4 days on-site. No longer a fan.

                          Yes, I'd like some cheese with my whine. I've had 8 hours of sleep total in the last 3 nights.

                          • This thread is a little like herpes: It's the gift that keeps on giving.

                            How long's it been? Six weeks? The tunnels died again. This time, it was after no updates or configuration changes whatsoever from the day of the reply above proclaiming my success in resolution.

                            What worked to fix things before no longer works. One tunnel is mysteriously (and happily) working, but the other resolutely shows green and passes no traffic.

                            This. Is. Ridiculous.

                            • Trane, via PM you mentioned that there's a fibre line involved.  In several locations with fibre, the solution was to have the ISP set fixed speed/duplex on their equipment and matching it on yours.  The problem was a conflict between the fibre router and the UTM when negotiating speed and duplex.  See #7.7 in Rulz for a description of the process.  Any luck with that?

                              Cheers - Bob

                               
                              Sophos UTM Community Moderator
                              Sophos Certified Architect - UTM
                              Sophos Certified Engineer - XG
                              Gold Solution Partner since 2005
                              MediaSoft, Inc. USA
                          x An error occurred. Please try again or contact your administrator.