Disable bad bugfix in 9.405-5 "Fix [NUTM-2840]: [AWS] UTM ignores MTU sent by DHCP server"

So glad I came across this thread...  For the past 24 hours or so I had been troubleshooting a number of issues that seemed to crop up after updating UTM 9.404 -> 9.405 yesterday late in the evening.  The update itself seemed to go with little to no drama, and after it was done I noticed there was another update pending that I hadn't seen yet (9.406), so I set it for a scheduled install and went to bed.

Of note, this is a UTM home system running as my main firewall between the home network and Comcast, my ISP.

Anyway, the next day I noticed all kinds of issues. But one that was of main focus was wifi calling was no longer working on my Android device. (Oddly my wife's phone continued to work through all this, it's a BlackBerry Q10 running BB10 OS and my phone is a BlackBerry Priv running Android 6.0.1)  I really need wifi calling at home as I don't get a good signal to hold a call anywhere in the house, so I started investigating with packet captures from the UTM to see what was happening.  I found that pretty much every packet that my phone was sending out over UDP was getting a 'Destination Unreachable (Fragmentation Needed)' error.  Seemed odd to me as the packet in question was only 592 bytes in length.  Upon checking UTM's MTU for my WAN interface with ifconfig I noticed that it was only 576.  Ultimately this lead me to this post after a few hours of digging.

I did try just setting the MTU manually with ifconfig and while it updated properly, there's a lot more going on in the back end of UTM that did not recognize this change. Additionally trying to change the value at all within UTM's GUI resulted in it being reset to 576.  The fix listed in the OP however worked a treat and resolved all of the other issues I was having.

Overall this seems really odd to me, one thing that cropped up is that this effectively killed all IPv6 traffic through my ISP.  Heck, even the gateway on the WAN interface showed as '::' with this issue in place.  As I was digging through the IPv6 stuff I noticed that it gave me a warning about the minimum MTU needing to be at least 1280 (which it said it would automatically configure for me), and at the time I thought the message was odd as I knew it was set at 1500.  So the logic here seems a little flawed to just take what the ISP's DHCP server hands out without regard for any other rules in place. 

Hopefully anyone else affected by this finds this thread before they find insanity, it was just the right combination of things I noticed and google searches that led me here. Thank you Vegard for the clear and concise fix on this!!!

Well for some reason (I don't learn from my past mistakes??) I applied the 407-3 update last night, after skipping 406. I had already applied Vegard's fix. After applying 407-3 the issue reappeared for me, this time my default.conf file looked like this:

interface "[<INTERFACE>]" {
timeout 20;
retry 60;
script "/usr/sbin/dhcp_updown.plx";
request subnet-mask, broadcast-address, time-offset,
routers, domain-name, domain-name-servers, host-name,
domain-search, nis-domain, nis-servers,
ntp-servers[<INTERFACE_MTU>];
[<HOSTNAME>]
}

Removing '  [<INTERFACE_MTU>]  ' resolved the issue again.

Well for some reason (I don't learn from my past mistakes??) I applied the 407-3 update last night, after skipping 406. I had already applied Vegard's fix. After applying 407-3 the issue reappeared for me, this time my default.conf file looked like this:

interface "[<INTERFACE>]" {
timeout 20;
retry 60;
script "/usr/sbin/dhcp_updown.plx";
request subnet-mask, broadcast-address, time-offset,
routers, domain-name, domain-name-servers, host-name,
domain-search, nis-domain, nis-servers,
ntp-servers[<INTERFACE_MTU>];
[<HOSTNAME>]
}

Removing '  [<INTERFACE_MTU>]  ' resolved the issue again.

If you were bitten by this bug again after installing 9.407 you should ready this link:

 https://community.sophos.com/products/unified-threat-management/f/52/t/80641

Sophos has added an option to disable auto MTU discovery... It's turned off by default (aka, your ISP MTU is used) and didn't bother to tell anyone about that change until we started complain in that forum string.

So I applied the update and then it set the MTU back to 576. I was not able to browse to this site to read anything. Ended up having to change the config file again.

Why would they need to do that? The MTU is under the advanced options on the interface. If I modify that setting it should override everything else......

Jasin, not every Interface type has that option.  If you got changed to 576, then that's an Interface without an MTU setting option.

Cheers - Bob

PS I'm curious, guys, does this change survive a reboot?

Not sure what you mean but mine has an MTU option. No matter what I set it to, it would revert back. When I made the change listed above i was able to manually set it to 1500 just fine.

BAlfson said:

Jasin, not every Interface type has that option.  If you got changed to 576, then that's an Interface without an MTU setting option.

Cheers - Bob

PS I'm curious, guys, does this change survive a reboot?

 

We are talking about DHCP interface type, under advanced there is absolutely an MTU option, you can change it manually from 576 back to 1500 but when you save the config the interface releases and renews automatically and gets the bad 576 option again (via dhcp of course).

So, no the setting doesn't even apply much less survive a reboot.

Unless you're talking about hacking the DHCPC default.conf file via shell to remove respect for the MTU dhcp option. Yes it does survive a reboot. Just not another bad bugfix :P

Paul, Astaro ACE 10/2004

Just to let you know, 9.407-3 is released, they have implemented a new shell command in "CC" to ignore the MTU setting af the DHCP server, follow the instructions I made here:

https://community.sophos.com/products/unified-threat-management/f/hardware-installation-up2date-licensing/80641/sophos-utm-9-407-3-released

And you are good to go :-)

The DHCP interface type disappeared with about 9.1, I think, maybe 9.2.  If you're on 9.4 and you have a DHCP option or you have a "Dynamic" Ethernet interface with an 'MTU' setting available, I think you're in need of a re-image from ISO.

Cheers - Bob

Again, This has always been available even with new builds. And the setting works with the removal of the config

Oh, I see. Me feels so foolish now. You're absolutely right, It's definitely not a DHCP interface. it's Ethernet with the new special "Dynamic IP" Protocol to auto-magically Configure the Host. That old DHCP stuff went out so long ago.

But what I don't understand now is how modifying the default.conf file in the dhcpc folder affects the new DynIP protocol???? And why the people that are following this thread running an un-patched 9.1 UTM are somehow affected by the 576 MTU bugfix from 9.405-5 on service providers that lease IP addresses via DHCP????

Please help! I'm so confused.

"But what I don't understand now is how modifying the default.conf file in the dhcpc folder affects the new DynIP protocol??"

Because the "new DynIP" is nothing more than the plain old DHCP client.

There are two fixes available for this issue: one is a workaround, from this thread here, on which you edit default.conf and remove the instruction for the DHCP client do accept the MTU option provided by the DHCP server. The issue with this approach is that if the default.conf file is replaced by a future update, as it has happened after upgrading to 9.407-3, this workaround ceases to work.

The other fix, from the thread over https://community.sophos.com/products/unified-threat-management/f/hardware-installation-up2date-licensing/80641/sophos-utm-9-407-3-released, is a "proper fix" that was introduced after 9.407-3 and should survive any future upgrades (even through I agree with others that think this should be available on Webadmin as an option under "Dynamic IP", and not buried under a shell configuration, it's still a fix).

"And why the people that are following this thread running an un-patched 9.1 UTM are somehow affected by the 576 MTU bugfix from 9.405-5 on service providers that lease IP addresses via DHCP?"

I don't know from where you got that. AFAIK, there has been no reports from this specific MTU issue before 9.405-4, specially because the behavior of the DHCP client was modified from this version on to allow higher MTU from AWS for customers running UTM on Amazon. Until 9.403, any MTU option provided by the DHCP server was simply ignored by UTM. Again, AFAIK. 

Regards - Giovani