5% upload speed on Sophos UTM

RE: 5% upload speed on Sophos UTM

Jay Jay — Wed, 28 Sep 2022 19:32:53 GMT

I see. I asked about static ip's earlier but didn't get a reply. Are you subscribed to static ip service? Trying to get a better understanding of your network topology.

RE: 5% upload speed on Sophos UTM

goose7791 — Wed, 28 Sep 2022 18:50:13 GMT

One for the business network and one for the home.

RE: 5% upload speed on Sophos UTM

Jay Jay — Wed, 28 Sep 2022 18:06:51 GMT

Glad you got it sorted but im still confused.

What purpose is UTM serving if you have a fw (firewall?) appliance?

RE: 5% upload speed on Sophos UTM

goose7791 — Wed, 28 Sep 2022 17:17:34 GMT

Firstly apologies to everyone that I left hanging. I as out of state for 2+ weeks and didn't want to make any changes to the setup for fear of borking the whole thing, which has happened on occasion.

@Jay Jay was on the right track with thinking it was on the ATT hardware side. As mentioned, I have a physical FW appliance and a virtual UTM. The ATT all in one modem, media converter, and switch, has 4 1GB ports and 1 5GB port. The 5GB port was connected to the physical FW appliance and the 1GB port was connected to the virtual infrastructure. I swapped ports and everything began working as expected. I am getting full throughput on the virtual and full gig on the physical. I have no idea why. Either the ATT modem does some funky MTU on the various ports or the core switch as part of my virtual infrastructure is going bad. Either way, not a Sophos UTM issue at all.

Thanks to everyone who messaged me on this. Such a great, intelligent community. Here's hoping the UTM sticks around for years to come.

RE: 5% upload speed on Sophos UTM

Jay Jay — Wed, 14 Sep 2022 20:39:10 GMT

Sounds like you got the new set up - bgw320. There's ways of bypassing that too but more complicated. Dslreports has a lengthy thread on the topic - https://www.dslreports.com/forum/r33442912-AT-T-Fiber-Bye-bye-802-1x-you-will-not-be-missed .

Only a single device behind the att gateway/media converter can get the public ip. Are you subscribed to multiple static ip's?

RE: 5% upload speed on Sophos UTM

BAlfson — Wed, 14 Sep 2022 15:37:25 GMT

I haven't tried the following with a VM, so be sure to get a good backup of your VM before you try changing from the command line.

1. Find the fastest speed&duplex that you can set fixed in the AT&T device and set that there. The below assumes that's 1 Gb full duplex

2. As root, get the REF_ of the hardware:

cc get_object_by_name itfhw ethernet 'eth1 VMWare VMXNET3 Ethernet Controller'| grep \'ref

3. Assuming that returns REF_ItfEthEth1VmwarVmxne, disable Auto negotiation with:

cc change_object REF_ItfEthEth1VmwarVmxne auto_negotiation_status 0

4. Change the speed to 1 Gb:

cc change_object REF_ItfEthEth1VmwarVmxne speed 1000

Did that work? Did it solve the speed problem?

Cheers - Bob

RE: 5% upload speed on Sophos UTM

goose7791 — Wed, 14 Sep 2022 01:55:18 GMT

So not sure I can do that. Before the ATT box is the actual fiber itself. The ATT box is a media converter. The fiber comes from the curb and goes directly into the ATT device which has multiple RJ45 ports on the back. One goes to a physical FW and one goes to a port on my core switch.

RE: 5% upload speed on Sophos UTM

goose7791 — Wed, 14 Sep 2022 01:53:16 GMT

RE: 5% upload speed on Sophos UTM

Jay Jay — Sat, 10 Sep 2022 19:41:45 GMT

No passthrough. Out of the picture entirely... removed, gone, does not exist.

That's why I suggested the dumb switch bypass just to rule out some weird issues between the gateway and the server nic.

RE: 5% upload speed on Sophos UTM

goose7791 — Sat, 10 Sep 2022 18:38:35 GMT

Oh, and yeah it's my home lab. Yeah I know crazy. I have Dell R720's as a homelab.

RE: 5% upload speed on Sophos UTM

goose7791 — Sat, 10 Sep 2022 18:36:52 GMT

I have the ATT box in passthrough mode as well. Public IP's are assigned to all FW's. The physical Unifi gets the full 1Gb so I don't think there is anything on the ATT that needs adjusting. And it's JUST uploads, too, so I don't get that.

I've moved the internal VMNIC between vswitches, too. The physical nics are enterprise grade. No RealTek. Broadcom and Intel enterprise 10GB SFP+ NICs. Maybe a firmware update messed them up? But no, if I can get full Gb from a VM by swapping to a physical FW that means the same connections can handle full gig both ways. Just typing out loud to see if my thinking makes any sense.

RE: 5% upload speed on Sophos UTM

BAlfson — Sat, 10 Sep 2022 18:32:33 GMT

The changes are in WebAdmin on the 'Hardware' tab of 'Interfaces'.

Cheers - Bob

RE: 5% upload speed on Sophos UTM

goose7791 — Sat, 10 Sep 2022 18:24:30 GMT

I haven't done UTM on a physical box yet. I think that is the next step after I make some virtual stack changes like swapping cables with the physical Unifi FW and dropping MTU on the switch and the ESXi port groups down to 1500. It really does feel like an MTU thing.

RE: 5% upload speed on Sophos UTM

goose7791 — Sat, 10 Sep 2022 18:11:57 GMT

I went looking but that is not an option as a VM. The only changes available to make are virtual MAC and does it look for HA traffic.

RE: 5% upload speed on Sophos UTM

Jay Jay — Sat, 10 Sep 2022 17:29:59 GMT

Alan, meant to comment on this earlier. I've had good results with i211/i219 and some other intel gigabit adapters.

Several boards purchased in the last year came with the i225-v. This adapter has had 3 hardware revisions and is still a POS in my opinion. Even with a rev3, i've had issues with it showing cable disconnected (while connected). Any manipulation of the adapter (disable/enable), causes mass system instability to the point where a full power disconnect is needed - reset/power button does not work.

Granted, this is under windows, not sure if linux is any better. There's numerous posts about this on intel's forum as well as reddit. It's bad enough that if I need/want onboard ethernet, I will pass on any board that comes with it. The rtl8125 on the other hand has worked flawlessly for me in both windows and linux. The other option is to disable the nic in bios and install a pcie nic. This may be a good alternative if you want quad port nic capability. Otherwise it wastes a slot.

My proxmox box has a i211 (pass through to utm), and the rtl8125 (used a vnic). This has been in operation since may 2021.

Hate to say it, but today prefer a rtl nic over intel given the above.

RE: 5% upload speed on Sophos UTM

BAlfson — Sat, 10 Sep 2022 16:36:08 GMT

Did you try modifying the speed&duplex settings as suggested above?

Cheers - Bob

RE: 5% upload speed on Sophos UTM

Jay Jay — Sat, 10 Sep 2022 14:35:27 GMT

I have a similar set up to the OP in terms of 1gb att fiber and virtualized utm.

Unless the OP forgot to mention any bypass, my config differs in that i'm using a full gateway bypass - the gateway box (bgw210) sits useless in a box on a shelf. This uses extracted certs and wpa_supplicant to handle 802.1x auth.

Also, the wan port is directly passed through to utm (i211).

Have you tested uploading directly from the utm shell? Not sure how as there's no ftp/tftp client. Maybe using curl or iperf3 (manually installed).

Is netstat -i or ifconfig reporting any ethernet errors?

The ubiquiti works properly with the same cables?

Given the OP is using a vnic for wan, it's worth exploring advanced configuration options for the vnic and vswitch. Maybe something got changed. Also check the network stack configuration. It's been about 18 months since I last used esxi. Esxi doesn't support the rtl8125 nic in my upgraded host (x570, 5800x cpu, 64gb ram).

You can also do a dumb switch bypass to test to ensure the gateway is not somehow affecting this - https://www.dslreports.com/forum/r32094182- .

This method still works if your area has not been upgraded to xgs-pon. This eliminates any issues the gateway may have with your nic.

I switched to proxmox in early 2019. Prior to that was running esxi with utm. Full bw upload/download (940/940 mbps). That was on a 6600K box where utm had 4 vcores assigned. Virtualizing utm is prob not best for large scale/enterprise applications but works very well for home lab.

RE: 5% upload speed on Sophos UTM

alan weir — Fri, 09 Sep 2022 23:54:31 GMT

Out of curiosity have you tried installing the UTM on the bare hard drives? Also Intel NICs are preferred over the others. You said the Dell server has Broadcom. Not sure if that is causing the issue. Since you were able to get the full 1Gbps on the Ubiquity firewalls VM on the same hardware, it seems everything is fine on your ISPs end.

Check firewall logs while you do speed tests on different sites. Is anything being blocked. I also had a thought that your upload speed is so fast it's triggering the TCP SYN/UDP flood alerts of the IPS. LOL. But you said it was disabled.

RE: 5% upload speed on Sophos UTM

goose7791 — Fri, 09 Sep 2022 23:23:03 GMT

If it ends up being something in the virtual stack that I cannot remediate, since this is a Home license and I want to have full gig both ways, what hardware device would you guys recommend?

RE: 5% upload speed on Sophos UTM

goose7791 — Fri, 09 Sep 2022 23:21:53 GMT

The ESXi hosts are Dell R720 servers with Broadcom and Intel physical NICS. The procs are Xeon 2.6GHz 8 cores 2 sockets. The VM NICS are VMXNET3.

I checked QoS as well on the initial existing UTMs and it was not configured. Same on the new ones I built.

Ooooo IPV6. I hadn't thought of that.

I agree about the virtualization overhead not being the best for performance. I am getting anywhere from 500-900Mbps on download. If I can get the same on upload, I would be ok.

RE: 5% upload speed on Sophos UTM

goose7791 — Fri, 09 Sep 2022 23:18:58 GMT

I've been off and on here before. Mainly off and then when something happens I can't remember what Email address I used on the forums so I make a new one.

I haven't messed with the settings of the ATT router much since I can get full gig on another device plugged into it. I may see about flopping cables and/or ports. I thought I had done that but perhaps not.

RE: 5% upload speed on Sophos UTM

dirkkotte — Fri, 09 Sep 2022 21:45:40 GMT

Possible you can change the interface-settings at VMWARE-level

RE: 5% upload speed on Sophos UTM

alan weir — Fri, 09 Sep 2022 20:34:46 GMT

Tell more about the specs of your system.

What CPU does it have? Amount of RAM?

Are you using Intel NICs or Realtek?

Would QoS cause these problems?

Can someone here say if using IPv4 or IPv6 would make a difference?

It's hard to tell if it's a hardware issue since you are running the UTM in a VM (instead of bare metal) which will create overhead and reduce performance.

RE: 5% upload speed on Sophos UTM

BAlfson — Fri, 09 Sep 2022 19:24:21 GMT

HI, welcome to participation in the UTM Community and thanks for the tip-o-the-hat!

I'm not a fan of the wireless routers that AT&T uses in homes. My experience when I tried AT&T for a year in my lab at home was that their router kept losing its configuration. I'd be interested in knowing if you're seeing that in today's equipment.

I do have two clients that I know of that use AT&T fiber. In both cases, we had to disable 'Auto negotiation' and set fixed speeds and duplex for the UTM interface hardware and the interface on the Cisco router provided by AT&T.

Any luck with that?

Cheers - Bob
PS I'm about 200 miles straight north of you.

RE: 5% upload speed on Sophos UTM

goose7791 — Fri, 09 Sep 2022 18:52:45 GMT

Went looking under interfaces... hardware. The only options available to the VMXNET3 adapters is HA monitoring and virtual MAC. I guess the virtual UTM is smart enough to know physical layer options aren't available to it. I thin my next guess is to try with E1000 NICs and see if that matters.