Full transparent proxy fails when on--all other traffic traverses. Works fine when off.

You have detailed information about how your network is configured and nothing really about how the web proxy is configured except "I turned it on".

Can you share some details on how you've configured it?

Michael,

Thank you for the reply!

You're right--I've detailed the configuration but only mentioned that I've turned the proxy on. Aside from setting the proxy to "full transparent" and configuring allowed ranges to encapsulate my subnets (192.168.0.0/16) I'm running the complete default policy and settings. HTTPS inspection is only for headers, so there's no certificate infrastructure. I didn't see anything else germane to functionality and thought that I should at least see blocks/failures hitting the log otherwise.

I've tested the allowed range with a 0.0.0.0/0, only a single network 192.168.10.0/24 as well only a single test host, 192.168.10.128/32, with the same result for each--http/https traffic fails to traverse, nothing hits the log, all other protocols are fine.

I haven't yet had a chance to put up any packet sniffing, but hopefully tonight I can carve time for it.

I should give regular transparent mode a try, though it's a different architecture and I really do need full transparent. My understanding of transparent/full transparent in Sophos UTM is that transparent proxies/NATs the traffic whereas full transparent does proper in-line transparent inspection. Since my border relies on my internal addressing and I'd rather not use the Sophos firewall, transparent would end up being a headache.

But, it's definitely a valid step!

The proxy behaves almost exactly the same between Transparent Mode and Full Transparent Mode.

The main difference is that the outgoing packets maintain the original IP of the client.  It still is a proxy, not in-line transparent inspection (eg like snort).

If you have something upstream (closer to the internet) that is looking at the srcip and doing things like traffic shaping it makes a difference.

Michael,

Still no good on Transparent.

Putting it back to full (which I'll need in the design anyway, as I'm NATing/routing from an IPSEC VPN into some internal subnets), and doing packet captures on the core, border and off of the bridge interface of the UTM, I found some weird MAC behavior and a whole lot of TCP resets. Traffic across the bridge looks like it loops around between the core MAC, the internal UTM MAC and the border MAC. I found some additional troubleshooting regarding vswitch bridges in the form of adding some additional flags to the related VM interfaces to ignore duplicate MACs and such, as detailed here: https://community.sophos.com/products/unified-threat-management/f/54/t/40286 but so far still no luck, though I haven't taken another pcap to see if the behavior has changed, as it's late.

--Robert

I'm a http proxy guy not a network guy so I may not be of much help then.

From my perspective I would packet capture everything on the UTM on port 80 on all interfaces.  Do you see the incoming HTTP request from the client browser? Do you see the outgoing HTTP to the far server?  What is in the proxy log?

Have you tried Explicit mode.  Put the UTM's FQDN in the Proxy field in the browser network config.  That should prove whether or not the proxy works (also that the proxy can reach the outside world) ignoring your network configuration.

Hi, Robert, and welcome to the UTM Community!

Have you given the bridge a default gateway?  I think that's something the HTTP Proxy needs.

Cheers - Bob

Bob,

Thank you for the post! The bridge does have a gateway. The bridge is in-place and in-use regardless of whether the transparent proxy is on or not, and it works without issue whatsoever.

Mark--I haven't had time to sit down and do more pcaps as well as try the explicit proxy, which will take a smidgen of rearchitecting. The initial pcap I took from both core/border and the UTM pointed at some odd MAC behaviour which I had hoped to have solved with some additional vswitch tweaking. I'll have time this weekend to do more captures.

Let me pull back a moment though, and this is for anyone: are there further considerations when implementing this in ESXi? I've now rebuilt the VM several times and I get the same behaviour each time. The steps to creating a bridge and throwing on Full Transparent mode web filtering are not complicated or vast. So it seems like there's something at some other layer of abstraction.

I was able to carve a bit more time on this. Here's what I've discovered for packet capturing, and I'm still researching how to fix it, especially since I have everything in place that I know of to deal with ESXi and Mac addresses.

Here are two conversations both to and from the same hosts, both captured on the Sophos bridge. The first pair traversing the bridge with transparent web filtering off:

Internet Protocol Version 4, Src: 192.168.10.128 (192.168.10.128), Dst: 64.90.40.65 (64.90.40.65) Ethernet II, Src: Vmware_b9:fb:25 (00:0c:29:b9:fb:25), Dst: Vmware_d9:32:17 (00:0c:29:d9:32:17)

Internet Protocol Version 4, Src: 64.90.40.65 (64.90.40.65), Dst: 192.168.10.128 (192.168.10.128) Ethernet II, Src: Vmware_d9:32:17 (00:0c:29:d9:32:17), Dst: Vmware_b9:fb:25 (00:0c:29:b9:fb:25)

(MAC ending in fb:25 is my core router's external interface. 32:17 is my Border's ingress interface. )

And this one with the web filter on. It pretty much does nothing but TCP Retrans the entire time:

Internet Protocol Version 4, Src: 192.168.10.128 (192.168.10.128), Dst: 64.90.40.65 (64.90.40.65) Ethernet II, Src: Vmware_b9:fb:25 (00:0c:29:b9:fb:25), Dst: Vmware_d9:32:17 (00:0c:29:d9:32:17)

Internet Protocol Version 4, Src: 64.90.40.65 (64.90.40.65), Dst: 192.168.10.128 (192.168.10.128) Ethernet II, Src: Vmware_88:a9:59 (00:0c:29:88:a9:59), Dst: Vmware_d9:32:17 (00:0c:29:d9:32:17)

(MAC ending in fb:25 is my core router's external interface. 32:17 is my Border's ingress interface, and a9:59 is the Core vSwitch MAC of the Sophos bridge.)

So to translate: in conversations with the web filter on, my local host packet comes in from the core, out to the border, across the bridge as normal. On the way back in, it comes in from the Sophos Bridge MAC (the MAC is the core-vSwitch side if it matters) but then it points at my local host IP but the dest MAC is the border again? What is munging my packets to tell it my internal network is on the border, only while the web filter is on?

Are you sure that all of the traffic between your PC at 192.168.10.128 and the rest of the world is passing through the bridge?  I would think you would do packet captures on the UTM's bridge and the PC, but I can't tell where you sniffed the packets.

I'm a visual-tactile learner, so I'm having a hard time following your description without a network diagram showing IPs and MACs.

Cheers - Bob

Bob,

Thank you for the reply. I don't have access to my network plans right now, but I'll MSPaint up a quick diagram.

To answer your question: the packet captures were issued from the root user of the Sophos UTM using tcpdump on the bridge interface as in this article: https://sophserv.sophos.com/repo_kb/115343/file/308674.pdf

The Sophos bridge is composed of the two Core/Border vSwitch interfaces. As I've said, (repeatedly at this point, I do realise this is a weird problem but seriously!) the bridge interface is the only configured topology that can get packets from my core to the border. I can easily work around it in the VM configuration, but that isn't the case. The bridge actively is forwarding packets, all the time. It just stops for 80/443 traffic when I turn the web filter on. When that happens, the bridge *still* works for any other protocols. I just can't wrap my head around what in virtual-land could be screwing with it when the web filtering proxy is on.

Thanks for the diagram, Robert,  That makes it much easier to visualize what might be happening.  In the Core and Border VMs, what MAC do they have in their ARP tables for 10.10.10.3?  What MACs are in  the ARP table in the UTM for 10.10.10.1 and 10.10.10.2?  What do all three have for 192.168.10.128?  If that all seems correct, what do you see when you do simultaneous packet captures on their10.10.10.x interfaces?  Raw data, please.

Cheers - Bob

Thanks for the diagram, Robert,  That makes it much easier to visualize what might be happening.  In the Core and Border VMs, what MAC do they have in their ARP tables for 10.10.10.3?  What MACs are in  the ARP table in the UTM for 10.10.10.1 and 10.10.10.2?  What do all three have for 192.168.10.128?  If that all seems correct, what do you see when you do simultaneous packet captures on their10.10.10.x interfaces?  Raw data, please.

Cheers - Bob