Hallo,

I modified #2.1 in Rulz (last updated 2021-02-16) based on your question.  Is that what you were looking for?

Cheers - Bob

Thank you for your answer, this is helpful.

I was basically looking for a document that shows the order in which the routes are interpreted by the engine.

(for new IPv4 Packets coming to the firewall, typically SRC x.x.x.  (client IP) DST x.x.x.x (any IP))

The firewall has to make a routing decision at some point, right ? 

As I understand from your document, it goes like this: (for new sessions)

1. Web filter profile "Optional interface for outgoing traffic"
2. Multipath Rules
3. IPSEC?
4. Policy Route?
5. Static Route?
6. SNAT (Firewall is making a routing decision = Outgoing interface based on what is NAT'ed?)
7. Masquerading ( same?)

What about SSL VPN?

If this order is followed, it means that Multipath Rules overrides IPSEC routes?

Ex : I have a client with internal ip 10.0.0.1. 

I have an IPsec remote subnet 11.0.0.0/24 (Which should be a directly connected route on the Sophos? - It doesn't seem so in the Sophos world??)

I have a multipath rule for SRC 10.0.0.0/24 destination ANY > WAN interface

Now, packet has following IPs : SRC 10.0.0.1 DST 11.0.0.1 

Will this packet be routed over the IPsec tunnel or other the wan interface (multipath rule)?

Of course, this was just an example, but this can apply to any of the previous mentioned routes 

Thank you for your answer, this is helpful.

I was basically looking for a document that shows the order in which the routes are interpreted by the engine.

(for new IPv4 Packets coming to the firewall, typically SRC x.x.x.  (client IP) DST x.x.x.x (any IP))

The firewall has to make a routing decision at some point, right ? 

As I understand from your document, it goes like this: (for new sessions)

1. Web filter profile "Optional interface for outgoing traffic"
2. Multipath Rules
3. IPSEC?
4. Policy Route?
5. Static Route?
6. SNAT (Firewall is making a routing decision = Outgoing interface based on what is NAT'ed?)
7. Masquerading ( same?)

What about SSL VPN?

If this order is followed, it means that Multipath Rules overrides IPSEC routes?

Ex : I have a client with internal ip 10.0.0.1. 

I have an IPsec remote subnet 11.0.0.0/24 (Which should be a directly connected route on the Sophos? - It doesn't seem so in the Sophos world??)

I have a multipath rule for SRC 10.0.0.0/24 destination ANY > WAN interface

Now, packet has following IPs : SRC 10.0.0.1 DST 11.0.0.1 

Will this packet be routed over the IPsec tunnel or other the wan interface (multipath rule)?

Of course, this was just an example, but this can apply to any of the previous mentioned routes 

It's just not possible to put things in a straight line.The packet will go via the IPsec tunnel.

Cheers - Bob

There is NO WAY the Engine of the Sophos UTM firewall doesn't make a routing decision following a logical and precise order.  

We had a hard time configuring BASIC routing (="I want you to forward all packets to this Gateway."), that a simple l3 switch can do.

• When policy routing was applied, packets were dropped.

> It seems the Sophos firewall doesn't like to forward internet traffic (destination any) to a gateway that is not configured as uplink??

We decided then to use multipath rules.

Based on your comment IPsec connected subnets (IPsec Phase 2 selectors) do have priority over multipath rules.   ( That's actually what's happening, I have also tested it :) )

> Of course if the phase 2 selector is 0.0.0.0, I doubt it will route all over the IPsec link.

This is totally fine, and this is called longest prefix match  

https://en.wikipedia.org/wiki/Longest_prefix_match

Still, the  engine looks at the IPSEC ROUTES FIRST.  

From what we tested  static routes will also be analyzed before multipath rules.

Normally, the routing table alone must be analyzed based on destination + longest prefix match logic.

The idea with policy routing is to build an independent second routing table that will be matched from top to bottom:  

 https://en.wikipedia.org/wiki/Policy-based_routing

I will try to summarize the behavior :

"if the policy route match, we apply it. If it doesn't match any policy route, we look at the normal routing table"

Does the multipath rules follow this logic ? I doesn't seem so (=in the Sophos World, even if the Multipath Rule selectors match, the rule will not be applied if we have a more specific static route)

Does policy routing follow this logic? I have no idea

Do you understand my point? 

That's why I asked if we have any document explaining the behavior.  

In the CISCO, JUNIPER, CHECKPOINT, FORTINET ...*add any other manufacturer here*.. It works like described in the blue paragraph.

We cannot just stay in front of a black box thinking "oh, what will you do with my packets based on your mood today ? Let's guess"

Disclaimer : Yes , there is no RFC on how PBR is precisely supposed to work. I don't pretend to teach Sophos how it should work. It just works like this most of the time

I just want to understand how they implemented it, so I can go on configuring my routing

This isn't just a switch.  WebAdmin is a GUI that manipulates databases of objects and settings.  A single change there can cause the Configuration Daemon to rewrite hundreds of lines of the code used to run the UTM.  The underlying code is freeware modified by Astaro/Sophos.  Different teams worked on different aspects of the code.  I know how to solve almost any problem, but I don't know of any place you can find a precise diagram (not a straight line!) that will answer your question.

Cheers - Bob

That's unfortunate.

What you describes with the "databases of objects and settings." and the underlying code is the same with Pan-OS (Palo Alto FW) , FortiOS (For FortiGates), Checkpoint OS etc...

Still, those manufacturers have implemented a functional and logical routing process.

The current behavior on Sophos UTM makes the whole design + troubleshooting very hard.

Even worse, it makes routing decisions unpredictable, and causes you to completely lose control of the network.

Right now I have the following problems :

1. I see that all my static routes have priority over Multipath Rules
2. A directly connected route (A subnet that belongs to an interface, because it is configured on the Sophos UTM itself.) has priority over multipath and policy route.

It's basically out of control.

If we just take problem 2 for instance. 

Source ip 10.10.10.1 > Destination IP 1.1.1.2/26  ( I changed the IPs, the destination is a whole Public IP range that is directly connected)

There is no single route configured on the Sophos that tells the traffic to be routed via eth1 ( eth1 is configured with 1.1.1.1/26 + it has ipv4 default gw option checked)

In fact, I have a multipath rules that says :

Source 10.10.10.0/24 > Destination Any  -> Route this traffic to eth2.

Traffic is routed via the connected route (via eth1).

We are talking about a L3 Device. Moreover, a Firewall. It is kind of strange that it makes this routing decision, right?

I mean I understand connected route has priority, but we cannot influence that.

Same applies for problem 1.

I can still drop the packets, because the Sophos is following his Firewall Rules Table.

But this is not what I want. I just want the Sophos to correctly route Packets.

I just hope there will be a documentation for this. In the meantime, I will just tinker.

The "routing precedence" on Sophos XG is better, props to the devs :)

Please show a picture of your Multipath rule.

Cheers - Bob

I cannot unfortunately do that. 
The name of the Rule itself is too much information to give out on the Forum.

The multipath Rule is just as I described :

Source 10.10.10.0/24 > Destination Any  -> Route this traffic to eth2.

I read on the official documentation for Policy Routing :

Policy Routes

When a router receives a data packet, it normally decides where to forward it based on the destination address in the packet, which is then used to look up an entry in a routing table. However, in some cases, there may be a need to forward the packet based on other criteria. Policy-based routing allows for forwarding or routing of data packets according to your own policies.

---> That is exactly what the Sophos is supposed to do.

So it is supposed to analyze the policy route or Multipath Rules FIRST

not the Connected Routes, for god's sake.

I could not make the Policy Routes neither the Multipath Rules work.

Static Route or connected Routes have priority, and sometimes even the Web filter proxy (LOL)

That doesn't make sense.

I tinkered something with uplink balancing which is working. This does the job so far.

Thank you for your help.

- The configuration is complicated and doesn't make sense

- The idea that you cannot ROUTE anything normally/directly when the direction is 0.0.0.0/0 is kinda crazy.

You have to use Uplink balancing which is more complicated and does some ICMP checks ( I don't want to use these checks and I can not deactivate it)

Anyway I'm done with Sophos UTM haha

Thanks for your help, appreciated :)

I wish you a happier life.

MfG - Bob

Hahaha thanks, you too my friend :)