Internet "crossed Failover" between two Branchs with Sophos UTM SG (over PtP Link Wireless)

you can use the link between "BRANCH A" and "BRANCH b" with default gateway configured. So you have 2 links with default dateway.

ISP-Balancing is active now.

Configure the ISP-balancing as active/failover or active/active with weighting 100:0.

So your other branch is the second ISP for you.

Dont forget to deny access from other branch to local ressources within firewall-rule-set.

Use other-branch-network -> some-services -> Internet (NOT ANY) within the rules.

Thanks for the answer but for me is not clear at all.

There are two gateways (Gateway of ISP WAN of "BRANCH A" and Gateway of ISP of "BRANCH B".

Anybody else want to suggest or comment this issue?

Thanks in advance

FAB

Hello Fab,

you can treat both connections from/through eth1 and/or eth2 the same. The physical link doesn't matter, if it's a direct patchcable or a wifi connection over some distance.

In case of the cable running from eth1 to your ISP-router that internal IP of the router is your gateway to "the internet", in case of the wifi connection going from eth2 to the other SG/UTM in branch B, that UTM's IP address is your second gateway to "the internet". If you enable a second gateway, the UTM tells you, that it enables "Uplink Balancing", where you can set a weight for the several interfaces depending on their line speed. Additionally you have to setup multipath rules to control the traffic over these lines, you find this under "Interfaces" in the UTM.

The other site is exactly the same, but vice versa. So in the end you have two sites having two gateways each, which would be 4 gateways im sum.

I have exactly the same setup running at two customer's sites with a Mikrotik 60Ghz Wifi pair of dishes between to buildings at a distance of 2 km approx. Both sites have a different ISP and can do a failover to each other.

Hi jprusch, about the gatways now is much clear. About to create the proper multipath rules to control the traffic on the Interfaces I have some difficulties how to setup these rules.

If I understood correctly, in normal condition ETH1 of "BRANCH A" UTM must to permit (or put at disposal) to ETH2 interface to share all the internet services available (Web surfing, SMTP, etc, etc,,,,) on ETH1. The same in the opposit side (UTM on "BRANCH B").

If you can suggest me a link or a document where this kind of rules are documented, I will really appreciate.

Many thanks in advance.

FAB

it is not necessary to create multipath rules.

Within "uplink-balancing" you can edit the "interface scheduler".

If you set the Interface from other branch to 0, it is only used if local ISP-interface is unavailable.

Dear Philipp,

I did the setting up of the PtP link and now both of UTMs are linked one to each other. If I ping (from the Service ping tool of UTM) the Branch1 Gateway from Branch2 Ptp Link interface, I can reach the Branch1 gateway and vice-versa.

Anyway I tried several kind of multipath rules but the Ptp Interface Status is always in"Error". I suppose it must be up to permit the main WAN Failover.

My last test was to grant the uplik PtP interface "UP" on Branch2,firstly, in order to copy the same rules to the Branch1 UTM, but this first step has been always negative.

Can you copy to me your Multipath Rules in order to verify/test them on my UTM??

Many thanks in advance.

Regards

Fab

Hello Fab,

Multipath rulkes are only needed to priorize traffic, they won't help you at all, if the interface/link itself is in "error"-state.

Please show us your interface settings for eth2 on both sides, as well as the network details of your "wifi"-link.

Hello Philipp,

first of all I appreciate your interest to help me in this setting.

You can see on the below pictures, that is the situation/configuration of the Ptp Interfaces.

I want to give you the following additional informations on the current situation:

-#1 Ptp link is established with success using a pair of Ubiquiti Litebeam CPEs with a performance of 4 Mbps speed and 2ms of time ping. Configuration of the CPE is: First CPE (Branch1) IP 192.168.1.112 Subnet 255.255.255.0 Gateway 192.168.1.1 and second CPE (Branch2) IP 192.168.1.113 Subnet 255.255.255.0 Gateway 192.168.1.1. This configaration has been used several time in other wireless link when I need to appear the link as a transparent "patch cable". The CPEs doesn't need of the internet service for them and I also do not need to reach the CPEs from other networks (when I need to configurate the CPE I will disconnect them from the firewall and I will connect on them directly by a notebook just only to configure them).

-#2 With the Ptp link active is possible to ping (from Branch2 UTM by the UTM ping tool over the PtP its interface) the IP address of the Branch1 main WAN (WAN Lenfiber) but IS NOT possible to ping the Gateway address of Branch1 main WAN (WAN Lenfiber)

-#3 With the Ptp link active is possible to ping (from Branch1 UTM by the UTM ping tool over the PtP its interface) the IP address of the Branch2 main WAN (WAN Trivenet) but IS NOT possible to ping the Gateway address of Branch1 main WAN (WAN Trivenet)

-#4 With the Ptp link active is NOT possible to ping 8.8.8.8 address (from Branch2 UTM by the UTM ping tool over the PtP interface) and the same thing in UTM Branch 1

-#5 Branch1 has already one spare second WAN connection (WAN Alice) working actually as failover succesfully but it will be dismantled/disactivated soon

-#6 PtP Interfaces are part of active interfaces on Uplink interface group of failover, and the weight setting is 100: because I want only failover and NOT Balancing

-#7 there are not additional rules concerning the PTP addresses/interfaces. I tried a huge quantity of Multipath rules, Masqarade rules, NAT rules without grant to bring up the PtP connection (in order almost to ping the 8.8.8.8 address from PtP interfaces)

If you need other information let me know.

many thanks in advance

FAB

Hello Philipp,

first of all I appreciate your interest to help me in this setting.

You can see on the below pictures, that is the situation/configuration of the Ptp Interfaces.

I want to give you the following additional informations on the current situation:

-#1 Ptp link is established with success using a pair of Ubiquiti Litebeam CPEs with a performance of 4 Mbps speed and 2ms of time ping. Configuration of the CPE is: First CPE (Branch1) IP 192.168.1.112 Subnet 255.255.255.0 Gateway 192.168.1.1 and second CPE (Branch2) IP 192.168.1.113 Subnet 255.255.255.0 Gateway 192.168.1.1. This configaration has been used several time in other wireless link when I need to appear the link as a transparent "patch cable". The CPEs doesn't need of the internet service for them and I also do not need to reach the CPEs from other networks (when I need to configurate the CPE I will disconnect them from the firewall and I will connect on them directly by a notebook just only to configure them).

-#2 With the Ptp link active is possible to ping (from Branch2 UTM by the UTM ping tool over the PtP its interface) the IP address of the Branch1 main WAN (WAN Lenfiber) but IS NOT possible to ping the Gateway address of Branch1 main WAN (WAN Lenfiber)

-#3 With the Ptp link active is possible to ping (from Branch1 UTM by the UTM ping tool over the PtP its interface) the IP address of the Branch2 main WAN (WAN Trivenet) but IS NOT possible to ping the Gateway address of Branch1 main WAN (WAN Trivenet)

-#4 With the Ptp link active is NOT possible to ping 8.8.8.8 address (from Branch2 UTM by the UTM ping tool over the PtP interface) and the same thing in UTM Branch 1

-#5 Branch1 has already one spare second WAN connection (WAN Alice) working actually as failover succesfully but it will be dismantled/disactivated soon

-#6 PtP Interfaces are part of active interfaces on Uplink interface group of failover, and the weight setting is 100: because I want only failover and NOT Balancing

-#7 there are not additional rules concerning the PTP addresses/interfaces. I tried a huge quantity of Multipath rules, Masqarade rules, NAT rules without grant to bring up the PtP connection (in order almost to ping the 8.8.8.8 address from PtP interfaces)

If you need other information let me know.

many thanks in advance

FAB

Hello Fab,

which interface of the UTM has the 192.168.1.1 as its IP Address?

Hello Philipp,

there are no interfaces in UTM using 192.168.1.x class of IPs, just because this network is only to create a link between PtP CPEs, in order that link appears as a "patch cable".

I though that CPE are not involved in routing and their IP addresses must to be hidden.

Let me know if you have other suggestion.

Many thanks in advance, Regards,

FAB

Hi Phillip,

just only for congruence I changed the IP addresses of the PtP CPEs putting them in the 10.2.2.x class of IP (Class of UTM PtP interfaces), but nothing heppened. The UTMPtP  interfaces are always in error, as before.

Regards

FAB

Hello Fab,

what you describe as "patch cable" is called a Bridge. I have a WORKING setup which I adopted to your IP-addresses in the diagram below:

Please try this setup and see how the status of your interface is.

Hello Fab,

please remove the checks at "IPv4 default GW" at ewth2 on both sides and see, if you can reach the other UTM over your bridge.

Hi,
The interface error-state is from link-monitoring...mostly.
You have to allow some special packets on the way to the internet for every external interface. (the bridge-link too)
We had problems with ASA within the path, while asa was not set to "traceroute visible".
SG monitors some special hop discovered by traceroute (sadly don't know if TCP/UDP/ICMP is used...)
Sometimes it is fixed by disabling "automatic monitoring" and put some known IP's to the monitoring-list. (1.1.1.1 8.8.8.8 ...)

Good morning Philipp,

yesterday night (as I wrote in my previous post) I put the wireless CPEs in the same subnet so now the situation is like this (I can not use 192.168.1.x subnet because I discovered it is already used on WLAN Interface):

Branch1 PtP Interface: IP 10.2.2.1/24 GW 10.2.2.2

Wireless CPE Branch1 side: IP IP 10.2.2.112/24

Wireless CPE Branch2 side: IP IP 10.2.2.113/24

Branch2 PtP Interface: IP 10.2.2.2/24 GW 10.2.2.1

This is equivalent of your diagram.

Result: the PtP interfaces both sides are still in ERROR

This morning (as you suggested) I removed the gateways unchecking the "IPv4 default Gateway" on both side then:

Branch1 PtP Interface: IP 10.2.2.1/24 (No Gateway)

Wireless CPE Branch1 side: IP IP 10.2.2.112/24

Wireless CPE Branch2 side: IP IP 10.2.2.113/24

Branch2 PtP Interface: IP 10.2.2.2/24 (No Gateway)

Result: the "PtP interfaces Branch1" and "PtP interfaces Branch2" side became in Up state.

NOTE1: Removing the gateway in the PTP Interface caused the Intefaces were removed from uplink Balancing "Active interface list" (just because probably in this list can be only interfaces with gateway)

NOTE2: In any case by Ping tool of the UTM, I'm not able to ping address 8.8.8.8 from both the "Ptp Interfaces"

I think that is impossible to have Failover without having "Ptp Interfaces" in Active interface list on Uplink Balancing.

Actually for my point of view, until I get to reach 8.8.8.8 address by ping UTM tool on Ptp Interfaces, and have those interfaces in Uplink Balance Active Interface list, the goal has not reached.

Morover I can not disconnect Internet Main connection during working time to test if the Failover works, so only checking the ping of 8.8.8.8 is the unique tool that I can use to make my trials.

Many thanks in advance in case you will suggest me other trials to solve this issue.

Regards

FAB

Hello Fab,

please try to solve your problem in small steps:

1.) Can you ping from UTM in branch1 to the ip 10.2.2.2 ?  Use "Support/Tools/Ping Check" on the UTM in branch1.

2.) Can you ping from UTM in branch2 to the ip 10.2.2.1 ? This time you got to do this from the UTM in branch2.

3.) If both tests above were successful, your problem is not with the Wifi-link.

Hello Philipp,

of course my problem IS NOT the wifi link (I'm an Ham Radio Operator pioner with wifi links since 1996).

Just to confirm that, today I configured PtP Interface in Branch1 as an additional LAN (LAN2) with DHCP server active and I went on Branch2 with a Laptop connected directly on the ethernet cable coming from the CPE on Branch2 (before connected to the Eth2 of UTM) and I can grant fully connection (DHCP address assigned on Laptop from UTM of Branch1) and navigated from Branch2 using Branch1 WAN connection. So Wireless link is 100% fully working. I performed also a internet speed test obtaining +/- 30 Mbps of troughput (WAN of Branch1 is 1Gbps FTTH connection and the bottleneck is the wireless link).

I Perform yesterdey and this morning all kind of ping from Branch1 to Branch2 and reverse, always using the "Support/Tools/Ping Check" tool on PtP interfaces.

On order of depht, these are the results of ping:

Pinging from Branch 1 (10.2.2.1) to Branch1 PtP CPE 10.2.2.2 = RESULT OK

Pinging from Branch 1 (10.2.2.1) to Branch1 PtP CPE 10.2.2.112 = RESULT OK

Pinging from Branch 1 (10.2.2.1) to Branch2 PtP CPE 10.2.2.113 = RESULT OK

Pinging from Branch 1 (10.2.2.1) to Branch2 UTM PtP interface 10.2.2.2 = RESULT OK

Pinging from Branch 1 (10.2.2.1) to Branch2 UTM WAN interface IP address xxx.x.154.185 = RESULT OK

Pinging from Branch 1 (10.2.2.1) to Branch2 UTM WAN interface IP address xxx.x.154.190 (the network is /29) = RESULT NEGATIVE

Pinging from Branch 1 (10.2.2.1) to Branch2 UTM WAN interface host google.com = RESULT NEGATIVE

Same results from Branch2

So the problem (maybe since the beginning) are on UTM

In order to keep the PtP interfaces Up on State and Up on Link, I must configurate the PtP interfaces without Gateway (and than not part of Uplink Interfaces).

For my point of View this issue can be solved by some particular kind of Rules to permit that each Ptp Interface acts as LAN and WAN Interface in the same time.

Thank for the support. I will stay tuned in this Forum, for any kindly suggestion.

Regards

FAB

Hello Fab,

I stumble each time over your descirption "PtP CPE and some IP address"

there is no such additional IP 10.2.2.2 on the branch1 side, what do you MEAN here?

I begin to believe there is a complete misconception here...

Let's collect our data for your physical layout again:

branch1 utm port eth2(=10.2.2.1)---(LAN-cable) ---ethernet port of wifi antenna1(10.2.2.112) ------(wifi bridge over air) ----ethernet port of antenna2(=10.2.2.113) --- (LAN-cable) ----branch2 utm port eth2 (=10.2.2.2)

So my questions/suggestions were to ping from utm in branch1 to utm in branch2, that is from 10.2.2.1 to 10.2.2.2 and vice versa. But you did not do that.

What you wrote up in your last post doesn't make any sense to me, sorry!