Internet "crossed Failover" between two Branchs with Sophos UTM SG (over PtP Link Wireless)

I manage two Sophos UTM SG in two different Branchs of the Company.

CURRENT SITUATION

"BRANCH A" is connected to internet by "ISP A" on interface ETH1 of its UTM. ISP is the Internet Service Provider

"BRANCH B" is connected to internet by a different ISP named "ISP B" on interface ETH1 of a second/its UTM.

UTM of "BRANCH" A is also connected to UTM of "BRANCH B" by a IpSEC VPN (this is only an additional info but is not the focus of this case)

SITUATION TO BE EXPLORED IF FEASIBLE (see below picture)

I want to setup a wi-fi/wireless PtP link (Hyperlan 5ghz link using Ubiquity Hardware) and connect "BRANCH A" UTM (by its ETH2 interface) to the "BRANCH B" UTM (by its ETH2 interface). Distance between branchs is 3 km (see below picture/diagram).

Wi-fi PtP Link acts like a "Ethernet cable patch" between the two firewalls.

The Focus in to obtain an "crossed Internet Failover Service" between the two Branchs, I mean if one of the two ISP connections go down the branch in failure will use the ISP connection of the other Branch (and vice-versa).

Any suggestion to setup this interesting Scenario??

Many thank in advance for the support

FAB

  • you can use the link between "BRANCH A" and "BRANCH b" with default gateway configured. So you have 2 links with default dateway.

    ISP-Balancing is active now.

    Configure the ISP-balancing as active/failover or active/active with weighting 100:0.

    So your other branch is the second ISP for you.

     

    Dont forget to deny access from other branch to local ressources within firewall-rule-set.

    Use other-branch-network -> some-services -> Internet (NOT ANY) within the rules.

  • In reply to dirkkotte:

    Thanks for the answer but for me is not clear at all.

    There are two gateways (Gateway of ISP WAN of "BRANCH A" and Gateway of ISP of "BRANCH B".

    Anybody else want to suggest or comment this issue?

    Thanks in advance

    FAB

  • In reply to FabItaly:

    Hello Fab,

    you can treat both connections from/through eth1 and/or eth2 the same. The physical link doesn't matter, if it's a direct patchcable or a wifi connection over some distance.

    In case of the cable running from eth1 to your ISP-router that internal IP of the router is your gateway to "the internet", in case of the wifi connection going from eth2 to the other SG/UTM in branch B, that UTM's IP address is your second gateway to "the internet". If you enable a second gateway, the UTM tells you, that it enables "Uplink Balancing", where you can set a weight for the several interfaces depending on their line speed. Additionally you have to setup multipath rules to control the traffic over these lines, you find this under "Interfaces" in the UTM.

    The other site is exactly the same, but vice versa. So in the end you have two sites having two gateways each, which would be 4 gateways im sum.

    I have exactly the same setup running at two customer's sites with a Mikrotik 60Ghz Wifi pair of dishes between to buildings at a distance of 2 km approx. Both sites have a different ISP and can do a failover to each other.

  • In reply to jprusch:

    Hi , about the gatways now is much clear. About to create the proper multipath rules to control the traffic on the Interfaces I have some difficulties how to setup these rules.

    If I understood correctly, in normal condition ETH1 of "BRANCH A" UTM must to permit (or put at disposal) to ETH2 interface to share all the internet services available (Web surfing, SMTP, etc, etc,,,,) on ETH1. The same in the opposit side (UTM on "BRANCH B").

    If you can suggest me a link or a document where this kind of rules are documented, I will really appreciate.

    Many thanks in advance.

    FAB

  • In reply to FabItaly:

    it is not necessary to create multipath rules.

    Within "uplink-balancing" you can edit the "interface scheduler".

    If you set the Interface from other branch to 0, it is only used if local ISP-interface is unavailable.

  • In reply to jprusch:

    Dear Philipp,

    I did the setting up of the PtP link and now both of UTMs are linked one to each other. If I ping (from the Service ping tool of UTM) the Branch1 Gateway from Branch2 Ptp Link interface, I can reach the Branch1 gateway and vice-versa.

    Anyway I tried several kind of multipath rules but the Ptp Interface Status is always in"Error". I suppose it must be up to permit the main WAN Failover.

    My last test was to grant the uplik PtP interface "UP" on Branch2,firstly, in order to copy the same rules to the Branch1 UTM, but this first step has been always negative.

    Can you copy to me your Multipath Rules in order to verify/test them on my UTM??

    Many thanks in advance.

    Regards

    Fab

  • In reply to FabItaly:

    Hello Fab,

    Multipath rulkes are only needed to priorize traffic, they won't help you at all, if the interface/link itself is in "error"-state.

    Please show us your interface settings for eth2 on both sides, as well as the network details of your "wifi"-link.

  • In reply to jprusch:

    Hello Philipp,

    first of all I appreciate your interest to help me in this setting.

    You can see on the below pictures, that is the situation/configuration of the Ptp Interfaces.

    I want to give you the following additional informations on the current situation:

    -#1 Ptp link is established with success using a pair of Ubiquiti Litebeam CPEs with a performance of 4 Mbps speed and 2ms of time ping. Configuration of the CPE is: First CPE (Branch1) IP 192.168.1.112 Subnet 255.255.255.0 Gateway 192.168.1.1 and second CPE (Branch2) IP 192.168.1.113 Subnet 255.255.255.0 Gateway 192.168.1.1. This configaration has been used several time in other wireless link when I need to appear the link as a transparent "patch cable". The CPEs doesn't need of the internet service for them and I also do not need to reach the CPEs from other networks (when I need to configurate the CPE I will disconnect them from the firewall and I will connect on them directly by a notebook just only to configure them).

    -#2 With the Ptp link active is possible to ping (from Branch2 UTM by the UTM ping tool over the PtP its interface) the IP address of the Branch1 main WAN (WAN Lenfiber) but IS NOT possible to ping the Gateway address of Branch1 main WAN (WAN Lenfiber)

    -#3 With the Ptp link active is possible to ping (from Branch1 UTM by the UTM ping tool over the PtP its interface) the IP address of the Branch2 main WAN (WAN Trivenet) but IS NOT possible to ping the Gateway address of Branch1 main WAN (WAN Trivenet)

    -#4 With the Ptp link active is NOT possible to ping 8.8.8.8 address (from Branch2 UTM by the UTM ping tool over the PtP interface) and the same thing in UTM Branch 1

    -#5 Branch1 has already one spare second WAN connection (WAN Alice) working actually as failover succesfully but it will be dismantled/disactivated soon

    -#6 PtP Interfaces are part of active interfaces on Uplink interface group of failover, and the weight setting is 100: because I want only failover and NOT Balancing

    -#7 there are not additional rules concerning the PTP addresses/interfaces. I tried a huge quantity of Multipath rules, Masqarade rules, NAT rules without grant to bring up the PtP connection (in order almost to ping the 8.8.8.8 address from PtP interfaces)

    If you need other information let me know.

    many thanks in advance

    FAB

  • In reply to FabItaly:

    Hello Fab,

    which interface of the UTM has the 192.168.1.1 as its IP Address?

  • In reply to jprusch:

    Hello Philipp,

    there are no interfaces in UTM using 192.168.1.x class of IPs, just because this network is only to create a link between PtP CPEs, in order that link appears as a "patch cable".

    I though that CPE are not involved in routing and their IP addresses must to be hidden.

    Let me know if you have other suggestion.

    Many thanks in advance, Regards,

    FAB

  • In reply to FabItaly:

    Hi Phillip,

    just only for congruence I changed the IP addresses of the PtP CPEs putting them in the 10.2.2.x class of IP (Class of UTM PtP interfaces), but nothing heppened. The UTMPtP  interfaces are always in error, as before.

    Regards

    FAB

  • In reply to FabItaly:

    Hello Fab,

    what you describe as "patch cable" is called a Bridge. I have a WORKING setup which I adopted to your IP-addresses in the diagram below:

     

    Please try this setup and see how the status of your interface is.

  • In reply to jprusch:

    Hello Fab,

    please remove the checks at "IPv4 default GW" at ewth2 on both sides and see, if you can reach the other UTM over your bridge.

  • In reply to jprusch:

    Hi,
    The interface error-state is from link-monitoring...mostly.
    You have to allow some special packets on the way to the internet for every external interface. (the bridge-link too)
    We had problems with ASA within the path, while asa was not set to "traceroute visible".
    SG monitors some special hop discovered by traceroute (sadly don't know if TCP/UDP/ICMP is used...)
    Sometimes it is fixed by disabling "automatic monitoring" and put some known IP's to the monitoring-list. (1.1.1.1 8.8.8.8 ...)

  • In reply to jprusch:

    Good morning Philipp,

    yesterday night (as I wrote in my previous post) I put the wireless CPEs in the same subnet so now the situation is like this (I can not use 192.168.1.x subnet because I discovered it is already used on WLAN Interface):

    Branch1 PtP Interface: IP 10.2.2.1/24 GW 10.2.2.2

    Wireless CPE Branch1 side: IP IP 10.2.2.112/24

    Wireless CPE Branch2 side: IP IP 10.2.2.113/24

    Branch2 PtP Interface: IP 10.2.2.2/24 GW 10.2.2.1

    This is equivalent of your diagram.

    Result: the PtP interfaces both sides are still in ERROR

    This morning (as you suggested) I removed the gateways unchecking the "IPv4 default Gateway" on both side then:

    Branch1 PtP Interface: IP 10.2.2.1/24 (No Gateway)

    Wireless CPE Branch1 side: IP IP 10.2.2.112/24

    Wireless CPE Branch2 side: IP IP 10.2.2.113/24

    Branch2 PtP Interface: IP 10.2.2.2/24 (No Gateway)

    Result: the "PtP interfaces Branch1" and "PtP interfaces Branch2" side became in Up state.

    NOTE1: Removing the gateway in the PTP Interface caused the Intefaces were removed from uplink Balancing "Active interface list" (just because probably in this list can be only interfaces with gateway)

    NOTE2: In any case by Ping tool of the UTM, I'm not able to ping address 8.8.8.8 from both the "Ptp Interfaces"

    I think that is impossible to have Failover without having "Ptp Interfaces" in Active interface list on Uplink Balancing.

    Actually for my point of view, until I get to reach 8.8.8.8 address by ping UTM tool on Ptp Interfaces, and have those interfaces in Uplink Balance Active Interface list, the goal has not reached.

    Morover I can not disconnect Internet Main connection during working time to test if the Failover works, so only checking the ping of 8.8.8.8 is the unique tool that I can use to make my trials.

    Many thanks in advance in case you will suggest me other trials to solve this issue.

    Regards

    FAB