Thread Info
  • State Verified Answer
  • Locked Locked
  • Replies 5 replies
  • Subscribers 3 subscribers
  • Views 16147 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

SNAT with Multipath Rules / Dual WAN Setup

MartinDE_01

Hi all,

I have a question on the correct configuration of SNAT in a Dual WAN Environment. Or if it's the right approach to do it at all.

As mentioned I have 2 ISPs connected to the UTM. They are both set up as active Uplink Interfaces, the slower Connection is weighted 0, the faster one 100, the purpose is distributing traffic manually over the two lines as needed with Multipath rules but having WAN Failover in place.

  • Multipath rules are used to route services over the specific Interfaces (skipping rule on Interface error active for failover).
  • Internal Network is masqueraded with Primary Interface Address -> Uplink Interfaces.
  • DNAT Rules are in place for each public IP Address to ensure that traffic gets to the correct internal destinations over both ISP connections.

This far everything works fine! 

Now I'm trying to make sure that traffic from several internal servers leave with the correct public IP for each Provider. The behaviour of outgoing traffic at this point is, that the Primary Interface address of the active WAN Uplink is used - which is as expected because of the masquerading rule above.

But I need several Hosts/Services use a specific public IP of the current active WAN Connection. Therefore I tried 2 SNAT Rules for each ISP like the following example for our Remote Desktop Session Host:

SNAT1: Traffic selector: RDSH1(Internal Host Address) -> RDSH-GW-Ports (HTTP/HTTPS) -> Internet IPv4 Source translation: WAN #1 RDSH1 (Additional Address)
SNAT2: Traffic selector: RDSH1(Internal Host Address) -> RDSH-GW-Ports (HTTP/HTTPS) -> Internet IPv4 Source translation: WAN #2 RDSH1 (Additional Address)

But it doesn't seem work. If both WAN Interfaces are up, traffic is always coming from SNAT1 rule WAN #1 RDSH1 Address instead of SNAT2 which would be the actively used ISP as defined by Multipath. Changing order makes no difference. But I can verify that the Multipath rule is working and traffic is leaving over the right WAN Interface (WAN #2). I can also confirm that the right DNAT rule is working for WAN #2 through Firewall log so the correct incoming public IP is used, but the answer is masked with the wrong source IP from the other SNAT1 Rule.

Strange thing is, an equal configuration for our mail Server seems to be working. The only difference in the config is the traffic selector of the SNAT rule, which is any instead of the host:

SNAT3: Traffic selector: ANY -> SMTP -> Internet IPv4 Source translation: WAN #1 MAILSVR1 (Additional Address) 
SNAT4: Traffic selector: ANY -> SMTP -> Internet IPv4 Source translation: WAN #2 MAILSVR1 (Additional Address)

But I can not use ANY as traffic selector for the Remote Desktop Session Host, as I have several other Server with the same Service set (HTTP/HTTPS) which need to take their own routes.

In another post, I read a reply from BAlfson which suggested that it might be smarter to use Masquerade Rules instead of SNAT/DNAT rules to route the traffic? May be this would be the proper way to do it?

I'm really stuck here, may be anyone could point me in the right direction?
Thanks a lot in advance!



This thread was automatically locked due to age.
Parents
  • +1

    Hi Martin,

    Yes, now that Masq rules are in an ordered list, you should use them.  However, if this is traffic that goes through Web Filtering neither NAT approach will work.  Multipath rules know the origin of traffic coming to them from the Proxy, but the Proxy has already changed the source of the packets, so you can't use the LAN addresses in NAT rules.

    In any case, I'm a little confused about the need to change the outgoing IP on responses to traffic that was initiated from the outside.  If a connection from the Internet was initiated on "WAN#1 [RDSH1] (Address)," the connection tracker will send all traffic back out from that address and the Multipath rules will not be consulted.  Is that what's happening?

    Maybe we should be talking about the goal instead of assuming that either of us has already asked the right question.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • 0 in reply to BAlfson

    Hi Bob,

    thanks a lot for your reply, I really appreciate your input! Read many of your responses on several topics over the past weeks and I'm sure we can work this out.

     

    BAlfson said:

    Yes, now that Masq rules are in an ordered list, you should use them.  However, if this is traffic that goes through Web Filtering neither NAT approach will work.  Multipath rules know the origin of traffic coming to them from the Proxy, but the Proxy has already changed the source of the packets, so you can't use the LAN addresses in NAT rules.

    Webfiltering / Proxy is not activated, so this shouldn't be an issue. I've tested behaviour the hard way this friday evening and simply unplugged Primary WAN #1 for further analysis. ;-) The result was that the existing SNAT rules as described in my original post are messing things up completely. I'll further explain below but i think my problems might be located somewhere else and I guess I'll follow your point that it might be more useful to explain you what i want to achive...

    BAlfson said:

    In any case, I'm a little confused about the need to change the outgoing IP on responses to traffic that was initiated from the outside.  If a connection from the Internet was initiated on "WAN#1 [RDSH1] (Address)," the connection tracker will send all traffic back out from that address and the Multipath rules will not be consulted.  Is that what's happening?

    Yes and no, i think the existing Masquerading Rule "Internal Network 10.10.8.1 /21" -> Uplink Interfaces" seems to be interfering here, as the Network Definition "Internal Network" includes my Server Network Segment 10.10.10.1/24. So all outgoing traffic is masked with the Primary active Interface address regardless of its origin, unless i use an additional SNAT rule (which leads to problems in failover scenario as i found out friday). Adding additional masquerading rules for a host/network and placing it top doesn't show any effect. So i might have configuration mistakes already at this point.

    BAlfson said:

    Maybe we should be talking about the goal instead of assuming that either of us has already asked the right question.

    Yes you're totally right with that. It might be a better approach to explain what I'm trying to achieve:

    Let me describe the Environment a bit further: 

    As I said I have two ISPs, WAN#1 is giving a network range with public IPs. From those I use two by Additional Addresses one for Mailserver (mail.domain.com / 212.15.0.2) and one for RDSH (gateway.domain.com 212.15.0.3) default traffic is leaving over Primary Address of WAN#1 (utm1.domain.com / 212.15.0.1). Second ISP WAN#2 is offering 5 static public IPs but cannot distribute them over a Network range, public IP is distributed over MAC DHCP reservation per WAN Interface. To get the two public IPs from this ISP I had to use two Interfaces WAN#2 (9.5.10.100) / WAN#3 (9.5.10.101), as I cannot use Additional Addresses in this case.   

    This is leaving me with 3 active WAN Interfaces, WAN#1 is weighted 100 as it is more stable and should be used for all Server / RED traffic, unless it is down and would be the Default Connection in my understanding through weight 100.

    WAN#2 and WAN#3 is weighted 0, using Multipath rules to route specific traffic over the ISPs like SMTP only on WAN#1, or less important traffic like Clients surfing, Wireless, etc. over WAN#2. If one ISP is down, it should failover to the other ISP but having key infrastructure reachable from the internet. I use external DNS Failover services to Monitor public IPs and switch  external DNS records in case of ISP failure. This makes sure mail Server, RED Network and RDSH FQDN is always pointing to the active ISP public IP. 

    My goal is now, that if both ISPs are online:

    • All Server / RED traffic is going over ISP1 WAN#1 (utm1.domain.com / 212.15.0.1)
    • All Client traffic is going over ISP2 WAN#2 to reduce load on ISP1 WAN#1 leaving it for the important tasks.
    • Mailserver should always go out to the Internet with Additional Address #1 (mail.domain.com / 212.15.0.2) WAN#1 and returning this way.
    • Remote Desktop Gateway should always go out to the Internet with Additional Address #2 (gateway.domain.com / 212.15.0.3) WAN#1 and return this way.
    • My RED Devices (RED 15) should connect to Primary Address WAN#1 (utm1.domain.com / 212.15.0.1)

    In Failover Scenario, assuming ISP1 WAN#1 is down:

    • Mailserver has to go out over ISP2 WAN#2 (mail.domain.com / 9.5.10.100) and return this way (to DNAT it to the Mailserver and SPF/Anti SPAM configuration etc.)
    • Remote Desktop Gateway has to go out over ISP2 WAN#3 (gateway.domain.com / 9.5.10.101) an return this was (to DNAT it to the RDSH Gateway Server).
    • RED Devices (RED 15) should connect to Primary Address WAN#2 (utm2.domain.com / 9.5.10.100) which is no Problem as RED 15 does Failover on its own having a Backup FQDN configured
    In case of ISP2 WAN#2/WAN#3 would be down, Multipath rules are skipped to get traffic back to WAN#1 and nothing special needs to happen because critical infrastructure is using WAN#1 and therefore the " all day standard" configuration already.
     
    While writing this, I think it would make sense to draw a Network diagram as it is a bit confusing just through reading. I have it attached below!

    Thanks a lot for your effort,

    Martin

  • +1 in reply to MartinDE_01

    " WAN#2 is offering 5 static public IPs but cannot distribute them over a Network range, public IP is distributed over MAC DHCP reservation per WAN Interface."

    If your ISP's device supports VLANs, you could use a single physical NIC, but your current solution is good.  All of your configuration looks good with two exceptions...

    "Remote Desktop Gateway should always go out to the Internet with Additional Address #2 (gateway.domain.com / 212.15.0.3) WAN#1 and return this way."

    The only thing that determines with which public IP the traffic leaves is the IP that the initiating traffic arrived onn (connection tracking).  Only your monitoring service's changing the IP of your FQDN makes any difference here.  Any Multipath rule would be redundant.

    In your fail over scenario:

    "Mailserver has to go out over ISP2 WAN#2 (mail.domain.com / 9.5.10.100) and return this way (to DNAT it to the Mailserver and SPF/Anti SPAM configuration etc.)"

    I think I would have two MX records with the one at a lower priority that points at an FQDN for the second ISP.  A Multipath rule will take care of outbound traffic, and Masq or SNAT rules can determine the public IP.  You shouldn't use a DNAT for SMTP if you're using the SMTP Proxy - see #2 in Rulz.  If you're already configured like Basic Exchange setup with SMTP Proxy, you might want to consider using Webserver Protection for external client access: Exchange WAF Guide.

    If you still have troubles with SNATs and Masq rules, please insert pictures of them.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • 0 in reply to BAlfson

    Thanks a Million Bob, I think things clarify...

    BAlfson said:

    "Remote Desktop Gateway should always go out to the Internet with Additional Address #2 (gateway.domain.com / 212.15.0.3) WAN#1 and return this way."

    The only thing that determines with which public IP the traffic leaves is the IP that the initiating traffic arrived onn (connection tracking).  Only your monitoring service's changing the IP of your FQDN makes any difference here.  Any Multipath rule would be redundant.

    Okay I think I got that, I use the Multipath rules mainly to make sure that traffic is leaving the right ISP while both ISPs are up, like manual load balancing. But I think the main key to get this properly running is to rely on the Connection Tracker and not trying to mess in its work. :-)

    BAlfson said:

    In your fail over scenario:

    "Mailserver has to go out over ISP2 WAN#2 (mail.domain.com / 9.5.10.100) and return this way (to DNAT it to the Mailserver and SPF/Anti SPAM configuration etc.)"

    I think I would have two MX records with the one at a lower priority that points at an FQDN for the second ISP.  A Multipath rule will take care of outbound traffic, and Masq or SNAT rules can determine the public IP.  You shouldn't use a DNAT for SMTP if you're using the SMTP Proxy - see #2 in Rulz.  If you're already configured like Basic Exchange setup with SMTP Proxy, you might want to consider using Webserver Protection for external client access: Exchange WAF Guide.

    My external MX Setup is using round robin at the moment, with one MX record and one A record for each ISP, the external DNS Service is replacing the additional A if its not reachable anymore. I have very short TTLs on them and DNS propagation of the DNS Service provider is really blazing fast over the world (only seconds tbh). SPF configuration does reflect both IPs. Actually I can't use WAF, as we haven't this licensed on the UTM yet. I have taken the guide you've linked into account already the time I was configuring the Exchange SMTP Proxy, so my Email Protection setup is pretty similar.

    DNAT is only used for HTTP/HTTPS services like OWA, Autodiscover etc. SMTP is not affected, as incoming SMTP is handled by the Email Protection Proxy. I don't use the Email Proxy for outbound Email though and thus I needed to make sure that outgoing SMTP is either sending over WAN#1 Additional Address #1 or WAN#2 Primary Address.

    I changed my whole NAT configuration now as follows, as conclusion of our conversation:

     

    Masquerading:

    • Rule1: Mailserver --> WAN #1 / Additional Address #1
    • Rule2: Mailserver --> WAN #2 / Primary Address
    • Rule3: RDSH --> WAN #1 / Additional Address #2
    • Rule4: RDSH --> WAN #3 / Primary Address
    • Rule5: Internal Network --> Uplink Interfaces / Primary Address
    This is resulting from two assumptions: Masquerading Rules take their order into account now and therefore can replace SNAT while beeing more intelligent because Masquerading Rules only apply to Interfaces which are actually used (= not in a down state). Mailserver is masked over the correct IP on WAN#1 with Rule1 until the WAN#1 Interface goes down and therefore Rule2 is taken. Same logic for RDSH and Rule3/4. All other traffic is masked with the currently active Primary Interface Address Rule5.

    NAT:

    No more SNAT. I have only DNAT to distribute incoming connections to the right servers: 

    • Rule1: DNAT ANY HTTP/HTTPS arriving on WAN#1 Additional Address #1 -> go to Mailserver
    • Rule2: DNAT ANY HTTP/HTTPS arriving on WAN#2 Primary Address -> go to Mailserver
    • Rule3: DNAT ANY HTTP/HTTPS arriving on WAN#1 Additional Address #2 -> go to RDSH
    • Rule4: DNAT ANY HTTP/HTTPS arriving on WAN#3 Primary Address -> go to RDSH
    Multipath Rules:
    The purpose of these Multipath rules is only to route certain traffic over a specific WAN Interface while both ISPs are up.
    • Rule1: SMTP via WAN#1 [by Interface] / Mailserver --> SMTP --> Internet IPv4 --> WAN#1 (skipping on Interface error)
    • Rule2: RDSH-GW via WAN#1 [by Interface] / RDSH --> HTTP/HTTPS --> Internet IPv4 --> WAN#1 (skipping on Interface error)
    • Rule3: Printers via WAN#2 [by Interface] / Printer Network --> ANY --> Internet IPv4 --> WAN#2 (skipping on Interface error)
    • Rule4-6: same as 3 just for DHCP Client Network, IP Phones and so on.

    This should also solve my third problem, which was that in the case where only ISP2 is active the Upload Balancing weight setup is 0 for WAN#2 and WAN#3 therefore acting in Load Balancing Mode and both addresses are used. This messed up my outoing SMTP as mails were send over both public IPs of ISP2 (WAN#2/WAN#3), while SPF config only reflected WAN#2 Primary Address. This should now also be covered by Masquerading Rule2.

    I will give this configration a try tomorrow evening and unplug some Interfaces :-). 

    What do you think of the above?

     

    Regards,

    Martin

Reply
  • 0 in reply to BAlfson

    Thanks a Million Bob, I think things clarify...

    BAlfson said:

    "Remote Desktop Gateway should always go out to the Internet with Additional Address #2 (gateway.domain.com / 212.15.0.3) WAN#1 and return this way."

    The only thing that determines with which public IP the traffic leaves is the IP that the initiating traffic arrived onn (connection tracking).  Only your monitoring service's changing the IP of your FQDN makes any difference here.  Any Multipath rule would be redundant.

    Okay I think I got that, I use the Multipath rules mainly to make sure that traffic is leaving the right ISP while both ISPs are up, like manual load balancing. But I think the main key to get this properly running is to rely on the Connection Tracker and not trying to mess in its work. :-)

    BAlfson said:

    In your fail over scenario:

    "Mailserver has to go out over ISP2 WAN#2 (mail.domain.com / 9.5.10.100) and return this way (to DNAT it to the Mailserver and SPF/Anti SPAM configuration etc.)"

    I think I would have two MX records with the one at a lower priority that points at an FQDN for the second ISP.  A Multipath rule will take care of outbound traffic, and Masq or SNAT rules can determine the public IP.  You shouldn't use a DNAT for SMTP if you're using the SMTP Proxy - see #2 in Rulz.  If you're already configured like Basic Exchange setup with SMTP Proxy, you might want to consider using Webserver Protection for external client access: Exchange WAF Guide.

    My external MX Setup is using round robin at the moment, with one MX record and one A record for each ISP, the external DNS Service is replacing the additional A if its not reachable anymore. I have very short TTLs on them and DNS propagation of the DNS Service provider is really blazing fast over the world (only seconds tbh). SPF configuration does reflect both IPs. Actually I can't use WAF, as we haven't this licensed on the UTM yet. I have taken the guide you've linked into account already the time I was configuring the Exchange SMTP Proxy, so my Email Protection setup is pretty similar.

    DNAT is only used for HTTP/HTTPS services like OWA, Autodiscover etc. SMTP is not affected, as incoming SMTP is handled by the Email Protection Proxy. I don't use the Email Proxy for outbound Email though and thus I needed to make sure that outgoing SMTP is either sending over WAN#1 Additional Address #1 or WAN#2 Primary Address.

    I changed my whole NAT configuration now as follows, as conclusion of our conversation:

     

    Masquerading:

    • Rule1: Mailserver --> WAN #1 / Additional Address #1
    • Rule2: Mailserver --> WAN #2 / Primary Address
    • Rule3: RDSH --> WAN #1 / Additional Address #2
    • Rule4: RDSH --> WAN #3 / Primary Address
    • Rule5: Internal Network --> Uplink Interfaces / Primary Address
    This is resulting from two assumptions: Masquerading Rules take their order into account now and therefore can replace SNAT while beeing more intelligent because Masquerading Rules only apply to Interfaces which are actually used (= not in a down state). Mailserver is masked over the correct IP on WAN#1 with Rule1 until the WAN#1 Interface goes down and therefore Rule2 is taken. Same logic for RDSH and Rule3/4. All other traffic is masked with the currently active Primary Interface Address Rule5.

    NAT:

    No more SNAT. I have only DNAT to distribute incoming connections to the right servers: 

    • Rule1: DNAT ANY HTTP/HTTPS arriving on WAN#1 Additional Address #1 -> go to Mailserver
    • Rule2: DNAT ANY HTTP/HTTPS arriving on WAN#2 Primary Address -> go to Mailserver
    • Rule3: DNAT ANY HTTP/HTTPS arriving on WAN#1 Additional Address #2 -> go to RDSH
    • Rule4: DNAT ANY HTTP/HTTPS arriving on WAN#3 Primary Address -> go to RDSH
    Multipath Rules:
    The purpose of these Multipath rules is only to route certain traffic over a specific WAN Interface while both ISPs are up.
    • Rule1: SMTP via WAN#1 [by Interface] / Mailserver --> SMTP --> Internet IPv4 --> WAN#1 (skipping on Interface error)
    • Rule2: RDSH-GW via WAN#1 [by Interface] / RDSH --> HTTP/HTTPS --> Internet IPv4 --> WAN#1 (skipping on Interface error)
    • Rule3: Printers via WAN#2 [by Interface] / Printer Network --> ANY --> Internet IPv4 --> WAN#2 (skipping on Interface error)
    • Rule4-6: same as 3 just for DHCP Client Network, IP Phones and so on.

    This should also solve my third problem, which was that in the case where only ISP2 is active the Upload Balancing weight setup is 0 for WAN#2 and WAN#3 therefore acting in Load Balancing Mode and both addresses are used. This messed up my outoing SMTP as mails were send over both public IPs of ISP2 (WAN#2/WAN#3), while SPF config only reflected WAN#2 Primary Address. This should now also be covered by Masquerading Rule2.

    I will give this configration a try tomorrow evening and unplug some Interfaces :-). 

    What do you think of the above?

     

    Regards,

    Martin

Children
Unfiltered HTML