FOS18: Access to webproxy (Direct Mode) does require allowing HTTP and HTTPS within firewall rule?

Dear all,

in FOS17.5 we allowed access to the XGs webproxy on port 3128 by creating a firewall rule with allowing traffic from LAN to WAN with allowed service 3128 as documented in the following KB article: https://community.sophos.com/kb/en-us/125585.

After migrating to SFOS 18.0.0 GA-Build379 this does not work anymore. When a LAN client accesses the webproxy on port 3128, no website can be opened and the connction on the client times out. In the XGs logfile I can see the following:

1. Traffic from the LAN client to the webproxy (3128) is allowed
2. Traffic from the XGs WAN interface to the destination website is dropped

When adding HTTP & HTTPS in addition to tcp/3128 in the firewall rule everything is working fine - that however means, that transparent proxying is also allowed. To disable the transparent proxy, I need to create an additional firewall rule with allowing HTTP & HTTPS from LAN to WAN with Web Policy set to deny as mentioned in https://community.sophos.com/kb/en-us/132117.

My firewall rule for allowing access to the webproxy looks like the following:

Is this behavior normal in FSOS18 because of the new drop rule at the end, so that in addition to tcp/3128 HTTPs & HTTPs always have to be explicitly allowed in the firewall rule as well?

Thanks
Michael

  • Hi  

    Please check explanation from the thread - https://community.sophos.com/products/xg-firewall/f/recommended-reads/116102/understanding-new-decoupled-nat-and-firewall-changes-in-v18

    8) What is the new disabled “Drop ALL” rule at the bottom of the firewall rule table?

    The default drop rule provides a visual indication to user/admin that if none of the firewall rules gets a match, traffic will be dropped.

    You reported about two specific challenges that admin faces in v17.x.

    1. New admins are confused about the default behavior on the firewall rule table – that is the behavior when no rule matches. The new disabled Drop ALL non-editable rule is a step to resolve this.
    2. Log viewer should show traffic being dropped by the default-drop behavior of the firewall rule table – this is planned to be released post v18.

    Currently, the logs that you see with firewall rule id ‘0’ are NOT for the traffic dropped by Drop ALL rule. In later EAP releases, we would replace them with “N/A” as those are for the traffic dropped before the firewall rule matches – for example – invalid traffic. And actual logs for traffic dropped by Drop ALL default behavior will be available in the release post v18. Meanwhile – as a workaround, one can add a drop rule at the bottom to log the dropped traffic not matched by any other firewall rule.

    Please also refer to this article - https://community.sophos.com/kb/en-us/131968

  • In reply to Keyur:

    Hi Keyur,

    thanks, I know that explanation. So it indeed is normal that in v18 HTTP and HTTPS need to be allowed in order for the Direct Web Proxy Access to work? This would mean that the steps in https://community.sophos.com/kb/en-us/125585 for configuring Direct Access proxy are not correct anymore (only allowing access to port 3128...)

    Best Regards

    Michael

  • The problem you are facing is most likely due to NAT.  It is possible that in your case the NAT was not migrated correctly to handle this scenario.  Delete the new firewall rule you created.

    Create a new NAT file (second tab) from Any to Any Service Any SNAT MASQ.  Try it then.  If it works you know it is a NAT problem.  You'll probably want to review and clean up all the NAT rules.  The migration does its best to make 18.0 work exactly like 17.5 but that can cause a tangle of NAT rules bound to firewall rules that don't look like they would if you created things natively.

  • In reply to Michael Dunn:

    Hi Michael,

    thanks for your reply. I actually already deleted the original rule that was migrated and created a new one. Regarding NAT I deleted all linked rules that were created during the mirgration and just enabled the new default NAT rule that MASQS everything from that enters the Inbound interface and leaves the Outbound interface:

    So if I understand you correctly even in v18 it should not be necessary to allow HTTP and HTTPS within the firewall rule just for using the Direct Proxy mode. To be honest I'm currently loosing the track here. Even more confusing is that in https://community.sophos.com/products/xg-firewall/f/firewall-and-policies/99918/direct-proxy-mode you wrote that for hybrid mode, port 3128 does not even need to be included at all:

    ..but that didn't work in my case either. Well, until a few minutes ago...where I figured out that this is working .... in combination with a Reject Rule as just posted: https://community.sophos.com/products/xg-firewall/f/firewall-and-policies/120741/fos18-reject-rule-is-allowing-traffic

    I'm really getting confused with the XG...or maybe the migration just broke it.

    Thanks
    Michael

  • In reply to layer9:

    This Standard vs Transparent Proxy is somehow the old School of doing proxy. 

    XG is moving to the Stream based, with will likely replace the "Proxy" scenarios, whenever it can. 

    As i posted in the other thread, Transparent Proxies are both, Standard and Transparent, but in the end they are proxies. They will take a request and ask the destination for the content. 

    As the rule set of XG implies, you have to deal with both. You have to allow the traffic coming from the Client (Port 3128) and you have to allow the traffic generated by the Proxy itself (Port 80/443). This will cause the duplicated need of having a firewall rule with 3128 and 80/443. 

    Most likely talking to customers, i am asking about the use case of "only allowing Port 3128". As maybe this model is in place for years. Maybe moving to Stream based could be a better solution after all?  

  • In reply to layer9:

    My quick take on it is:

    1) In general AFAIK any direct mode only config that worked in 17.5 should work in 18.0

    2) The proxy not being able to make an external connection, which appears to the admin/user as a timeout, can be a symptom of NAT rules.

     

    The underlying problem (reason for confusion) is that firewall rules are meant for traffic flowing from somewhere inside the network to somewhere outside the network.  Firewall rules are for traffic flowing through the firewall.  Direct web proxy mode is not traffic flowing through the firewall.  It is for traffic that goes from the client that then terminates on the XG port 3128.  After that the XG creates a new connection out from the XG to the web server.  In one sense because the traffic does not flow through the firewall (in that way normal traffic does) no firewall rule should be needed at all.  But we do need one, because the traffic does actually have to flow through the firewall twice.  Once coming in to the web proxy and once coming out.  The firewall and nat rules for this are...  not obvious.  :)

     

     

  • In reply to LuCar Toni:

    Hi Toni,

    thanks for you reply and explanation. I'm actually fine with using both port 3128 and 80/443 within the rule - I was just asking because I got confused because the behavior in v18 now is different than in v17.5 (in that a rule with only allowing port 3128 doesn't work anymore). I also can confirm (after another test) that the behavior in v17.5 is the same when a manually created drop all rule is inserted at the end of the rule table. That should mean the information in KB132117 (How to resolve issues related to web proxy when a drop all firewall rule is added) always applies to v18 if HTTP and HTTPS are not specified in the rule.

    In general I find it a little confusing that access to the local webproxy service also needs to be allowed within in a firewall rule since access to port 3128 is handeled in a Local ACL for device access anyway. After all, access to other local ports like SSH and HTTPS for management don't need to be specified within firewall rules either.

    I guess the reason of this is to have the ability to further restrict who actually can access the webproxy. From my point of view it however would be easier if access to port 3128 would solely be based on the Device Access settings and the firewall rules were used only for specifying who actually can access HTTP/HTTPS/FTP over it. 

    That is: The setting in Device Access should make the webproxy respond to the zone, the firewall rules specify who can actually send traffic over it. If no firewall rule is configured with that access, the webproxy should answer to hosts in the zone that proxy access is denied.

    My other thread regarding the issue with allowed access in conjunction with a reject-all rule shows that this should be possible - since in that scenario access to port 3128 is not configured in a firewall rule at all but only via Device Access.

    btw I agree with you in that stream based solutions are the future.

    Thanks
    Michael

  • In reply to Michael Dunn:

    Hi Michael,

    thanks for your reply as well. As written in my reply to Toni I did another test with version 17.5 (had a snapshot from before the migration to v18) and can reproduce the issue 1:1 if I manually create a drop all firewall rule and insert it at the end of the ruleset. In that case, access to the web proxy does not work anymore unless I specify HTTP and HTTPS access in the firewall rull as well or follow the instructions in KB132117 and create an additional rule.

    This also should demonstrate that NAT is not the issue here since in 17.5 the MASQ settings for the webproxy is set directly within the firewall rule as detached NAT was not available yet.

    I however noticed that before the migration from v17.5 to v18 my web proxy firewall rule did not include port 3128 but only http and https (as I have tested around before the migration). If only port 3128 is specified as service it might be possible that the migration script recognizes this and adds http and https to the rule as part of the migration. Would be interesting to see an v18 installation where access to the webproxy works without allowing http and https access in a firewall rule as well.

    Also thanks for your additional explanation. I also see that the situation is a special case in terms of firewall rules vs. local termination at the proxy. However I think it could be solved with less confusion (see my reply to Toni for details). 

     

    From my point of view however the main problem is the block all rule at the end - be it a manually created one or the new default one in v18. I guess this needs to be optimized in the future.

    Best Regards
    Michael

  • In reply to layer9:

    Talking about this in the other thread, agreeing the improvement of the default drop in the product for the future is needed. 

  • In reply to LuCar Toni:

    Thanks,

    two last questions on this one. As port 3128 needs to be included in a firewall rule for allowing direct access to the webproxy of the XG: If I set the destination to Any Host this should also allow clients access to port tcp/3128 on other hosts on the Internet, correct? To restrict access to tcp/3128 ONLY to the XGs webproxy and enable HTTP/HTTPS over it I guess we have to create two rules:

    1. One rule for port 3128 with destination set to the internal interface / IP address of the XG (allows access to web proxy)
    2. Another rule for http/https to WAN and Any Host (allows http/https over webproxy)


    Secone question: I noticed that if another protocol than HTTP/HTTPS over the webbroxy in Direct Mode needs to be used (like FTP), this protocol has to be added to the firewall rule as well - even though the general webproxy settings list other destination ports in its config as allowed:

     

    However, unless FTP is not allowed in the firewall rule as well, FTP access over the webproxy does not work. Is this correct as well? As this is also misleading - the list from the screenshot above implies that once direct access to the webproxy over port 3128 is used, all ports listed are implictily allowed over it.

    Best Regards
    Michael

  • In reply to layer9:

    layer9

    two last questions on this one. As port 3128 needs to be included in a firewall rule for allowing direct access to the webproxy of the XG: If I set the destination to Any Host this should also allow clients access to port tcp/3128 on other hosts on the Internet, correct? To restrict access to tcp/3128 ONLY to the XGs webproxy and enable HTTP/HTTPS over it I guess we have to create two rules:

    1. One rule for port 3128 with destination set to the internal interface / IP address of the XG (allows access to web proxy)
    2. Another rule for http/https to WAN and Any Host (allows http/https over webproxy)

    Sounds correct.  The QA in me says you'd have to test it to be sure.

     

     

    Secone question: I noticed that if another protocol than HTTP/HTTPS over the webbroxy in Direct Mode needs to be used (like FTP), this protocol has to be added to the firewall rule as well - even though the general webproxy settings list other destination ports in its config as allowed:

    However, unless FTP is not allowed in the firewall rule as well, FTP access over the webproxy does not work. Is this correct as well? As this is also misleading - the list from the screenshot above implies that once direct access to the webproxy over port 3128 is used, all ports listed are implictily allowed over it.

    Again, sounds correct.  However the list of ports here is not configuring the firewall, it is configuring the web proxy.  This is the list of ports the web proxy will accept, rejecting anything on other ports.  It is important security that the web proxy maintain its own list, as there are easily ports you might want to allow normal services on but you wouldn't want to allow HTTP on.

    Take this request:

    curl -x myfirewall:3128 http://www.example.com:1111/

    This tells the curl client to make a connection to myfirewall on port 3128, to speak HTTP, and to ask that proxy to make a connection out to port 1111.  When myfirewall receives it on port 3128 it first tries to match it to a firewall rule that would tell it what web policy should be used.  If it finds a firewall rule it hands the connection to the web proxy telling it which policy to use (if there is no firewall rule then it hands it to the web proxy telling it Deny All).  It is important to note that 1111 is not involved here because that is data within the connection, part of the HTTP header, not the connection itself.  The web proxy then looks at and parses the HTTP headers.  It looks at its policy to determine the destination category is allowed or not.  It also looks at its policy to determine if the destination port (1111) is allowed or not - this is the configuration you posted.  If it isn't allowed then it sends back a block page saying it is a blocked destination.  After the web proxy determines it is allowed it tries to make an external connection to that port (1111).  The firewall now gets involved - is the XG allowed to make connections to that port.  If the firewall allows it fine, but if the firewall does not the connection hangs open for 60 seconds before the proxy gives up and says the destination timed out (it has no idea that the firewall was the reason for the timeout, it just knows it never got a reply), and the proxy returns a error page to the client.

    This scenario is also true if you do ftp to port 3128, in something called ftp-over-http.  There is an extra process involved on the XG but the principal is the same.

  • In reply to Michael Dunn:

    Hi Michael,

    thank you very much for your further clarification which now makes sense to me. I guess my confusion comes form seeing everything from an iptables perspective. Considering an installation of Linux with Squid as webproxy I would configure the following for allowing http/https access over the webproxy:

    1. Allow traffic to port 3128 of the webproxy on the local machine in the INPUT chain of iptables
    2. Allow traffic to http/https from the webproxy service to the Internet in the OUTPUT chain of iptables
    3. Further specifying allowed clients and filter settings in the Squid configuration

    I guess those steps are equivalent to the following on the XG:

    1. Allowing traffic to port 3128 of the webproxy is configured in the firewall rule from LAN to the XG interface/port 3128
    2. Allowing traffic to http/https form the webproxy to the Internet is configured in the firewall rule from LAN to WAN/port 80/443 Star
    3. Filter settings are derived from the web filter settings of a matching firewall rule

    I'm however not sure at which step the rule for allowing web proxy access in Device Access comes into play. Maybe in step 3: If the webproxy receives packets on port 3128 (after the rule in step 1 has allowed them) it will actually not allow access if the checkbox for the corresponding source zone in Device Access is not set.

    Star I guess what caused confusion to me is that the rule from step 2 (outgoing http/https access from webproxy to the Internet) does not indicate that it is also used as an "OUTPUT" rule (from an iptables perspective) for local outgoing connections from the webproxy as well. For that I would have expected that we have to configure the rule with source zone WAN and source network as the external interface of the XG (as that is used from the webproxy for outgoing connections to the Internet). However I've tested that and it doesn't work:

    From that perspective it also seems not logical that in the rule for allowing access to the webproxy on port 3128 (step 1) we have to configure WAN as destination (since actually the packets are destined for the webproxy listening on an interface in the LAN zone):

    I however somehow can understand the logic of the XG as from a practical perspective the configuration is easier - we can set up everything in one rule only and have not to think of different chains and so on.

    I am not sure if you can follow me on my thoughts - maybe I've worked to long with iptables and I'm just thinking to much :)

    Best Regards
    Michael

  • In reply to layer9:

    I don't know iptables that well, so I personally don't think in those terms.

    layer9

     

    1. Allowing traffic to port 3128 of the webproxy is configured in the firewall rule from LAN to the XG interface/port 3128
    2. Allowing traffic to http/https form the webproxy to the Internet is configured in the firewall rule from LAN to WAN/port 80/443 Star
    3. Filter settings are derived from the web filter settings of a matching firewall rule

    I'm however not sure at which step the rule for allowing web proxy access in Device Access comes into play. Maybe in step 3: If the webproxy receives packets on port 3128 (after the rule in step 1 has allowed them) it will actually not allow access if the checkbox for the corresponding source zone in Device Access is not set.

    When a packet arrives at an interface the XG can ignore it (eg "closed port") or it can look at it and send it to the firewall to be processed.  This is what the Device Access controls.  If the packets arrives on a port and the port is WAN then "Web Proxy" is unchecked and the packet is ignored.  If it arrives on LAN and Web Proxy is checked, it is sent to the firewall which looks at its rules and may send it to the proxy or may choose to ignore it.  Note: what matters is the zone of the port the packet arrives at, not the zone of the port the packet is addressed to.

    layer9

     Star I guess what caused confusion to me is that the rule from step 2 (outgoing http/https access from webproxy to the Internet) does not indicate that it is also used as an "OUTPUT" rule (from an iptables perspective) for local outgoing connections from the webproxy as well. For that I would have expected that we have to configure the rule with source zone WAN and source network as the external interface of the XG (as that is used from the webproxy for outgoing connections to the Internet).

    In 17.5 and earlier you did not need to configure a rule, the "default drop rule" was not applied to internal->WAN connections.  You only needed a rule allowing 80/443 if there was an explicit Drop All rule.  It appears that in 18.0 it is now needed all the time, likely due to some system hardening.  When I get time (haha) I'm retest and update the KB (https://community.sophos.com/kb/en-us/132117).

  • In reply to Michael Dunn:

    Hi Michael,

    thank you very much for your help - all information you provided were very helpful to me! I would be interested to see your opinion on the following topics as well to be honest:

    https://community.sophos.com/products/xg-firewall/f/firewall-and-policies/120870/cannot-access-server-published-through-waf-from-lan-over-webproxy 

    https://community.sophos.com/products/xg-firewall/f/firewall-and-policies/120866/dnat-rule-in-fos18-reflexive-nat-rule-for-return-traffic-from-published-server-not-being-used 

    But I understand you indeed don't have a lot time.

    Thanks again and have a great weekend
    Michael

  • In reply to layer9:

    Neither of those topics are in areas I know well, and I cannot offer any help with, sorry.