Lots of default drops in firewall log

Hi Jay Jay,

This is Chris Hill here posting from a different account since I've changed job.

You need to think about this as a failure of the 'stateful' nature of the Sophos firewall. There are three basic 'chains' in iptables (along with others), INPUT, OUTPUT and FORWARD. INPUT and OUTPUT handle all traffic destined to or coming from the host itself; FORWARD handles all traffic which is 'passing through' the host using IP forwarding. Because the web filtering operates as a 'local' application on the host (even when in transparent mode), INPUT and OUTPUT are both relevant rules.

Under most circumstances in order to make configuration simpler, modern 'stateful' firewalls have a rule near the top of the OUTPUT chain which allow any traffic 'established' or 'related' to an existing connection to get 'through' the OUTPUT rule. This means that instead of creating two rules for any given service e.g.:

iptables -A INPUT <server source port> <any client destination port>

iptables -A OUTPUT <any client source port> <server destination port>

you can just create the rule on the INPUT side where the connection is first initiated:

iptables -A INPUT <server source port> <any client destination port>

and the ESTABLISHED, RELATED rule will handle the OUTPUT for you. This is the basic 'stateful' firewalling.

BUT that ESTABLISHED, RELATED rule is *essential* for smooth operation. Try disabling it - you'll find that clients cannot initiate connections - the initial packet will be received, but the moment the server tries to respond, it's blocked via the OUTPUT rule.

What's happening here is that the ESTABLISHED, RELATED rules aren't treating the final 'ACK, FIN' or 'RST' from the REMOTE SERVER to the INTERNAL CLIENT in an abnormally terminated connection as ESTABLISHED or RELATED - so they are instead dropped. These are valid packets which are required to be RFC compliant, as well as filling up the logs, so allowing them through is better than not, and saves a (tiny amount) of memory on your client's state table (each client on your network is waiting for 'ACK, FIN' or 'RST' to completely close the connection - it will get closed automatically after a while, but it has to wait for the timeout).

The reason you're seeing a source IP from the INTERNET whereas the source MAC is the LAN interface is probably because the web filtering is operating in transparent mode. In other words, the Sophos box is 'pretending' to 'be' the remote IP address in its replies instead of 'being' itself - so on a Layer 3 level the IP is Internet-based, but on Layer 2 the MAC address is local. You are seeing this on the OUTPUT interface though because it's still a local service. Basically it's a side-effect of transparent filtering - if you were accessing the web filtering service using the 'non-transparent' port 8080, you'd have the local IP address to match the MAC as well.

ChrisAtMaf Thanks for the detailed response.  I've read it over a few times now to digest.

If i'm understanding correctly the bottom line is the webfiltering isn't allowing certain connections to close properly (per spec). This results in the "noise" which manifests itself as default drops in the firewall. 

Your previous thread was from 5 years ago. As this issue remains on going, one would think this implementation of webfiltering is by design. Wonder what the behavior is with squid on pfsense.

All that noise in the firewall log is for the most part innocuous other than inflating logsize and making it more difficult to recognize legitimate external connection attempts. What would be the logic in this type of design? The firewall logs should contain useful and meaningful entries. This dilution of the data doesn't help.

What do those implementing UTM in an enterprise environment (hundreds of users) do to combat all this firewall noise in the log?

Thank you for clarifying the source mac and internet ip discrepancy. What you're saying is those packets are originating from the UTM, but IP is spoofed to the external source (wan) of those packets.  I get that.

Jay Jay You're pretty much spot on, except the issue is more with the firewall configuration ('network protection' rules in Sophos) and the way the stateful firewall rules operate than the web filtering implementation. A similar issue us reported here: https://unix.stackexchange.com/questions/223151/why-do-some-tcp-reset-packets-show-up-in-my-iptables-log/223240#223240
It is down to the way that Sophos have written the iptables rules. Other firewall configurations may not restrict OUTPUT traffic (since it tends to be generated by the host itself), or not log dropped OUTPUT packets (again, it's a fairly low risk) or include a rule that specifically drops 'invalid' packets without logging (some say this is best practice when using RELATED, ESTABLISHED - https://stackoverflow.com/questions/251243/what-causes-a-tcp-ip-reset-rst-flag-to-be-sent/57404552#57404552)

TL;DR - it's an issue created by the way the 'network protection' firewall rules  are created/ordered by the Sophos engineers rather than a specific web filtering component issue. It could probably happen on the mail protection component under the right circumstances. But web browsing is particularly likely to generate ACK/FIN or RST when a user clicks 'cancel' on a browser. Not many other applications do that.

Jay Jay You're pretty much spot on, except the issue is more with the firewall configuration ('network protection' rules in Sophos) and the way the stateful firewall rules operate than the web filtering implementation. A similar issue us reported here: https://unix.stackexchange.com/questions/223151/why-do-some-tcp-reset-packets-show-up-in-my-iptables-log/223240#223240
It is down to the way that Sophos have written the iptables rules. Other firewall configurations may not restrict OUTPUT traffic (since it tends to be generated by the host itself), or not log dropped OUTPUT packets (again, it's a fairly low risk) or include a rule that specifically drops 'invalid' packets without logging (some say this is best practice when using RELATED, ESTABLISHED - https://stackoverflow.com/questions/251243/what-causes-a-tcp-ip-reset-rst-flag-to-be-sent/57404552#57404552)

TL;DR - it's an issue created by the way the 'network protection' firewall rules  are created/ordered by the Sophos engineers rather than a specific web filtering component issue. It could probably happen on the mail protection component under the right circumstances. But web browsing is particularly likely to generate ACK/FIN or RST when a user clicks 'cancel' on a browser. Not many other applications do that.