My experience with WAF has been disappointing. I have had so many false positives that about all I can reliably enforce is Site Path Routing. I have found no documentation that provides "hints and tricks" to minimize problems and maximize success. Level 1 support simply advises that if a feature causes problems, turn it off.
During the sales cycle, Forms Hardening seemed conceptually easy to implement with full transparency. WAF simply needs to ensure that the returned form has the same field names as the one sent out, that the max_length parameter is observed on all returned fields, and that the returned character set is consistent with the requested character set. In practice, I have never had a site which did not throw Form Hardening errors.
Form Hardening and Cookie Hardening have to be used together, so Cookie Hardening fails because Form Hardening fails.
Strict rules throw false positives so I add them to the ignore list without really understanding the implications of the exception, then I hit a wall because I have what appears to be a false positive on an infrastructure rule which should never be disabled. So instead I disable Strict Rules.
Bad Client checking uses the email RBLs. The email RBLs include residential IP addresses in their blocklists because residential users on dynamic IPs should not be running mail servers (but might be running mail-sending malware). I don't want to abandon those RBLs for email protection, so I have turned Bad Client checking off for WAF protection. (One person in this forum points out the Bad Client checking can be enabled without the RBL lookup, which performs some undocumented filtering based on GEO-IP.)
Of course, it is difficult to know during testing whether the tests have been extensive enough to expose all of the needed WAF exceptions. Once the site is in production, it is extremely difficult to know whether a log entry represents a false positive or not. The logs are obscure, but even after studying them carefully, it seems impossible to know how to respond to them.
Has anyone figured out how to make Form Hardening work, or figured out whether certain development practices that can either cause it to fail or allow it to succeed?
Do we even know whether WAF supports both HTML4 and HTML5? Are there different limitations based on the HTML version?
Does anyone have any advice for making the other features useful?
Thanks for sharing the concerns with details. I admit that there's not much documentation available for the WAF in UTM9. With WAF, we always have to take a careful approach. I'll like to introduce this WAF troubleshooting article for some of the basic issues a user might face.For the form hardening issues caused by the form hardening HTML-parser changing the initial web request, you need to use the "Never change HTML during Static URL Hardening or Form Hardening" option, which can be enabled in the exceptions. For the other concerns, it would be different for each situation.Regarding Rigid filtering, you're never supposed to add the Infrastructure rules in the exception, it is clearly mentioned here. If you face any issue where your requests are rejected due to this, it might be something that needs to be checked with the Senior Support team.Regarding limitations with HTML4 and HTML5, I don't know for sure if we Support it or not, I will try to get in touch with someone who knows and will update you here.Regarding documentation or the best practice guide, I think we can prepare something that can help everyone but that will take some time.
Since the Best Practice Guide has not been forthcoming, I decided to beat my head against this wall again. Below is my current understanding of the Webserver Protection subsystem and how to make it behave. I plan to make changes best on my learning or corrections received from others, so I will start with a revision timestamp:
Last Revisioin: 2019-08-20
Securing Web Application Firewall (WAF)
WAF detects and logs three types of incidents:
Session events include disconnects and timeouts. Their log entries are in the ModSecurity format, as explained below, but are of no particular importance for WAF configuration.
The proposed model for deployment is:
UTM WAF provides three aspects of path management:
The list of entry points is more complicated than might appear at first glance. Obviously, the application design entry points should be included. For example, the guidelines for Outlook Web Access recommend site paths for /owa and /ecp. These should also be configured in the URL hardening entry point list.
However, there are several entry points that browsers may reference unilaterally. It is recommended that these be included in the manual entry points list, even if the file does not exist, so that missing files are handled as session errors rather than being handled as security threats.
Problems will be reported in the log file as a URL Hardening error. If a false positive is detected, add the appropriate file to the Entry Points list.
UTM WAF is only compatible with sites that use static entry points. If entry points are randomly generated to uniquely identify a user or an event, URL Hardening will not be viable for that website. I assume that manual entry points can have querystrings appended, and still be accepted, but I have not actually tested this scenario.
If a web browser script generates a link dynamically, the link will not be hardened, because UTM can only harden static links that it sees as the web page is being sent from the server to the browser. In these cases, URL hardening must be disabled or the application must be redesigned.
The documentation says that URL Hardening and Form Hardening need to be used together. My inference from the provided examples is the following:
As stated above, From Hardening and URL Hardening should be used together. I have consistently experienced problems with unexplainable false positives, but I want to use URL Hardening, so I am currently re-testing. Stay tuned for details.
I have experienced many problems with Cookie Hardening, so I have been running with it disabled, but I have begun retesting.
Content Filtering is enabled by checking the option for “Common Threats Filter”. The common threats are divided into categories, which can be enabled or disabled individually:
Each of these categories is associated with a set of rules, with each rule uniquely identified by a rule ID. When a rule fires, a log entry is created. The log entry will include the Rule ID.
False positives are identified during log review. Details of log interpretation will be addressed later in this document.
The “Rigid Filtering” option causes all of the filtering rules to be applied more strictly. The documentation warns that this is likely to create false positives. So if it is enabled and false positives are occurring, the first step should be to disabled Rigid Filtering.
If the website does not use a SQL database, the SQL Injection Attacks can reasonably be disabled, which may prevent false positives related to this category.
For all other false positives, there are two options for creating an exception:
The documentation indicates that this option uses two sources: the Maxmind GeoIP database and the Sorbs reputation block list (RBL).
The minimum functionality, based on GeoIP, appears to be applicable to most situations, and should be enabled.
Like most other RBLs, the Sorbs list is optimized for email filtering, so it blocks dynamic IP addresses, especially dynamic IP addresses associated with residential ISP service. If you expect to receive connections from home users, or even from small businesses with a static IP address, you should disable the RBL check by using the option to “Skip remote lookups for clients with bad reputation.”
These options are applicable to websites that accept file uploads.
The “Pass Outlook Anywhere” should be enabled if (and only if) the firewall profile is being used for an Outlook Anywhere website.
The WAF log has two types of entries, UTM standard-format entries and ModSecurity WAF entries. The two entries types have very different formats.
The UTM log entries simply document the request that was received and the status code that was returned, similar to the contents of an IIS webserver log file. The example below has been anonymized and reformatted onto multiple lines for readability. These substitutions have been used during anonymization:
Example 1: UTM-format log entry
2019:08:18-01:15:03 utmname httpd:
The ModSecurity WAF log entries use tokens enclosed in square brackets, and free-form text between the tokens, as in these examples:
Example 2: URL Hardening Error entry
2019:08:18-01:15:12 defense httpd:
No signature found, URI: www.example.com/webpage.asp
Example 3: Protocol violation Error Entry
2019:08:18-01:15:12 utmname httpd:
ModSecurity: Warning. String match "HTTP/1.1" at REQUEST_PROTOCOL.
[msg "Pragma Header requires Cache-Control Header for HTTP/1.1 requests."]
For the URL Hardening error, a false positive is corrected by adding the file to the Entry Points List.
For the Protocol Violation error, the token [id "960020"] indicates the rule number. A false positive can be corrected either by adding a Skip Filter Rule entry for 960020, or by unchecking “Protocol Violations” from the Command Threat Filter Categories. Rule IDs are always a six-digit number.
The bad news is that enabling form and cookie hardening broke the web interface to my email system - users received an error message of the form "oops, IIS server threw an error." I have not yet parsed the log file to determine which type of hardening was most like the cause of the problem. 9Because someone will ask, I will mention that is not an OWA/Exchange environment, so the OWA configuration guide is not directly applicable.)
Apparently monitor-mode hardening is not transparent to the back-end application, because UTM alters the cookie or form to apply the hardening, but does not successfully strip the information back off on the return trip. Shame on me for not testing immediately after making the change, but if makes me afraid to try again on any website.
One of the issues in my original post was, "After I go into production, how do I know if an alarm is a true attack or merely a false positive that was not exposed during testing?"
This is the approach I am taking right now:
I have recently discovered these resources, which I am only beginning to digest:
The ModSecurity Handbook. Available online in HTML format. Available as a PDF download if you create a (free) account on the site. I have also signed up for their SSL/TLS newsletter.
OWASP Best Practice Recommendations for WAF
Neither of the previous documents have the information that I wanted.
I found this page with the ModSecurity reference manuals for both v2 and v3. I have not yet determined which version is in my UTM firmware.
When I do a web search for ModSecurity and hardending, most of the links are to this forum.
I cannot find anything that explains when the three hardening methods (URL, Form, Cookie) will definitely work, when they will definitely fail, or how to predict when they might fail. With all three disabled, it seems like I have lost a lot of protection. But I do not seem to have a safe way to even find out if they are usable, because I cannot assume that Monitor mode is non-disruptive.
A have determined that every log entry with a rule ID will have tokens for the file and line number which define the rule. An example:
The path is actually truncated. Full path is /var/chroot-reverseproxy/usr/apache/conf/waf/modsecurity_crs_protocol_anomalies.conf
There are other file types in that folder, but all of the WAF log entries refer to one of the *.conf files.
I looked at one of those files to see if i could parse it to build a reference list of the form: Rule Category, Rule ID, Rule Description
Unfortunately, the rule definitions use a complex syntax which would be non-trivial to parse. However, this does allow us to see that specific GUI options can be presumed to enable or disable specific *.conf files, as follows:
Presumed relationships betwen Common Threat Filter Categories and *.conf filesmodsecurity_crs_protocol_violations.conf, Protocol Violationsmodsecurity_crs_protocol_anomalies.conf, Protocol Anomaliesmodsecurity_crs_request_limits.conf, Request Limitsmodsecurity_crs_http_policy.conf, HTTP Policymodsecurity_crs_bad_robots.conf, Bad Robotsmodsecurity_crs_generic_attacks.conf, Generic Attacksmodsecurity_crs_sql_injection_attacks.conf, SQL Injection Attacksmodsecurity_crs_xss_attacks.conf, XSS Attacksmodsecurity_crs_tight_security.conf, Tight Securitymodsecurity_crs_trojans.conf, Trojansmodsecurity_crs_outbound.conf, Outbound
Presumed relationship between other options and *.conf files10-REQUEST-IP-REPUTATION.conf, Block Clients with Bad Reputation,Filtering
Other *.conf files, purpose not investigatedbase.confmodsecurity_crs_common_exceptions.confmodsecurity_crs_correlation.conf,modsecurity_crs_inbound_blocking.confmodsecurity_crs_outbound_blocking.conf