413 Request entity too large

I have been working with Sophos on this very issue since 7/25/19 as well! what a strange small world we live in.

So far I have been escalated through the tiers and I'm currently waiting for a developer to return a phone call. 

I know this much so far, The error is pertaining to the setting defined for ModSecurity : SecRequestBodyNoFilesLimit.  

The default setting is: 1048576bytes of course you already know that. 

It seems apparent that the UTM machines and the XG get and store this information in very different ways and even the top level engineers aren't sure how to apply the tweak.

The problem I see so far is this, older UTM KB documents tell the engineers to bump the tblwafsecurityprofile record item sec_request_body_no_files_limit ="<new value>" restart the WAF and call it good..

This doesn't work on the XG.

So for example lets say you update the db to the size you want, and then to trigger a regeneration of the /cfs/waf/reverseproxy.conf (Included to Apache via httpd.conf) file you add an exception(skip) to your policy from the GUI and click save, The page generates a new /cfs/waf/reverseproxy.conf file with the new exception and updates the database.

But if you take a look at the record now, What you will see is that the db will revert back to the default size setting.

The SecRequestBodyNoFilesLimit is defined in another file somewhere..and this is where the GUI is getting the default value for this setting that it uses to populate the /cfs/waf/reverseproxy.conf file that controls your WAF.

I can see that /usr/apache/conf/waf/base.conf contains an entry defining SecRequestBodyLimit 1GB (1073741824) which is ironically the hard stop max size for this configuration directive but for us we need to find where to define SecRequestBodyNofilesLimit..

I speculate that this is the file we're looking for.. I can only assume that if the configuration directives are not explicitly defined they will simply behave with their defaults.

Perhaps a simple addition of the entry would override the defaults to our size of choice? i.e. "SecRequestBodyNofilesLimit 10485760" for a 10Mb limit

The issue I'm dealing with is that we aren't permitted to browse the directory structure and make changes without the supervision of an engineer or you risk violating your service agreement, and since the XG doesn't have an ftp server enabled easy directory tree structure perusing isn't available.

I'm hoping to have a developer on the line today.. if so I will update this post and this issue should be in their KB.

If you hear anything let me know as well.

Good luck.

I have an XG at home that I could test this on, just let me know step by step what you are looking for and how to confirm it?

Learning is fun lol.....

Also, I only ask because I am following some of what you got here but I went into the /cfs/waf/reverseproxy.conf and noticed I have "SecRequestBodyNofilesLimit 10485760"  set.  So are you saying that this aspect is getting ignored in the WAF and the setting is being pulled from another config such as the base.conf?  

I think I'll be firing up a VM and trying it myself as well.. 

I think that if this is correct it would be a useful addition to the KB.

Let me know if you had a chance to try it out.

Your reply could save me a ton of time wasted spinning up a VM and pushing my other projects aside.

Thanks again for checking it out.

I attempted to add the 

SecRequestBodyNofilesLimit 10485760 right under the SecRequestBodyLimit in the base.conf  but I cannot seem to save the conf file, even with :wq!

I am thinking this may require to stop a service, apache perhaps? But on this aspect I am not 100% sure.

Darn.. 

It appears that the file is owned by root..

The permissions are not there for the user "admin" to edit it.

XG330_WP02_SFOS 17.5.7 MR-7# ls -l
-rw-r--r-- 1 root 0 1844 Jul 4 08:40 base.conf 

In order to make any changes to it the admin user would need to be a member of the sudoers group and/or you'd need root access.. (which you probably do)

https://community.sophos.com/products/xg-firewall/f/sophos-xg-firewall-general-discussion/76101/advanced-shell

Wonderful.

So the only way to test this now is to have their dev team do it.

I guess I'll be getting back on the phone.. mind you, the hold music loop is atrocious at best.. but hey why not, I haven't got anything better to do with my day.. 

No root huh?.. grrrr.. 

Yes I do not know why they don't replace the hold music with IT related news.

I've spoken to some people in some pretty high places and I'm still waiting to hear from someone at the very very top.

So as of the time of this post the only thing we can currently do to get this applied and working is to manually update the db record and force an update to the waf_reconfig.

First is that you need to know all of the exclusions(skips) you'll need to enter into the policy under the GUI. 

Add them them in and then save the policy. 

You also need to know the policy ID, so you would run:  psql -U nobody -d corporate -c "select * from tblwafsecurityprofile;"

(my query returns)

Look for the number at the beginning of the line listing your policy, in my case the ID I'm interested in is 7.

Next you'd need to apply the change to the value in the sec_request_body_no_files_limit record. in my case I'm defining it to be 10MB and I'm telling the query to apply it to policy ID = 7

psql -U nobody -d corporate -c "update tblwafsecurityprofile set sec_request_body_no_files_limit ="10485760" where id ="7";"

Then finally you need to force the system to update the important files with:

opcode waf_reconfig -t json -b '{"Entity": "waf_advanced_config", "Event": "UPDATE"}' -ds nosync

This will rebuild the reverseporxy.conf and get the WAF up and running with the new larger limit.. 

After this is done, If you make ANY changes to this policy in the GUI you will need to do the above process again because the default 1048576 value will be reinserted into the db and reverseproxy.conf bringing you back to the broken and unusable 413 issue.

Hi

We also have a remote sesion with support, and also he use the commands you mention:

"We run the following commands in the XG to increase the file size allowed in the WAF from 1048576 [1MB] to 52428800 [50MB]. With the following command:

SFVUNL_VM01_SFOS 17.5.7 MR-7# psql -U nobody -d corporate -c "update tblwafsecur
ityprofile set sec_request_body_no_files_limit ="52428800" where id ="7";"
UPDATE 1
.
SFVUNL_VM01_SFOS 17.5.7 MR-7# opcode waf_reconfig -t json -b '{"Entity": "waf_ad
vanced_config", "Event": "UPDATE"}' -ds nosync
200 OK
{ "status": "200", "statusmessage": "success" }
.
SFVUNL_VM01_SFOS 17.5.7 MR-7# psql -U nobody -d corporate -c "select name,id,sec
_request_body_no_files_limit from tblwafsecurityprofile;"
           name            | id | sec_request_body_no_files_limit
---------------------------+----+---------------------------------
 Exchange AutoDiscover     |  1 |                         1048576
 Exchange General          |  2 |                         1048576
 Microsoft Lync            |  4 |                         1048576
 Microsoft RDG 2008        |  5 |                         1048576
 Microsoft RD Web 2008     |  6 |                         1048576
 Exchange Outlook Anywhere |  3 |                         1048576
 WAF Police                |  7 |                        52428800

but we got the error again with files of more than 49KB, smaller than these worked fine.

On the IIS we increase this value "" uploadReadAheadSize "which by default is -49KB to a larger one, after this all the files that failed previously it was possible to load them, although with 30MB files this still failed, but this change lets us see that the problem indicates that it is a configuration in the IIS that needs to be changed."

So we´re waiting for the reply to this, because if we try to upload a file without passing through the WAF, and we did´t have this problem.

I´ll let you know

I'm glad to read abut your progress. I'd be curious if you're working with Arjun.. He's been most helpful.

Yes, IIS uploadReadAhaeadSize will definitely cause this issue, Actually that's the first thing we changed before I dove into the rabbit hole chasing this configuration directive override.

If the WAF is blocking the transaction you can easily monitor it by watching the /log/reverseproxy.log during the post to the IIS server.

That way you can see if it's the WAF or IIS dropping your file transfer at 30mb.

tail -f reverseproxy.log | grep -i "<Testing PC IP Addy>"

There is the possibility that another scan is killing the transfer for some other reason like a false positive on a Restricted SQL Character Anomaly, or some other pattern match.

...then again you wouldn't get a 413..

Happy hunting. Just keep in mind that if you find that you need to add another rule ID to the skip rules list, you'll also need to reapply the value change to the db and rerun the opcode to apply the change again to your system.

...at least until they get a fix rolled out for this bug.

I'm glad to read abut your progress. I'd be curious if you're working with Arjun.. He's been most helpful.

Yes, IIS uploadReadAhaeadSize will definitely cause this issue, Actually that's the first thing we changed before I dove into the rabbit hole chasing this configuration directive override.

If the WAF is blocking the transaction you can easily monitor it by watching the /log/reverseproxy.log during the post to the IIS server.

That way you can see if it's the WAF or IIS dropping your file transfer at 30mb.

tail -f reverseproxy.log | grep -i "<Testing PC IP Addy>"

There is the possibility that another scan is killing the transfer for some other reason like a false positive on a Restricted SQL Character Anomaly, or some other pattern match.

...then again you wouldn't get a 413..

Happy hunting. Just keep in mind that if you find that you need to add another rule ID to the skip rules list, you'll also need to reapply the value change to the db and rerun the opcode to apply the change again to your system.

...at least until they get a fix rolled out for this bug.

I just ran into the same problem, I will apply the commands, thanks all for your effort and sharing that information. But did any of know if this bug will be fixed? and if so, in which release?

Thanks in advance

I just wanted to comment that this is still a ongoing issue and the fix listed here does not seem to be working for me.  We have a standard ActiveSync rule setup and it uses the "Exchange General" protection policy.  After upgrading from 17.5 MR 12 to 18 we no longer could receive attachments on emails from all our mobile clients.  After finding a similar thread (https://community.sophos.com/products/xg-firewall/f/email-protection/121372/exchange-server-via-activesync) and this thread I disabled the "Common threats filter" and email started working.  Turn it back on and it blocks them.  The log files show its because of file size but it doesn't make any sense since there is no "size" constraint under the common threats filter settings. The log file clearly shows its a size limit issue:

[Wed Jul 22 10:01:48.054627 2020] [security2:error] [pid 14626:tid 140011471025920] [client 173.230.141.208:4131] [client 173.230.141.208] ModSecurity: Request body no files data length is larger than the configured limit (1048576).. Deny with code (413) [hostname "our.domain.com"] [uri "/Microsoft-Server-ActiveSync"] [unique_id "XxhGy38AAAEAADkiHDcAAABK"]

Been going back and forth with support about it with no solution yet.  I have both the Exchange General and Exchange Outlook Anywhere policies set up to 30Mb with no change.  Turn off "Common threat filter" and it works fine but that's not the answer.  Also worked fine before the upgrade to 18.