This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

WAF issues after updating to 9.709-3

Hi,

anyone else noticed that after updating to 9.709-3 Exchange Web Services is not working anymore? We get HTTP Error 500 when connecting to EWS published trhrough WAF. Also, the virtual server changes to orange when this error occurs. Accessing EWS through the browser shows the service page after authentication, but when interacting with EWS by using the Exchange Remote Connectivity Analyzer or EWS Editor generates the HTTP 500 error and the WAF rule turns orange.

When directly connecting to EWS and bypassing UTM works fine and we can interact with EWS.

Before the update everything worked fine.

Franc.



This thread was automatically locked due to age.
Parents
  • Sophos support came around and they opened a second case. Last week they implemented a patch for the issue and everything is working again now for us. I don't know the specifics of the fix. They created a development reference number for it: NUTM-13540

    Franc.

Reply
  • Sophos support came around and they opened a second case. Last week they implemented a patch for the issue and everything is working again now for us. I don't know the specifics of the fix. They created a development reference number for it: NUTM-13540

    Franc.

Children
  • Great news! Thanks for the update....It´s been a while.

    Hopefully they will release the fix soon.

  • We have published the workaround in this KBA: Sophos UTM: WAF Not Working - [proxy_http:error] read less bytes of request body than expected

    Our team is also actively working in integrating this fix in a future release. 

    Best,

    Karlos
    Community Support Engineer | Sophos Technical Support

    Knowledge Base  |  @SophosSupport  | Sign up for SMS Alerts
    If a post solves your question use the 'This helped me' link.
  • Thanks!

    Are there any (negative) side effects after changing this option in Apache?

  • @FrancWest The KB article mentions newer apache version not compatible with non-standard protocols used newer MS products. Is more specific information available?   What nonstandard protocols?  Thank you.

  • I just wanted to chime in that I've added this along with a fix for slash encoding (described here: https://community.sophos.com/utm-firewall/f/general-discussion/92242/utm-9-reverse-proxy-configuration/436406#pi2353filter=answers&pi2353scroll=false)

    Ultimately this is (part) of my current reverseproxy script with all the custom modifications:

    log() {
        ${APACHE2CTL_LOG}
    }
    
    apache_allowslashes() {
     grep -q 'PATH "/AllowEncodedSlashes/"' ${CHROOT}/usr/apache/conf/reverseproxy.conf
     MYRESULT=$?
    
     if [ ${MYRESULT} -eq 0 ]; then
     echo "AllowEncodedSlashes found - fixing" | log
     sed -i 's|WAFExceptions PATH "/AllowEncodedSlashes/" SkipAntiVirus|AllowEncodedSlashes On|g' ${CHROOT}/usr/apache/conf/reverseproxy.conf
     else
     echo "AllowEncodedSlashes not found - skipping" | log
     fi
    }
    
    apache_disable100continue() {
        grep -q 'PATH "/Disable100Continue/"' ${CHROOT}/usr/apache/conf/reverseproxy.conf
        MYRESULT=$?
    
        if [ ${MYRESULT} -eq 0 ]; then
          echo "Disable100Continue found - fixing" | log
          sed -i 's|WAFExceptions PATH "/Disable100Continue/" SkipAntiVirus|Proxy100Continue Off|g' ${CHROOT}/usr/apache/conf/reverseproxy.conf
        else
          echo "Disable100Continue not found - skipping" | log
        fi
    }
    
    apache_nocanon() {
     grep -q '/ProxyPassNocanon/' ${CHROOT}/usr/apache/conf/reverseproxy.conf
     MYRESULT=$?
    
     if [ ${MYRESULT} -eq 0 ]; then
     echo "ProxyPassNocanon found - fixing" | log
     sed -i 's|/ProxyPassNocanon/" lbmethod|/" nocanon lbmethod|g' ${CHROOT}/usr/apache/conf/reverseproxy.conf
     sed -i 's|/ProxyPassNocanon/|/|g' ${CHROOT}/usr/apache/conf/reverseproxy.conf
     else
     echo "ProxyPassNocanon not found - skipping" | log
     fi
    }

    This permits changes to the Apache config on a per-webserver-basis and lets me use encoded slashes and disable the "100 continue" (whatever it actually is) for our on-prem DevOps server.

    Any chance you could permit some other settings for selected web servers in an official capacity (so custom scripts don't have to be used)? Like the "AllowEncodedSlashes On" setting showcased above?

  • I don't know. I didn't write the KB article ;-)

  • This option is only recommended for use with Exchange. It may cause performance issues if WAF is also used to protect other sites, especially if they are used to download/upload large files.

    Karlos
    Community Support Engineer | Sophos Technical Support

    Knowledge Base  |  @SophosSupport  | Sign up for SMS Alerts
    If a post solves your question use the 'This helped me' link.
  • You may submit a request for this via ideas.sophos.com

    Karlos
    Community Support Engineer | Sophos Technical Support

    Knowledge Base  |  @SophosSupport  | Sign up for SMS Alerts
    If a post solves your question use the 'This helped me' link.
  • It's related to the handling of the '100 Continue' message in the HTTP protocol. This message is sent by a Server after receiving the headers for a request to indicate that it is ready to receive the body of the request from the client - it allows a server to check the headers and potentially reject the request before the client unecessarily sends all the data. Normally, the client would send an additional "Expect:" header to indicate that it is going to wait for the server to send this message before it sends the request body.

    The recent update to Apache changed how this process is handled by the proxy, in a way that didn't work well with Exchange.

    The old behaviour was that the proxy would itself respond to the client with a '100 Continue' and would not wait for the server to do so. The new behaviour of Apache forwards the "Expect:" header to the server and waits for the server to respond with 100 Continue before passing the "100 Continue" to the client. This change defends against potential issues where a client could send a very large request, which the WAF proxy would have to buffer until the server is ready for it.

    The change should be fine for servers that respond to 'Expect' headers and use 100 Continue strictly according to the HTTP specification, but it seems that Exchange does not.

    Changing the configuration option as specified reverts this behaviour so that the proxy itself responds with a "100 Continue" message instead of waiting for the Server to do so. The risk of this behaviour is that the proxy has to buffer the entire body of the request before the server is ready to receive it. This shouldn't be a problem in most situations because request bodies are usually quite small.

  • Hi
    We too are having this issues with EWS - especially when trying to migrate a mailbox from Exchange on-prem to Exchange online.

    I think I'm going to try out the workaround.

    You said that this "reverts this behaviour". Do you mean that it should work like before the update to 9.709? In other words, if we had no performance issues before the update we should not have any with this workaround?

    Is there a way to activate the "Proxy100Continue Off" option only for specific virtual webservers?

    Thanks!