Thread Info
  • State Not Answered
  • Locked Locked
  • Replies 10 replies
  • Subscribers 2 subscribers
  • Views 20933 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

HTTP pipelining broken after upgrade to UTM 9.3

ondrej.holas
In 9.2, the HTTP pipelining works well on HTTP proxy:

$ telnet 192.168.1.22 8080
Trying 192.168.1.22...
Connected to 192.168.1.22.
Escape character is '^]'.
GET 192.168.1.2/.../1.1
Host: 192.168.1.2

GET 192.168.1.2/.../1.1
Host: 192.168.1.2

HTTP/1.1 200 OK
Date: Fri, 26 Dec 2014 19:59:05 GMT
Server: Apache
Last-Modified: Fri, 26 Dec 2014 19:24:08 GMT
Accept-Ranges: bytes
Content-Length: 6
X-Frame-Options: deny
Keep-Alive: timeout=5, max=100
Content-Type: text/plain
Proxy-Connection: Keep-Alive

test1
HTTP/1.1 200 OK
Date: Fri, 26 Dec 2014 19:59:05 GMT
Server: Apache
Last-Modified: Fri, 26 Dec 2014 19:24:12 GMT
Accept-Ranges: bytes
Content-Length: 6
X-Frame-Options: deny
Keep-Alive: timeout=5, max=99
Content-Type: text/plain
Proxy-Connection: Keep-Alive

test2
^]
telnet> q
Connection closed.


In 9.3 (tested on 9.304-9 and 9.305-4), HTTP proxy processes only the first request and times out after one minute:

$ telnet 192.168.1.22 8080
Trying 192.168.1.22...
Connected to 192.168.1.22.
Escape character is '^]'.
GET 192.168.1.2/.../1.1
Host: 192.168.1.2

GET 192.168.1.2/.../1.1
Host: 192.168.1.2

HTTP/1.1 200 OK
Date: Fri, 26 Dec 2014 20:11:57 GMT
Server: Apache
Last-Modified: Fri, 26 Dec 2014 19:24:08 GMT
Accept-Ranges: bytes
Content-Length: 6
X-Frame-Options: deny
Keep-Alive: timeout=5, max=100
Content-Type: text/plain
Proxy-Connection: keep-alive

test1
Connection closed by foreign host.


The second request is neither processed nor logged in /var/log/http.log.

Real world scenario affected by this bug is Debian apt configured to pipeline requests (by default in squeeze).

Maybe something to do with changes in ID31116.

Brgds,

Ondrej


This thread was automatically locked due to age.
  • 0
    Something like this can only get to developers via Support.  Make sure to stress the real world reproduction.
  • 0 in reply to Michael Dunn
    Something like this can only get to developers via Support.  Make sure to stress the real world reproduction.


    Michael, since I am only home-licensed, I cannot open support case.

    Due to reliably reproducible behavior, every application using HTTP pipelining and UTM as a proxy should suffer from this bug. Example of real-world scenario can be the one where I've observed it first - Debian apt. I know this exact problem with apt and UTM was discussed here back in 2007 and the only workaround then was to disable HTTP pipelining on the client side. Later UTM versions have this problem fixed and up to 9.2 the HTTP pipelining works well.

    Brgds,

    Ondrej
  • 0
    usually with apt gets you setup an exception in http proxy so that gets bypassed as apt and yum and others tend to not like the http proxy interception at all.

    Owner:  Emmanuel Technology Consulting

    http://etc-md.com

    Former Sophos SG(Astaro) advocate/researcher/Silver Partner

    PfSense w/Suricata, ntopng, 

    Other addons to follow

  • 0 in reply to William Warren
    usually with apt gets you setup an exception in http proxy so that gets bypassed as apt and yum and others tend to not like the http proxy interception at all.


    Proxy interception is different scenario, although it could be affected as well (I haven't tested it yet). The bug I've described above relates to "explicit proxy" scenario, i.e. the client is configured to use proxy, connects to the proxy and sends full URLs in requests.

    Nevertheless, this bug is not about particular apt problems; principally all apps using HTTP pipelining are affected.

    Brgds,

    Ondrej
  • 0
    I'm not disputing your findings, Ondrej, but we now have run a beta for several months and have over ten thousand UTMs on 9.3 in production systems.  So far none of them have reported this problem, at least that has gotten to Dev.  That suggest that even if there is a difference in 9.2 and 9.3 the real world impact might not be noticeable.

    As a QA person, I think every defect in a product should get fixed.  The reality is that prioritization must go to defects that are affecting paid customers that they are reporting to us.
  • 0
    FWIW I am seeing this issue in two use cases:

    1. Unable to run apt-get update on Ubuntu boxes with web proxy enabled.
    2. Clients unable to install apps on iOS devices

    Hopefully some paying customers will enjoy the same issue. [:)]
  • 0 in reply to packetwerks
    FWIW I am seeing this issue in two use cases:

    1. Unable to run apt-get update on Ubuntu boxes with web proxy enabled.
    2. Clients unable to install apps on iOS devices

    Hopefully some paying customers will enjoy the same issue. [:)]


    apt-get not working is a long standing issue.  It is one solved by putting the proper exceptions into the http proxy area.  if it is a server i just put the server into the proxy bypass.  if it is a desktop you jsut make a regex that excepts the appropriate domain.  Some webserver d not like having anything intercepting them.

    The ios exceptions placed into the current 9.3x utm installs alleviate ios downloads(unless you have https inspection on.)  If you are having issues you need to put an https exception for ios into the proxy as apple can(and often does) properly detect the MITM functionality required for htps inspection and breaks.

    Owner:  Emmanuel Technology Consulting

    http://etc-md.com

    Former Sophos SG(Astaro) advocate/researcher/Silver Partner

    PfSense w/Suricata, ntopng, 

    Other addons to follow

  • 0 in reply to William Warren
    I am able to reproduce this bug in Firefox with OpenStreetMap and the following settings:

    about:config

    • network.http.max-persistent-connections-per-server 12
    • network.http.pipelining true
    • network.http.proxy.pipelining true

    Browse on OSM with MapQuest layer (Akamai CDN)

    The bug is triggered with HTTP pipelining enabled and having multiple simultaneous (at least 5) threads/connections.

    I would really appreciate it if someone could try the above steps to confirm.


    By the way, the .NET framework has pipelining enabled by default in the WebClient and WebRequest classes, so any program using those can be affected.
  • 0
    We've built a very simple .Net program that demonstrates Ondrej's problem on SG210 / 9.352. Downloading map tile files with .Net (WebClient), asynchronous download. With pipelining on, first 6 tiles come quickly, then a pause for exactly 60 seconds before the rest arrive. With pipelining disabled, map tiles flow freely. Is support looking at this issue?
  • 0 in reply to BobK1

    Hi, Bob, and we,come to the UTM Community.

    These forums aren't monitored by technical folks from Sophos except the forums for beta tests.   If you have a paid subscription, please get Sophos Support involved so that this issue reaches the developers.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
Unfiltered HTML