This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

[9.310-11]Web Proxy Freezing

Hi folks,

Weird issue that has just started today. After awhille the proxy just seems to stop working than start again. According to the logs it is not restarting and top is showing activity. When the problem is starting to happen cpu usage spikes to 100% across all cores (httpproxy)

Below are the live logs as it starts to happen.


2015:04:23-14:03:39 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="confd_network_filter" file="confd-client.c" line="1851" message="failed to get network: Operation not permitted (req=[{ "id": 3990, "method": "get_object", "params": [ "" ] }])"
2015:04:23-14:03:40 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="confd_network_filter" file="confd-client.c" line="1851" message="failed to get network: Operation not permitted (req=[{ "id": 4482, "method": "get_object", "params": [ "" ] }])"
2015:04:23-14:03:45 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="aptp_reload" file="aptpscanner.c" line="133" message="reloading ATP pattern"
2015:04:23-14:03:45 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="aptp_reload" file="aptpscanner.c" line="151" message="reloading ATP pattern finished"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="plain_write_vector" file="epoll.c" line="1098" message="Write error on the epoll handler 395 (Broken pipe)"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0001" severity="info" sys="SecureWeb" sub="http" request="0xd543c000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd543a800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="confd_config_reload_func" file="confd-client.c" line="634" message="reloading config done, new version 1017"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd543d000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0001" severity="info" sys="SecureWeb" sub="http" request="0xa500000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd542c000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0001" severity="info" sys="SecureWeb" sub="http" request="0xa500800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd542b800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa4fe000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd58b2800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd540c800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xe49d5800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd5af3800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xe33d7800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd5af3000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xe33d7000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xe33d6800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd5af2000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xe33d6000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd5af0800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xe33d5000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa75f800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa75f000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd4b76800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa75e000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0001" severity="info" sys="SecureWeb" sub="http" request="0xa75d800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xe580b800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa43c800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa75c800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa43c000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd56f7800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa43b800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd56f7000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xa43a000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd5d15800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd58b0800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xdd397800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd52fa000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xdd396800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xdd396000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xdd395800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xd4eab800" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"
2015:04:23-14:07:09 HOSTNAME_HERE httpproxy[22795]: id="0003" severity="info" sys="SecureWeb" sub="http" request="0xdd395000" function="p0f_add_request" file="p0f-client.c" line="345" message="connect: Resource temporarily unavailable"


Things tried:

  •  Stopping Proxy, Clearing Cache, Restarting
  •  Rebooting VM
  •  During the outage if i bypass the proxy by going straight to the gateway (our edge router on dmz) using the same dns servers, it all works.


System CPU usage is usually in the 10-40% range; memory is only 40% of 8gb.

Proxy is in standard mode w/ http/s filtering only + AV scanning.


This thread was automatically locked due to age.
Parents
  • p0f is used for device type (operating system) detection so that different types of authentication can be used.  It cannot be turned off.  But I think the log of p0f is a symptom not a cause.

    I agree this is something that should go to support.
  • p0f is used for device type (operating system) detection so that different types of authentication can be used.  It cannot be turned off.  But I think the log of p0f is a symptom not a cause.

    I agree this is something that should go to support.


    Thanks for the extra info, Michael.

    CTO, Convergent Information Security Solutions, LLC

    https://www.convergesecurity.com

    Sophos Platinum Partner

    --------------------------------------

    Advice given as posted on this forum does not construe a support relationship or other relationship with Convergent Information Security Solutions, LLC or its subsidiaries.  Use the advice given at your own risk.

  • Ticket raised. For now i've cloned a previous snapshot and rolled back.
  • Ticket raised. For now i've cloned a previous snapshot and rolled back.



    Have you gotten anywhere with this?  We're seeing the exact same problem.
Reply Children
No Data