This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Http proxy not running - restarted hourly

Any suggestions?

 

Version 9.603-1

home license

getting Info - 141 messages approximately hourly

 

web proxy live log is a continuous repeat of:

 

2019:07:06-09:41:25 another httpproxy[31645]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="286" message="reading configuration"
2019:07:06-09:41:25 another httpproxy[31645]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="312" message="caching templates"
2019:07:06-09:41:25 another httpproxy[31645]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="315" message="reading profiles"
2019:07:06-09:41:26 another httpproxy[31645]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_read_keys" file="ssl.c" line="444" message="failed to read public key"
2019:07:06-09:41:26 another httpproxy[31645]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_init" file="ssl.c" line="761" message="failed to setup contexts"
2019:07:06-09:41:56 another httpproxy[31803]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="286" message="reading configuration"

 

 
Proxy logs looked normal until
2019:07:02-12:50:27 another httpproxy[20183]: id="0002" severity="info" sys="SecureWeb" sub="http" name="web request blocked" action="block" method="GET" srcip="192.168.10.127" dstip="0.0.0.0" user="" group="" ad_domain="" statuscode="502" cached="0" profile="REF_HttProContaInterNetwo (Youview)" filteraction="REF_HttCffYouview2 (Youview)" size="2910" request="0x998b8e00" url="aax-eu.amazon-adsystem.com/.../update_dev_info referer="" error="Connection refused" authtime="0" dnstime="651" aptptime="1590" cattime="394" avscantime="0" fullreqtime="6994" device="0" auth="0" ua="Dalvik/2.1.0 (Linux; U; Android 5.1.1; AFTT Build/LVY48F)" exceptions="" overridecategory="1" category="154" reputation="neutral" categoryname="Web Ads" country="N/A" country="N/A" country="N/A" application="amazon" app-id="22"
2019:07:02-13:26:00 another URID[19461]: T=19461 ------ 2   - Warning: EARLY TIMEOUT: dns context 3 has 5937 ms before it should time out\n
2019:07:02-13:26:36 another URID[19461]: T=19461 ------ 1 - [exit] SIGTERM: exiting
2019:07:02-13:26:37 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="saviscanner_log" file="saviscanner.c" line="496" message="Reloading SAVI configuration"
2019:07:02-13:26:37 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="saviscanner_log" file="saviscanner.c" line="496" message="Reloading configuration file finished, config version: 6"
2019:07:02-13:26:38 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="saviscanner_log" file="saviscanner.c" line="496" message="SAVI reload done, reducing the number of SAVI instances from 7 to 0 (usage stats: min 5/avg 2787/max 8109)"
2019:07:02-13:27:19 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="confd_config_reload_func" file="confd-client.c" line="594" message="reloading config"
2019:07:02-13:27:38 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="epoll_loop" file="epoll.c" line="861" message="starting exit cleanup"
2019:07:02-13:27:40 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_read_keys" file="ssl.c" line="444" message="failed to read public key"
2019:07:02-13:27:40 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="confd_config_reload_func" file="confd-client.c" line="650" message="re-setup ssl contexts failed"
2019:07:02-13:27:40 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="confd_config_reload_func" file="confd-client.c" line="658" message="reloading config done, new version 24962"
2019:07:02-13:27:50 another httpproxy[20183]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="scan_exit" file="scanner.c" line="955" message="scanner subsystem shutting down"
2019:07:02-13:27:50 another httpproxy[20183]: [tid 3058038816]: [aptp_disconnect]: aptp socket has been disconnected
2019:07:02-13:27:50 another httpproxy[20183]: [tid 3058038816]: [aptp_exit] APTP cleanup done
2019:07:02-13:28:13 another httpproxy[3651]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="286" message="reading configuration"
2019:07:02-13:28:15 another httpproxy[3651]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="312" message="caching templates"
2019:07:02-13:28:15 another httpproxy[3651]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="315" message="reading profiles"
2019:07:02-13:28:16 another httpproxy[3651]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_read_keys" file="ssl.c" line="444" message="failed to read public key"
2019:07:02-13:28:16 another httpproxy[3651]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_init" file="ssl.c" line="761" message="failed to setup contexts"
2019:07:02-13:28:46 another httpproxy[3953]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="286" message="reading configuration"
2019:07:02-13:28:47 another httpproxy[3953]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="312" message="caching templates"
2019:07:02-13:28:47 another httpproxy[3953]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="315" message="reading profiles"
2019:07:02-13:28:48 another httpproxy[3953]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_read_keys" file="ssl.c" line="444" message="failed to read public key"
2019:07:02-13:28:48 another httpproxy[3953]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_init" file="ssl.c" line="761" message="failed to setup contexts"
2019:07:02-13:29:04 another httpproxy[4088]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="286" message="reading configuration"
2019:07:02-13:29:05 another httpproxy[4088]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="312" message="caching templates"
2019:07:02-13:29:05 another httpproxy[4088]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="315" message="reading profiles"
2019:07:02-13:29:06 another httpproxy[4088]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_read_keys" file="ssl.c" line="444" message="failed to read public key"
2019:07:02-13:29:06 another httpproxy[4088]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_init" file="ssl.c" line="761" message="failed to setup contexts"
 
 
 


This thread was automatically locked due to age.
  • Hello Moto (heh),

    This sounds like a proxy reset normally associated with disk IO, what hardware are you running?

    Also, I'd recommend following this KB to track the performance statistics when the proxy fails:

    Emile

  • looking at the selfmonitoring log I am seeing continuous:

    2019:07:06-11:56:15 another selfmonng[3899]: I check Failed increment httpproxy_running counter 1 - 3
    2019:07:06-11:56:20 another selfmonng[3899]: I check Failed increment httpproxy_running counter 2 - 3
    2019:07:06-11:56:25 another selfmonng[3899]: W check Failed increment httpproxy_running counter 3 - 3
    2019:07:06-11:56:25 another selfmonng[3899]: W NOTIFYEVENT Name=httpproxy_running Level=INFO Id=141 suppressed
    2019:07:06-11:56:25 another selfmonng[3899]: W triggerAction: 'cmd'
    2019:07:06-11:56:25 another selfmonng[3899]: W actionCmd(-): '/var/mdw/scripts/httpproxy restart'
     
    So it looks like it never runs successfully.
     
    As for the hardware I would have to open it up for details but I assume a dual core pentium on an Intel motherboard with 3 GB of memory and a 500GB disk.
     
    CPU never rises above 10%, memory at 54%. 2 users across 10 computers.
     
    On the day that this problem started my ISP acknowledged it had problems and I power cycled my cable modem a number of times. At the time that the problem
    started I had messages that middleware, http proxy, smtpd, application control, dhcp client and IPV6 hurricane tunnel broker client were restarted.
  • I think the key is this error message:

     

    2019:07:02-13:28:16 another httpproxy[3651]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="ssl_read_keys" file="ssl.c" line="444" message="failed to read public key"

     

    The proxy is failing to start because it cannot get what it needs to do something with certificates.

    Have you installed your own certificate for WebAdmin or changed the signing CA?  

     

  • Hi  

    As Michael said, have you changed the WebProxy CA or added a new one of your own? If you did, reverse the changes and restart the HTTPProxy manually.

    Regards

    Jaydeep

  • No, I haven't changed the Webproxy CA. I'll look at regenerating it.

  • DOH!

     

    looks like progress

    sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="286" message="reading configuration"
    2019:07:13-10:12:37 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="312" message="caching templates"
    2019:07:13-10:12:37 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="315" message="reading profiles"
    2019:07:13-10:12:38 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="disk_cache_read" file="diskcache.c" line="1766" message="calculated md5sum: [D41D8CD98F00B204E9800998ECF8427E], file md5sum: [D41D8CD98F00B204E9800998ECF8427E]"
    2019:07:13-10:12:38 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="scanner_init" file="aptpscanner.c" line="167" message="oculusd port = 25000 ; initializing ATP..."
    2019:07:13-10:12:38 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="scanner_init" file="aptpscanner.c" line="172" message="ATP loaded"
    2019:07:13-10:12:51 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="scanner_init" file="saviscanner.c" line="517" message="Successfully loaded SAVI threat data, engine 3.76.0, threat data 5.62 from 9/4/2019 (33419268 detected threats)"
    2019:07:13-10:12:51 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="libnavl_log" file="navl_externals_posix.c" line="43" message="E: InitInstance: Error initializing instance of plugin HPACK_UTIL"
    2019:07:13-10:12:51 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="356" message="notifiying argos daemon"
    2019:07:13-10:12:51 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="argos_notify" file="httpproxy.c" line="210" message="connect: Connection refused"
    2019:07:13-10:12:51 another httpproxy[11313]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="main" file="httpproxy.c" line="362" message="finished startup"
    2019:07:13-10:12:51 another httpproxy[11313]: Integrated HTTP-Proxy (c) 2007-2016 Sophos Ltd, Release 356.g39864e02.rb4
    2019:07:13-10:12:58 another httpproxy[11313]: [tid 2868378480]: [aptp_connect]: aptp socket connect succeeded