This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

UTM Reverse Proxy still tries to connects to an old AWS ELB IP

Situation is UTM is on AWS. Using WAF where in real ws is pointing to Amazon ELB.

Virtual WebServer shows up an error from time to time.

Noticed that it's happening whenever Amazon changes their IP address but not sure why would it matter because UTM Real WebServer is pointing to a DNS Host of the ELB.

Only way to resolved it was to disable and enable the real webserver. 

ReverseProxy.log

Still tries to connect to x.x.x.100, that ip doesnt exist anymore

2015:11:13-21:20:43 ort_utm2 reverseproxy: [Fri Nov 13 21:20:43.113375 2015] [proxy:error] [pid 17116:tid 3971750768] (110)Connection timed out: AH00957: HTTPS: attempt to connect to x.x.x.100:443 (xxxxx.us-west-2.elb.amazonaws.com) failed

ELB IP address is now x.x.x.25.

Any thoughts?

Thanks in advance!



This thread was automatically locked due to age.
Parents
  • I noticed your post was last year - still using Sophos or have you moved on to a different product?

  • We have the same issue.  Did you have "disable connection pooling" checked on your Real WebServer?  Unfortunately I have no way to get an ELB to change its IP addresses so no way to verify if this issue is fixed or not.  

  • I didn't have it checked - see screenshot below.

    In my case, the AWS ELB scaling event occurred for no apparent reason which was a blessing in disguise. You will need to put load on your ELB in order for it to execute a scale up / down event. Here is guidance from AWS: https://aws.amazon.com/articles/1636185810492479 I would use jMeter to perform various tests to initiate a scaling event. 

    Another way to have a scaling event to take place without doing load testing is to create a new ELB. When ELBs are created they are not created with the smallest instance size - therefore if there is no or low load on that ELB it will scale down to the smallest instance size which generated an IP change.

    I have been communicating with Sophos Support on this and as of now, there is no fix. 

    In the meantime, I am trying a crontab to restart the named service every 15 minutes. You'll need to SSH into your appliance to do this.

  • You have to enable "Disable backend connection pooling" for the real webserver if you're using ELB. Unfortunately the KB article doesn't mention this.

  • I have confirmed that this works however, there seems to be a knowledge gap about this with Sophos Support. I directed them to this article but the reply that I got was that I can try disabling backend connection pooling but Sophos is working on a "fix".

    Anyway for those that aren't sure when your AWS ELB is having ENI changes (i.e. creates and deletes) you can use this CloudWatch metric filter to see the events:

    { $.userIdentity.type = "Root" && $.userIdentity.invokedBy = "elasticloadbalancing.amazonaws.com" && $.eventName = "CreateNetworkInterface" || $.eventName = "DeleteNetworkInterface"}


    To verify that enabling the "Disable backend connection pooling" setting works, I used the above CloudWatch filter to create an Alarm and then used JMeter to increase the load on my ELB. I observed the AWS ELB create ENI (i.e. new ELB IPs) event and then approximately 1.5 hours later an AWS ELB delete ENI (i.e. old ELB IPs) events in which previously caused the Connection timed out errors.

Reply
  • I have confirmed that this works however, there seems to be a knowledge gap about this with Sophos Support. I directed them to this article but the reply that I got was that I can try disabling backend connection pooling but Sophos is working on a "fix".

    Anyway for those that aren't sure when your AWS ELB is having ENI changes (i.e. creates and deletes) you can use this CloudWatch metric filter to see the events:

    { $.userIdentity.type = "Root" && $.userIdentity.invokedBy = "elasticloadbalancing.amazonaws.com" && $.eventName = "CreateNetworkInterface" || $.eventName = "DeleteNetworkInterface"}


    To verify that enabling the "Disable backend connection pooling" setting works, I used the above CloudWatch filter to create an Alarm and then used JMeter to increase the load on my ELB. I observed the AWS ELB create ENI (i.e. new ELB IPs) event and then approximately 1.5 hours later an AWS ELB delete ENI (i.e. old ELB IPs) events in which previously caused the Connection timed out errors.

Children
No Data