Turning on Web Protection destroys PC's

Hi,

If you experienced any issues please contact Technical Support.

Support@sophos.com

Please send in an SDU of a machine affected.

http://www.sophos.com/support/knowledgebase/article/33533.html

Thanks

Jon

Hi Scott2020,

Sorry to hear you're having problems. As explained by my colleague Jon above, the most effective approach here would be to talk to Support direct. This is simply to get you the fastest possible response, so we can get your systems back up and working to full effectiveness. We would like to document the outcome here in this thread, but safeguarding your environment comes first. If at all possible, we'll put in another reply to your post that documents how to deal with this situation, for any other SophosTalk reader who sees the same thing. Otherwise, get in first, and give us your impressions of the outcome.

Cheers,

spike

Hi,

What does the output of:

netsh winsock show catalog > out.txt

show?

I see that Sophos Web Protection uses a LSP to do its work, maybe there is a conflict with another on the machine.

I assume you can start the machines in safe mode with command prompt, i.e. without networking and from there you can perhaps rename C:\ProgramData\Sophos Web Intelligence\swi_lsp.dll (or similar on yout platform) which might allow these machines to boot in order to find any other applications which have installed a LSP?  Hopefully you can find a conflict which would explain the problem.

Thanks,

Jak

Hello all,

I did open a ticket with Sophos yesterday.  Initial response was fast, and they advised me to download the boot CD and do a scan of my computer.  The scan showed no infections.  I highly doubt my machine was infected.  I was running 9.5 fine after upgrading, until I turned on the website scanning option.  I have 3 machines that were brought down when I enabled this option.  One of them I was able to boot with last known good configuration and saved it.  The others are still down.  I haven't heard from Sophos since yesterday morning.

Just curious if there was a resolution.  I am a new customer and 3 out of 15 new installs have resulted in 3 system crashes. 1 completely lost rebuild

1 sysinternals allowed a recovery

a third we are trying to trouble shoot with Support.  We had the Web stuff set to log, I had a feeling this might still be an issue.

This is a major deal since we are trying to get a global deployment going.

We are able to keep the machines up by issuing and shutdown /a  

Most services need to keep the machine operation have failed including the services needed to uninstall the Sophos app

No, not yet but it is somewhat my fault.  I haven't had time to work on their troubleshooting steps yet.  Part of the problem is that I can't even get into Windows to run the things they wanted me to run.  First support made me run a scan using their boot CD because they thought the machines were already infected, but they were not.  I was able to run version 9.5 without any problems, and still have about 100 machines running 9.5 and they are working great.  It is just turning on the web protection that started taking them out.  I feel your pain, but I'm glad it isn't just me!

Hey can you post what they instructed you to try.  We had the same issues with the app crashing then rebooting -- as soon as we got in we opened the cmd prompt and every time that windwo popped up with type in the shutdwon /a to stop the reboot.

That is good to hear about the 100 xp machines with out issues?

I will post the steps they had us try but did not work as of yet.

My name is David and I am the escalation engineer that will be working with you on this issue. I emailed Ron earlier to inform him that I was reviewing the data. From the data I see a number of problems, and these could be compounded due to the system stability issue you are seeing. The blog post you were refereeing too does highlight one of the issues.

The hips system has a detour dll that we created via Microsoft standards, any number of vendors also use and create their own detour dll's.

In this case I see the following:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\AppInit_DLLs
C:\PROGRA~1\Google\GOOGLE~1\GOEC62~1.DLL,C:\PROGRA~1\Sophos\SOPHOS~1\SOPHOS~1.DLL C:\PROGRA~1\Google\GOOGLE~1\GOEC62~1.DLL

normally this is not an issue at all, but in this case there is a double entry for Google and no comma between our entry and one of theirs. this can cause issues as the file is not listed correctly, as it needs a comma between the entries. Further to that the double entry is the same so its not needed anyway. so deleting the extra portion at the end of the entry "C:\PROGRA~1\Google\GOOGLE~1\GOEC62~1.DLL " and rebooting may resolve the issue. Now it is possible that we are conflicting with Google’’’’s detour dll, I haven't seen this in the past with their products but it is possible. To resolve the issue we can change the load order of the detours which in many cases can resolve the issue or remove our entry which also can resolve the issue.

I would recommend the following actions:
1) Open regedit and navigate to the following location: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\
2) Select the Windows hive and select export. This will make a back up so that the registry can be returned to its original state (this is a precaution)
3) Select the AppInit_DLLs key and select modify. remove the following data at the end of the key "C:\PROGRA~1\Google\GOOGLE~1\GOEC62~1.DLL "
4) reboot the system did this resolve the issue with system stability?
5) If not open regedit and navigate to the following location: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\
6) Select the AppInit_DLLs key and select modify. Delete the data on the key and replace it with the following data:
C:\PROGRA~1\Sophos\SOPHOS~1\SOPHOS~1.DLL, C:\PROGRA~1\Google\GOOGLE~1\GOEC62~1.DLL
7) reboot the system did this resolve the issue with system stability?
5) If not open regedit and navigate to the following location: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\
6) Select the AppInit_DLLs key and select modify. Delete the data on the key and replace it with the following data:
C:\PROGRA~1\Google\GOOGLE~1\GOEC62~1.DLL
7) reboot the system did this resolve the issue with system stability?
If the last step has resolved the issue please let me know as there are steps that can be taken on other systems to prevent the issue.

The next issue I notes dealt with updating it appears that there is an issue accessing files in our temp locations which is normally due to the indexing service holding the files and not allowing us to access them. This is possibly due to the systems state. If this is not the case and after resolving the first issue you continue to see an update issue please perfrom the following:
1) Stop the indexing service.
2) Right click the shield and select update now.
3) Once the update has completed you can turn the indexing service back on.

Please let me know if you have any questions.

So for now we were able to recover the machine with Supports help.  We had to shut off the new Web filtering feature.  We manually did on the crashing machine and all is good.  We Modfied our policy to turn off for all machines until they address the issue

Hi everyone,

Just wanted to add some notes here for benefit of others facing similar issues. The new Live URL Filtering feature ("Web Protection") in 9.5 uses a Layered Service Provider (LSP) to provide protection while browsing the web. Generally this has proven to be quite reliable and useful, but there are some exceptions. Unfortunately we didn't catch all of these issues during internal testing and beta testing, and in this specific case the results have been ugly. Apologies for that.

There are at present two different third-party software packages (using LSPs) which are known to cause this type of issue:

1. NVIDIA App Filter (installed by default by some PC vendors)

2. APC InfrastruXure Client version 4.7 (discontinued, but still quite popular for managing APC products)

The issue with the NVIDIA software will be addressed in the 9.5.2 update.

The issue with the APC software has unfortunately only been discovered in the field after release, and the engineering team are investigating with the intention of addressing this as soon as possible.

As with all issues related to Sophos' software, please contact our Support team when you run into difficulties. In particular to compatibility issues like this thread highlights, they can help get systems back into working order rapidly. They will also escalate to the engineering team so the underlying software issues can be addressed.

For clarity, the new Web Protection feature is unrelated to the Detours and BHO feature. Disabling those features is not a recommended course of action when dealing with incompatibilities like this one.

Thanks,

Bob Cook

Development Manager, Sophos

(I'm the manager for the engineering team that developed the Web Protection technology)