This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

After Update to 9.410-6 I get core dumps of cssd

Hi,

installed yesterday 9.410-6 Firmware on my Home Firewall Zotac Zbox CI323

Since midnight I got every hour a core dump of cssd, filling up my data disk.

I deleted the dumps and restarted, it seems to be OK now.

Could it be that it was a matter of malformed Up2date files?


UPDATE:
cssd is still crashing from time to time.

UPDATE2:
Seems there were some big AV-malware-names-* files left over in /tmp which prevented to install newer version.
I cleaned up /tmp and wait if its stable again.

UPDATE3:
After a couple of days it happens again. cssd has a bug. I use now the workaround: Disable Dual Scan for SMTP, and use only Avira-Engine.

BR
Mundl



This thread was automatically locked due to age.
Parents
  • I have the same Problem, i use the UTM VMWare Version.

    Please fix the bug asap.

  • Hi,

    it seems the full /tmp directory was the cause.

    Somehow some AV-malware-names-* files (each ~230 MB) were left over after upgrade. After deleting them and cleaning up some other old files in /tmp it works again.

    My /tmp partition is rather small, only 835MB. My SSD is much bigger (32 GB), I wonder why the installer created such a small /tmp partition, and does not monitor it...

  • /tmp is ok, i have 70% /var/storage before the update 55%

    What can i do?

  • Check if there are any crash dumps in /var/storage/cores.

    Delete them.

  • Mundl, I think the minimum recommended is 40GB, even for a home installation.  I agree that you should not have had this problem though.  I have a test unit setup on a 20GB drive, and it had no problem with the 9.410 Up2Date.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • Are these all dumps only:

    srv-utm:/var/storage/cores # ll
    total 3169440
    -rw-r--r-- 1 root root  20647936 Nov 26 12:05 admin-reporter..15124
    -rw-r--r-- 1 root root 511131648 Feb  3 12:48 cssd.5364
    -rw-r--r-- 1 root root 512008192 Feb  3 12:51 cssd.7468
    -rw-r--r-- 1 root root 510996480 Feb  3 12:54 cssd.7880
    -rw-r--r-- 1 root root 511229952 Feb  3 12:57 cssd.8541
    -rw-r--r-- 1 root root 511221760 Feb  3 12:45 cssd.9491
    -rw-r--r-- 1 root root 655581184 Dec 10  2015 httpproxy.Confd.22712
    -rw-r--r-- 1 root root  12664832 Jan 30  2016 ips-reporter.pl.11755
    srv-utm:/var/storage/cores #

    Expert-Zone.Net IT Consulting
    Neuenhofer Weg 23 • D-52074 Aachen

  • Yes, those are all dumps.  Wow, cssd was really choking on something!

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • Hey, Sophos, are you sleeping?

    I have about 20 or 30 Sophos UTM installations and with really all systems updated to 9.410-6, cssd is crashing :( Unfortunately not all customers are asking me to release the update.

    Cssd ist crashing inside of SMTP wirh some incoming emails, but of corse if you get some of that emails they are retried every few minutes and cssd is crashing again and again. The second virusscan engine (avira) is filling up /tmp then with AV* files and all cssd depending Services (HTTP/S, WAF) are denying their forward until cssd runs again, for example Exchange Outlook Web App users are kicked out of their WAF sessions.

    Sophos, please fix your UTM 9.410-6 cssd problem soon !!!

    The only workaround is to disable SMTP malware scan at all by SMTP malware exception to any. After that, /tmp is to be freed from AV* files.

    I'm very disapointed about this big quality misstake.

    Regards,

    Manfred

  • The lab PC-based UTM and AWS instance were upgraded to 9.410.  Neither has exhibited this behavior.  Both were on single-scan Avira at the time of the Up2Date.  I changed one to Sophos and have seen no issue with it, either.  I wonder if the problem occurs only when Up2Dating a unit running the Sophos antivirus.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • Hi Bob, I didn't have any problems with updates while running avira AV either. I use avira at parameter due to better detection rates and I had just converted my lab back to UTM9 from XG, so I wasn't brave enough to try sophos AV. However this brings up a bigger problem with UTM9 in general and Sophos in particular. The last few updates have been pretty bad consistently. I always try to stay a version or two behind and let other people be the guinea pigs but when they have CVE vulnerability patches, I usually update pretty quickly.

    9.409 update was botched and they had to issue an update and now 9.410 is not only broken, but is effectively rendering their sandstorm protection useless. Similar thing happened to XG over the holidays and an IPS pattern update broke ALL categorization for everyone https://community.sophos.com/kb/en-us/125754 and took almost 3 days to issue an update. 

    I remember back in astaro days, we had ONE bad rollout and everyone including top management came and apologized in open astaro forum and promised that was not going to happen again. However, sophos seems to be taking their resellers and users base for granted and keep dropping updates without any QA. You would think that as a leader in magic quadrant they would be more worried about the image and reputation of their flagship products, but their track record is seriously indicating otherwise.

Reply
  • Hi Bob, I didn't have any problems with updates while running avira AV either. I use avira at parameter due to better detection rates and I had just converted my lab back to UTM9 from XG, so I wasn't brave enough to try sophos AV. However this brings up a bigger problem with UTM9 in general and Sophos in particular. The last few updates have been pretty bad consistently. I always try to stay a version or two behind and let other people be the guinea pigs but when they have CVE vulnerability patches, I usually update pretty quickly.

    9.409 update was botched and they had to issue an update and now 9.410 is not only broken, but is effectively rendering their sandstorm protection useless. Similar thing happened to XG over the holidays and an IPS pattern update broke ALL categorization for everyone https://community.sophos.com/kb/en-us/125754 and took almost 3 days to issue an update. 

    I remember back in astaro days, we had ONE bad rollout and everyone including top management came and apologized in open astaro forum and promised that was not going to happen again. However, sophos seems to be taking their resellers and users base for granted and keep dropping updates without any QA. You would think that as a leader in magic quadrant they would be more worried about the image and reputation of their flagship products, but their track record is seriously indicating otherwise.

Children
  • Hello,

    the problem is not while the update and it deosn't matter which av runs while the update. The Problem is, that after the update cssd crashes all the time while scanning emails and then avira av fills up tmp if avira is used in any policy.

    ep-cssd-9.40-27.gf72484e.rb3.i686.rpm is buggy and crashes all time !!!!!

    Sorry for my bad english, if you didn't understand.

    Regards, Manfred

  • Hi, Manfred, and welcome to the UTM Community!

    Thanks for your clear explanation of the problem.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • Hello Bob,

    more clear: not all emails are affected, some special emails let the cssd crash. In my case for example, alert emails from customer Veeam Backup systems (HTTP Format without attachments) let my Sophos UTM cssd crash. And because the customer Sophos UTM tries to send these emails again and again, my cssd crashes every 3 minutes. And, all other areas using cssd too (Web Protection, Web Server Protection), stop working and reset user sessions while cssd is crashing.

    I can't understand, why Sophos do not withdraw this update. And i hope, a cssd fix is coming soon. I can not imagine, that Sophos do not have a lot of support cases about this problem. 5 days now and no solution from Sophos :-(

    Regards, Manfred