Disk fragmentation

Hello Matt,

didn't notice the behaviour you describe and therefore I've never checked what is copied and what not - but I'd say it's very unlikely that this is "normal" behaviour: I have a laptop which recently had free space as low as 150MB and Sophos was updating without problems. But maybe you are right and I just have been lucky.

Taking nothing for granted I checked the file creation dates (if they are a valid indicator of a fresh copy) in the Sophos Anti-Virus folder and - surprise! - found that 112 IDEs (from adloa-lu.ide to bredo-ab.ide) had been (re-)created in the last update cycle. The dates in the autoupdate cache though were correct (i.e. as expected). As I'm checking it gets even more obscure: on several clients 112 IDEs (either the same or all but adloa-lu.ide instead of which the most recent dwnl-lbn.ide is in the list) have been recreated in one of today's update cycles and one has re-written all except adloa-lu.ide. Just as I'm writing this the numbers are rising (adloa-lu.ide and one or two next in the alphabet joined the pack).  The management servers are not affected.

Don't ask me for an explanation (The Observer Creates Reality Simply By Observing?).  

Christian

Hi Christian,

Here's an easy way to see this in action. Look carefully next time you get an update at the c:\program files\sophos\autoupdate\cache folder. You need to do this when there's actually an update going to happen. You'll see a tempsavxp folder get created (if it doesn't already exist). You can then see lots of disk activity and you'll see this folder grow in size to match the original SavXP folder. Since both folder exist and are the same size, this is a 'copy' and not a 'move' process so now we have duplication of the install folder. The new downloads are then integrated into the tempsavxp folder and then checksummed. Assuming all is good, the SavXP folder is emptied and the tempSavXP copied back to SavXP (I actually think this process may use the 'move' function rather than copy but haven't proved that completely yet). Be carefull not to confuse the autoupdate cache with the Sophos installation folder. Once the cache update is complete, the installation will integrate into the sophos folder correctly only updating a few files there. It's the cache folder that's causing the fragmentation issues.

The duplication of all these files causes windows to gradually fragment over time and since this is a significant chunk being duplicated several times a day, it doesn't take very long before fragmentation starts to slow down disk access rates. This isn't so noticeable on Vista/7 because they have weekly timed tasks to defragment in the background and it wasn't such a noticeable problem when SAV was only 30mb's or less. With the gain in package size, this is becoming a real headache. I guess if you've all got superfast high-spec'd machines it's not going to worry you much but going back to the real world and especially now when the economic climate is not good for upgrading and unecessary impact on performance is being watched very closely.

Needs a revamp if you ask me.....

Matt

Yes, this is something we are looking at improving. It's current behaviour is a side-effect of the way it tries to do 'atomic' updates where it verifies the consistency of the install set, and backs out if problems occur.

However, there are several optimisations we are looking at making to this process.

Cheers,

John Reynolds

Thanks John,

I'm glad you're aware of this. It's becoming more noticeable as the install package gets larger and larger. More and more fragmentation occurs with larger copies and with the role out of EPS v.9, typically I'm seeing 2-3 weeks before I've got > 30% fragmentation whereas that was around 3-4 months on v.7. It's just the shear package size and the necessary frequency of updates that's causing this.

FYI, typical machine to observe would be a P4 2.4Ghz running XP-Pro SP3, around 30Gb's disk capacity which I state to my users should remain at < 60% usage (although most seem to run around 70-75%). Bigger disks reduces the problem but then if you give users bigger disks, they go fill them up unecessarily and that's another issue.

Matt

P.S. Would be interested in any betas trials relating to this.....

Thanks for the feedback. We've been looking at the copying issue to improve overall updating performance. Disk fragmentation isn't something we've analysed in detail, but cutting down on the copies should help with this.

I'll try to find out where we've got to with this.

Cheers,

John Reynolds

John,

It's the copying causing this so you're correct, if you cut down on that, the fragmentation problem will go away on it's own. As I say, this was as noticeable when SAV was but a few MB's but the increasing update frequency and overall size now is really showing this problem.

Matt

about 20 to 30 new IDEs per update cycles 2 to 3 times a day :)

It's good that Sophos is looking into this, however: It is a known fact that NTFS and Windows NT+ do not work well on drives which are nearly full. It is good practice to have a good amount of free drive space on a system drive, and administrators should take appropriate measures if this is no longer the case.

If an increase in hdd-space is not possible, some place might be gained by deleting some uninstall-folders for patches or uninstall-folder of the last service pack.

Best regards,

Detlev

Hi Detlav,

I agree. This is why I state clearly that my users should not fill beyond 60% capacity. In reality that's hitting 70 to 75%.

If you draw an arc of performance against usage, you'll see that at around 66% (of disk platter usage - not capacity), the curve rapidly increases exponentially until we hit around 98% then it tails off.

Unfortunately, NTFS and FAT aren't very good at preventing fragmentation so you get the problem even though the FS has no real need to fragment. This is mostly down to rapid movement of files and journalising (for NTFS - so the problem is worse on NTFS than FAT). You can see a disk that's only 20% full show very heavy fragmentation problems in relatively no time at all by copying a folder of around 100Mb's total size containing 100 to 200 files. If you copy this folder over and over back and forth between two directories, a 100GB disk with 20GB used copying 100MB's 300-400 times will show up as heavily fragmented. As you increase capacity, you'll see the fragmentation issue occurs much sooner reducing from 400 times at 20% to around 100 times at 60%. Once you hit around 30% fragmentation, you'll see that the entire disk structure is now utilised and the only space left is the gaps between files with no contiguous space available.

To prevent the above, you need to control the amount of copying going on and reduce it as much as possible. If the Sophos update can be rearranged such that the downloaded IDE gets integrated directly and safely into the install folder (that's the c:\program files\sophos\autoupdate\cache\SavXP folder) and the process doesn't rely on copying this entire folder out, integrating and copying back, then fragmentation caused by the 5-10 updates per day will dramatically reduce.

I have little else going on on the local users disk as all data is stored up on servers with roaming profiles so the users own movements of data are not causing problems. Looking carefully at the disk clusters I'm able to see that the SAV updates are the fragmentations. Once a disk is fragmented, the update cycle slows the machine down dramatically until it becomes basically unusable for around 2-3 minutes and the reason for the slowdown is the disk thrash that's now going on while it copies the cache folder, integrates the download and copies back. The actual install after that takes just a few moments.

Matt

Same behaviour today - not it's 118 IDEs which seem to be written anew (Date Created) in the \Sophos\Sophos Anti-Virus folder. List extends to bredo-ah.ide. Anyone else encountering this?

Christian