Speed testing on a GB internet connection: To VM or not to VM

Wow - lots of work and very detailed results.  I run a pretty big ESXi shop over the last 5-6 years and also have an ESXi box at home running UTM 9.  It's a pretty new installation with a Gigabit fiber that's less than 2 months old.  I'm sure you've heard it - but turning off IPS will shoot your connection right up to the low to mid 900mbs for upload and download.  It's just how the UTM threads are built.  It really doesn't matter the resources you throw at it - it won't get faster.  To get IPS to run at gigabit speeds, you need more money for home gear than is smart.

I've got some packet issues with my ISP right now which requires PPPoE for my gigabit fiber.  But everytime I enable IPS - I see a speed drop big enough that I'm willing to run w/o it enabled.  My host is Core i5 3.9Ghz, 32GB RAM, SSD boot and various internal drives as well as iSCSI to my Synology DS 1815+.  I have the default specs provided to the UTM and from what I read - doesn't matter if I had a Dell box like yours or something double in size, the overall network speeds are held back by the IPS scanning engine and will cut your gigabit speeds in half.

Nice work on the testing.

it has been a while since i have used vmware but here is something you can do:

create a pool of resources equal to 4 cpu worth of ghz and 8 gigs of ram.  Then create a vm with two vcpus and assign 100% of that pool of ghz to that vm.  Try your tests again.

Two vCPUs will never use more resources than 2 cores worth of GHz, regardless of how you configure it.  If you have 8 cores of 2ghz of power and assign all 8 cores to a resource pool, then assign a VM 2vCPUs, that VM will never see more than 4ghz of processing power.  If that is wrong, please point me to the document from VMWare that states it can.

it used to be you could pool ghz(v3) i guess they took that away to work more like hyper-v.  I ahd setup multiple vm's using that philosophy.

you can get to near gigabit with the firewall only with 3.4 ghz..as you add things the performande is going to go down.  The only way tog et close is to drastically increase the number of simultaneous users(50 or more) AND the number of vcpus(4+) depending on your user counts.

it looks like i am going to have to setup vmware on a spare server of mine and re-test my theories when it comes to vmware.  I prefer hyper-v myself.

aha found it.  it is claled a resource pool:

https://communities.vmware.com/thread/405968?start=0&tstart=0

so it will work exactly as i have talked about...i didn't think vmware would take that away..:)

www.google.com/webhp

Resource pools only compartmentalize resources for scheduling purposes.  There is no way to force multiple CPUs to work as a single thread.  It simply isn't possible.  You could not make a 1 vcpu box consume 16ghz of processing power with cpu resource pools.  Test it out, you will see.  It will spread that one cores worth of gigahertz between different physical cores, but the guest/vCPU will never see more than a cores worth of performance.

The handling of the data from a single transfer is clearly a single thread.  When the transfer is happening, The CPU utilization only spikes to show a single core going all out no matter which config I use.  Neither ESXi nor any other OS out there has the ability to split code from a single thread onto multiple cores. The person who figures this out would be very rich indeed as CPUs have essentially been frequency bound for many years now. 

first of all there is a known bug with Linux and the 82571 chipsets.  Vmware runs off a Linux kernel.  This chipset is deprecated by intel and no further fixes are coming.  I would use either broadcom or intel i-211 series nics.  The 219's are too new for right now.  Search for your chipset number I posted about this bug a while back..:)  either way with only 5 of you yes you are waaaay overkill and i bet you are getting hit by the clock throttling issues in modern chips i have talked about before(search for my dissertations on that).  Go into your bios of the host and turn off all power management.  In essence lock the cpu at max frequency.  In modern chips this has little to no impact on power usage.  I would then put your UTM vm down to 2 cpu cores.  You can go 4 if you want to use http proxy.  With 4 cores you will get two instances of snort each capable of about 250-450 mbps per thread.  If you are successful in locking the clock to max then you can go six cores which with IPS on should allow you to get very very close to 1 gigabit..but only if you have 3 or more simultaneous users.  Search for my dissertations on snort as well..:)