Hardware needed for 300/300 router

I have a 500/50 WAN connection and I'm able to hit the full speed with the following hardware when IPS is turned OFF. If IPS is turned ON, I can only get ~130 Mbps. It doesn't matter what rules are turned on for IPS, once enabled there is a cap at ~130 Mbps. Currently on 9.355-1. I've never seen the CPU go above 20% usage and it usually idles at ~1%-5%. About seven devices on the network.

Hardware:
Lenovo M81 Desktop (ThinkCentre)
Intel Core i3-2100 (3.1 GHz)
8 GB RAM
Intel NICs
500 GB 7200 RPM HD

Enabled:
Firewall
Web Filtering w/ SSL Decrypt
Application Control
Endpoint Protection
Remote Access
Web Application Firewall
Antivirus
Antispyware

Exact the same behavior here. 500/50 Mbps connection and with IPS ON the troughput on speedtest is extremly low, down to ~160 Mbps. Even with exceptions in IPS (which has NO impact). The CPU idles around ~5-10% in this time....

I think we need an 4.5GHz CPU to get our 500MBit downstream.... which is ridiculous for an UTM.

Guys, search here for posts by William concerning throughput and Snort.

Cheers - Bob

BAlfson, other than the technical reasons why snort slows down throughput is there something I should be looking for when searching Williams' posts? Are there some workarounds or something to get throughput higher with IPS on?

Thanks!

Hi,

1. 'we' used to prefer Intel NICs over Broadcom... BC doesn't have an open-source driver, etc.

However, there were (are?) some major problems with the Intel drivers after Linux kernel 3.0 came out, and afaict they're not fully fixed in UTM 9.355 (I still have trouble with 1 NIC (an older Intel PCIe model), but it's also possible I have a bad NIC or motherboard).

Because of this, I'd go Broadcom over Intel if given a choice.

However, finding a modern, compact (mini-ITX) board with 2 Broadcom NICs seems impossible... the closest match I've found is the HP MicroServer Gen8, but that uses an older CPU and clock speeds are very limited by HP's thermal design and unwillingness to let it compete with their more expensive servers, as well as the fact that only ECC CPUs are supported (although this may have changed in recent BIOS).

Another option is to get a dual-port Broadcom PCIe NIC... I have one, and am now using it in a new SuperMicro VMWare ESXi server. I couldn't get the NIC to work on a Gigabyte Haswell board (my most recent firewall) for some reason; the PC wouldn't turn on and GBT was extremely unhelpful.

Anyways, since you're probably going to have to get a board with Intel NICs, make sure the case has room for a PCIe card so you can add a dual-port Broadcom if you have to.

2. Re William (and my) posts... there is (was?) an issue with Intel's clock speed scaling/throttling (Intel SpeedStep); a system that was lightly loaded would not spin the CPU up to full speed. This cause problems for home users with quad-core i5/i7 CPUs, as a single user's traffic would usually only go through 1 Snort thread, and SpeedStep thinks that 1 thread isn't enough to bother to spin up the CPU clock for. This could cause worse performance with a quad-core CPU than a dual-core... I did verify it happened frequently on my i5-4670.

This _may_ be fixed by the new CPU scheduler in Linux 3.0+, but I haven't tested it nor seen confirmation that the UTM has the new scheduler. Unfortunately (for you), I am no longer running my UTM on bare-metal so I can no longer test that effectively.

If you have a lot of active users, you might want to go with the i5 anyways, but if not, the i3 is probably a safer bet. The i3 has Hyperthreading anyways, so it can still handle 4 threads.

I don't know if you've seen my benchmarks (https://community.sophos.com/products/unified-threat-management/f/52/p/29110/93717#93717 ), but a 3.4GHz Haswell CPU can handle 320mbps of HTTP traffic through the IPS.

IPS Notes:

a. HTTP is the only traffic you care about; the IPS is much faster handling anything else, as it does a lot of special 'preprocessing' for HTTP.

b. My tests were probably done before the IPS rule aging feature was added. This should help a bit, but I still think the preprocessing is the real bottleneck.

c. Sophos XG supposedly has some improvements in IPS speed. I haven't benchmarked it though. Anyways, 320mbps should meet your requirements.

3. Chips:
Fastest-clocked Skylake i3 you can get... The Skylake i3-6100 seems to be the fastest (3.7GHz) available at the moment, with 3.8 & 3.9GHz on the way. Don't worry about the 51 watt TDP, as all CPUs since Haswell idle at much lower power (all 3 of my i5 and i7 Haswell & Skylake systems idle around 30 watts (full system)), and they'll never hit full TDP unless you max out all threads and the GPU. In fact, a lower TDP CPU of the same generation is unlikely to be lower power at idle. The only time to worry about it at all is if you're trying to go fanless.

en.wikipedia.org/.../Skylake_(microarchitecture)

4. Boards:

I got a Skylake SuperMicro board for my new ESXi server... it's quite good... dual Intel NICs, vPro remote management features, ... However, it only takes SO-DIMM memory... good news is they're not too expensive anymore; I was able to get 2 Kingston Hyper-V 16GB SODIMMs (32GB total) for under $200.

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182990

or, you can buy it with a chassis with 4 hot-swap SATA bays & PSU for about $200 more; that's what I did as I needed the drives for ESXi. http://www.newegg.com/Product/Product.aspx?Item=N82E16816139104  The system is quite compact and very quiet; a little taller than my Fractal Design Node 304, but much shorter. It has a PCIe slot, I have my dual-port Broadcom NIC in there. I can't remember if it's half-height or full though.

Other options for smallish cases include the Node 304 or the Cougar QBX: http://www.newegg.com/Product/Product.aspx?Item=N82E16811553020 -- I built a system for a friend a few months ago, and the Cougar is easier to work with than the Node 304.

Both the Node 304 and QBX are extremely cramped for power supplies; it's much easier to buy a SFX PSU with ATX bracket than to try to fit any ATX PSU in. The best option (that includes a bracket) seems to be Silverstone: http://www.newegg.com/Product/Product.aspx?Item=N82E16817256097

If space is not a factor, there's lots of uATX or ATX choices for boards and cases.

5. RAM: get 4 or 8GB, DDR4 2133 low-latency RAM, e.g. CAS 15 or faster. The Hyper-X I got is CAS 13.

6. Hard drive: SSD is completely unimportant unless you're using the HTTP Proxy with caching enabled. SSD won't 'accelerate' anything else in the UTM other than rebooting.

Either way, configure automatic weekly backups via email.

Barry

Hi,

1. 'we' used to prefer Intel NICs over Broadcom... BC doesn't have an open-source driver, etc.

However, there were (are?) some major problems with the Intel drivers after Linux kernel 3.0 came out, and afaict they're not fully fixed in UTM 9.355 (I still have trouble with 1 NIC (an older Intel PCIe model), but it's also possible I have a bad NIC or motherboard).

Because of this, I'd go Broadcom over Intel if given a choice.

However, finding a modern, compact (mini-ITX) board with 2 Broadcom NICs seems impossible... the closest match I've found is the HP MicroServer Gen8, but that uses an older CPU and clock speeds are very limited by HP's thermal design and unwillingness to let it compete with their more expensive servers, as well as the fact that only ECC CPUs are supported (although this may have changed in recent BIOS).

Another option is to get a dual-port Broadcom PCIe NIC... I have one, and am now using it in a new SuperMicro VMWare ESXi server. I couldn't get the NIC to work on a Gigabyte Haswell board (my most recent firewall) for some reason; the PC wouldn't turn on and GBT was extremely unhelpful.

Anyways, since you're probably going to have to get a board with Intel NICs, make sure the case has room for a PCIe card so you can add a dual-port Broadcom if you have to.

2. Re William (and my) posts... there is (was?) an issue with Intel's clock speed scaling/throttling (Intel SpeedStep); a system that was lightly loaded would not spin the CPU up to full speed. This cause problems for home users with quad-core i5/i7 CPUs, as a single user's traffic would usually only go through 1 Snort thread, and SpeedStep thinks that 1 thread isn't enough to bother to spin up the CPU clock for. This could cause worse performance with a quad-core CPU than a dual-core... I did verify it happened frequently on my i5-4670.

This _may_ be fixed by the new CPU scheduler in Linux 3.0+, but I haven't tested it nor seen confirmation that the UTM has the new scheduler. Unfortunately (for you), I am no longer running my UTM on bare-metal so I can no longer test that effectively.

If you have a lot of active users, you might want to go with the i5 anyways, but if not, the i3 is probably a safer bet. The i3 has Hyperthreading anyways, so it can still handle 4 threads.

I don't know if you've seen my benchmarks (https://community.sophos.com/products/unified-threat-management/f/52/p/29110/93717#93717 ), but a 3.4GHz Haswell CPU can handle 320mbps of HTTP traffic through the IPS.

IPS Notes:

a. HTTP is the only traffic you care about; the IPS is much faster handling anything else, as it does a lot of special 'preprocessing' for HTTP.

b. My tests were probably done before the IPS rule aging feature was added. This should help a bit, but I still think the preprocessing is the real bottleneck.

c. Sophos XG supposedly has some improvements in IPS speed. I haven't benchmarked it though. Anyways, 320mbps should meet your requirements.

3. Chips:
Fastest-clocked Skylake i3 you can get... The Skylake i3-6100 seems to be the fastest (3.7GHz) available at the moment, with 3.8 & 3.9GHz on the way. Don't worry about the 51 watt TDP, as all CPUs since Haswell idle at much lower power (all 3 of my i5 and i7 Haswell & Skylake systems idle around 30 watts (full system)), and they'll never hit full TDP unless you max out all threads and the GPU. In fact, a lower TDP CPU of the same generation is unlikely to be lower power at idle. The only time to worry about it at all is if you're trying to go fanless.

en.wikipedia.org/.../Skylake_(microarchitecture)

4. Boards:

I got a Skylake SuperMicro board for my new ESXi server... it's quite good... dual Intel NICs, vPro remote management features, ... However, it only takes SO-DIMM memory... good news is they're not too expensive anymore; I was able to get 2 Kingston Hyper-V 16GB SODIMMs (32GB total) for under $200.

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182990

or, you can buy it with a chassis with 4 hot-swap SATA bays & PSU for about $200 more; that's what I did as I needed the drives for ESXi. http://www.newegg.com/Product/Product.aspx?Item=N82E16816139104  The system is quite compact and very quiet; a little taller than my Fractal Design Node 304, but much shorter. It has a PCIe slot, I have my dual-port Broadcom NIC in there. I can't remember if it's half-height or full though.

Other options for smallish cases include the Node 304 or the Cougar QBX: http://www.newegg.com/Product/Product.aspx?Item=N82E16811553020 -- I built a system for a friend a few months ago, and the Cougar is easier to work with than the Node 304.

Both the Node 304 and QBX are extremely cramped for power supplies; it's much easier to buy a SFX PSU with ATX bracket than to try to fit any ATX PSU in. The best option (that includes a bracket) seems to be Silverstone: http://www.newegg.com/Product/Product.aspx?Item=N82E16817256097

If space is not a factor, there's lots of uATX or ATX choices for boards and cases.

5. RAM: get 4 or 8GB, DDR4 2133 low-latency RAM, e.g. CAS 15 or faster. The Hyper-X I got is CAS 13.

6. Hard drive: SSD is completely unimportant unless you're using the HTTP Proxy with caching enabled. SSD won't 'accelerate' anything else in the UTM other than rebooting.

Either way, configure automatic weekly backups via email.

Barry

Only thing i would add is the Intel I series does not have the Linux driver bug that i have seen.