DPI on 4 GB RAM system - Linkedin still does not open

I realize this doesn’t actually help you but just wanted to mention I enabled SSL/TLS decryption and have no issues accessing LinkedIn. I also have 4GB of RAM but running an Intel Core i5-5250U. My memory utilization is sitting around 85% with IPS and Application scanning off on all my firewall rules. It’s around 90% with IPS and Application scanning applies to my firewall rules.

I am beginning to think DPI will need a couple of versions to mature. I don't have any problems like  when accessing linkedin but the proxy seems a lot more mature compared to DPI and I am sticking with it. Also, DPI seems starved on a 4GB system. I am not running any IPS rules nor am I using ATP and my memory is around 85 percent with only a couple of firewall rules and only ONE user ME.

I don't think the system is tuned so far to intel that a very recent amd chip can't compete with older intel units. Hyperscan https://software.intel.com/en-us/articles/hyperscan-and-snort-integration may have a little advantage towards intel but I don't think that is the issue here. 

I have said many times during this beta that snort can only do so much with memory caps that have to be used in a XG type device. I think we are running into the limitations of snort here. The non multi-threaded design also doesn't help here and we may be in situation where we are too cutting edge even for the very fast cheap processors we have available.

Can one of you guys with access to higher end appliances run a test on DPI, IPS, ATP, and application categorization enabled and see if it brings an appliance to its knees. If I had to guess, I would think the performance would be very similar that we are seeing on our own setups in labs and home.  

I definitely agree that 4GB seems to be "bottoming out" on the XG and has done since v16.5, it's the main reason why I hate the XG8x and XG1xx series (minus the 135, he's ok).

The other issue which causes the high memory load is regardless if it is use or is licensed, pretty much all services will be active/live or will be loaded into memory or not running. This includes Web Proxy and IPS which are the two largest memory consumers.

Optimisations don't always mean "performance increase", they can be also be for stability and various other niggles and issues to be ironed out which may not apply to a competitor product. I have no doubt that an EPYC 7H12 will take a steaming <you know what I mean> all over a dual Xeon 8180 system and then make your breakfast for you in the morning while you hobble down the stairs. But that's not the point why a specific architecture is selected and optimised for.

I am extremely disappointed still that the multithread Snort is not in use and have instead opted for a multi-process approach which is something I mistook for quite a while. This means that proper threading loses out and performance is left on the table.

I sadly do not have access to the higher end equipment or even software to test this, the largest I have direct access to test on are Xeon E5 second gens. Would that help?

Emile

I definitely agree that 4GB seems to be "bottoming out" on the XG and has done since v16.5, it's the main reason why I hate the XG8x and XG1xx series (minus the 135, he's ok).

The other issue which causes the high memory load is regardless if it is use or is licensed, pretty much all services will be active/live or will be loaded into memory or not running. This includes Web Proxy and IPS which are the two largest memory consumers.

Optimisations don't always mean "performance increase", they can be also be for stability and various other niggles and issues to be ironed out which may not apply to a competitor product. I have no doubt that an EPYC 7H12 will take a steaming <you know what I mean> all over a dual Xeon 8180 system and then make your breakfast for you in the morning while you hobble down the stairs. But that's not the point why a specific architecture is selected and optimised for.

I am extremely disappointed still that the multithread Snort is not in use and have instead opted for a multi-process approach which is something I mistook for quite a while. This means that proper threading loses out and performance is left on the table.

I sadly do not have access to the higher end equipment or even software to test this, the largest I have direct access to test on are Xeon E5 second gens. Would that help?

Emile

Oh no, I didn't mean throw the largest processor you have on it. I am sure the packet processing will be a lot faster on a newer i9 compared to an i3. What I am interested in are the sophos appliances that you guys are selling. Would they be able to handle and provide the throughput numbers published by sophos or the new numbers will be published shortly? Also people that bought a low end device that was good enough for v17.xx very recently... would it be just as good for v18 or would they be calling every day due to performance issues with DPI and additional ram demands.

And as a basic rule, I don't mind ALL my ram being utilized but XG is a firewall at heart. With cloud deployments, we should be looking at lighter footprint and not a cpu and memory hog.

Regards.