QoS issues (again)

Hello,

What are the results when you perform the test without any users on the network and please share result? Also have you tried directly testing without the FW? PC->directly connected to router->Speedtest? could you share result if you're getting exactly 500 or at least near 500MB DL speed w/o the FW?

Also could you share your current SFOS version? Thank you

Regards,

I have replied to you, but my post was flagged as spam, possibly due to the fact that I pasted speedtest links to share the results (was running the windows app this time).

Anyway, since I don't know when/if my reply will be unflagged, I am replying again..

So at the moment there is not much bandwidth consumed at the house

Run a speedtest (from a linux machine now) without the firewall rule enabled:

Download: 422.91 Mbps (data used: 407.7 MB)
32.73 ms (jitter: 4.52ms, low: 3.74ms, high: 253.45ms)
Upload: 52.96 Mbps (data used: 25.2 MB)
44.28 ms (jitter: 9.33ms, low: 19.42ms, high: 292.46ms)
Packet Loss: 0.0%

I enable the firewall rule again:

Download: 313.15 Mbps (data used: 238.5 MB)
5.93 ms (jitter: 1.43ms, low: 2.98ms, high: 12.61ms)
Upload: 47.09 Mbps (data used: 22.4 MB)
2.84 ms (jitter: 0.41ms, low: 2.27ms, high: 12.94ms)
Packet Loss: 0.0%

I disabled the rule again and at the next speedtest I got 430/45

I enabled it then once more and got 310/47

The ISP's router is set to bridge mode, which effectively turns it to an ONT

I have seen many times the speed reaching at 500, but even if I didn't, the QoS rule (which has the correct numbers for KBps) should not limit me to 300, it should at least give me the full available speed since it is capped at a higher speed that the actual one. It should not contantly cap me at 300

I am running the latest sfos version SFVH (SFOS 20.0.2 MR-2-Build378)

Hello ChriZathens ,

For step 2, This is possible. Let me explain.

Enforce guaranteed bandwidth - enable

- Any firewalled traffic passing thru WAN (inbound/outbound), where traffic shaping policy is not applied, would go thru "default policy". In your case, it's wrong configuration because you now have 2 traffic shaping policy which exceeds total ISP bandwidth you have.

When you would be doing speed test and you might have other traffic which can compete for guarantee, you might see some drop which would make TCP connection to go in "slow start" mode and would lead to decrease in bandwidth usage.

Do you mind setting guaranteed value too low (like 2KBps) and check?

Enforce guaranteed bandwidth - disable

- Either admin has orchestrated all firewalled traffic to go thru some traffic shaping policies (either firewall/user/web/app) OR providing guarantee is not the primary use case. In such scenarios, default policy would be "disabled". 

If no other traffic is going thru during speed test at that time, speed test might give you good results but if you have some other traffic bypassing traffic shaping policy, it can affect traffic which is going thru traffic shaping processing. And you might see "decrease in bandwidth usage".

In general, if you just have simple use case of traffic shaping policy and planning to have 1-2 firewall rules only, you may want to consider "disabling" enforce guaranteed bandwidth. However, if you are planning to have more firewall rules, better to divide bandwidth among all firewall rules and "default policy". It shouldn't exceed "total available WAN bandwidth".

Hello again!

With Enforce guaranteed bandwidth - enable I tried setting the guaranted value to a low value, as you said. I set it to 20  and the results are:

Download: 231.99 Mbps (data used: 273.8 MB)
6.96 ms (jitter: 1.65ms, low: 2.01ms, high: 14.47ms)
Upload: 52.71 Mbps (data used: 23.8 MB)
40.62 ms (jitter: 1.22ms, low: 28.77ms, high: 48.41ms)
Packet Loss: 0.0%

Setting Enforce guaranteed bandwidth - disable

Download: 412.24 Mbps (data used: 364.3 MB)
23.45 ms (jitter: 1.59ms, low: 3.07ms, high: 39.31ms)
Upload: 52.76 Mbps (data used: 23.8 MB)
43.55 ms (jitter: 2.12ms, low: 5.30ms, high: 52.29ms)
Packet Loss: 0.0%

Additionally you say that 

" In your case, it's wrong configuration because you now have 2 traffic shaping policy which exceeds total ISP bandwidth you have."

But the thing is exactly that: I don't have 2 traffic shaping policies. The traffic shaping policy I have created is attached to a top level firewall rule, which only applies traffic shaping, nothing else, and for the purposes of these tests, this specific firewall rule is disabled. I am only messing with the "Traffic Shaping Settings"

So let me ask it more simply:

Let's pretend that I don't have set anything regarding traffic shaping. (please pretend that I have installed XG right now and it is out of the box.)

I have a 500/50 internet connection.

I want to limit ALL machines via a single rule to not consume more than 450/50. 

Can you please tell me what I need to do?

Hi,

1/. you don't need a firewall rule

2/. using the screenshot I posted but modifying the values as recommended by Sanket Shah.

Ian

Hello Ian!

But this is what I am saying form the start.. The values don't work.

1. There was only one firewall rule with QoS. This rule is disabled

 So there is absolutely nothing in settings limiting the bandwidth.

2. I run a speedtest. Results:

Download: 415.84 Mbps (data used: 466.4 MB)
27.21 ms (jitter: 1.99ms, low: 3.27ms, high: 40.02ms)
Upload: 52.87 Mbps (data used: 23.8 MB)
44.41 ms (jitter: 1.98ms, low: 13.42ms, high: 54.25ms)
Packet Loss: 0.0%

I then go to traffic shaping settings and set "Enforce guaranteed bandwidth" to Enabled with the below values:

Even if in the first speedtest I don't get the full speed, since QoS is set to 68500 (about 550Mbit), it should give me the full available speed, correct?

So with the above settings I run a speedtest. Results:

Download: 227.66 Mbps (data used: 215.1 MB)
9.57 ms (jitter: 12.67ms, low: 2.14ms, high: 232.48ms)
Upload: 52.79 Mbps (data used: 26.5 MB)
43.15 ms (jitter: 1.47ms, low: 4.20ms, high: 48.14ms)
Packet Loss: 0.0%

I 

I immediately go to traffic shaping settings and set "Enforce guaranteed bandwidth" to Disabled again and run another speedtest:

Download: 411.33 Mbps (data used: 465.5 MB)
19.72 ms (jitter: 1.77ms, low: 3.39ms, high: 41.71ms)
Upload: 53.02 Mbps (data used: 25.1 MB)
43.50 ms (jitter: 1.43ms, low: 16.28ms, high: 48.40ms)
Packet Loss: 0.0%

As you can (both) see, traffic shaping settings does not limit according to the numbers I set. This has been my problem from the beginning of this discussion.

you need to change the limit to less than your link speed otherwise the firewall has nothing to negotiate

and increase your guaranteed value.

Works fine for me. I have a firewall rule with  QOS policy for my VoIP service only.

Ian

I did what you proposed.

Run a speedtest and got 420 Mbits

I decreased the total bandwidth to 50500

I set guaranteed value to 5000 and limit to 50000.

I run a speedtest and I get 240Mbits.

I then set Enforce guaranteed bandwidth to disabled and again I run a speedtest.

I get again 420 Mbits

It just isn't working as it should...

The following is the results I get with the settings I posted earlier. I thought I had posted this previously.

Thanks for the screenshot. Never doubted that it works for you. It is just that for some reason it does not work for me..

And after a few tests, I believe there is a possibility that it would not work for you, either, if your speed was above the one you have.

What I mean:

I performed a first test using your numbers. (Total 32500, guaranteed 30000, limit 31000)

The results I get are similar to yours .. I got 204/44 - forgot to take a screenshot :)

I then cut the numbers in half (Total 16000, guaranteed 15000, limit 15500).  Run a speedtest and download speed indeed was cut in half

I then set the numbers at a quarter of yours (Total 8000, guaranteed 7500, limit 7800). Run a speedtest and again the speeds are cut at a quarter of your speeds.

So everything from about 250Mbits and below works as expected so far. I then set once again the speeds at about double your numbers. (Total 60000, guaranteed 58000, limit 59000)

And lo and behold, the speeds are not exceeding your numbers

I am starting to believe that there is a bug and it won't let speeds go above about 250Mbits.

And it is just a diabolical coincidence that your speeds don't exceed 250, so you have never experienced this issue.

But of course this is purely hypothetical. I hope there is someone with speeds above 400 that can run a test (??)

I previously had a 1000/50 link which maxed out the xg115w at around 600-700.

Don't know then...

Can't be sure. I can't understand why I am the only one having this issue.

Is there perhaps any bug introduced lately that is causing that behavior?

If there is anyone with the latest version kind enough to test with a connection faster than 400 Mbits, it would be great.

At least I will be sure that the problem is only on my side (although I can't comprehend why this would happen only to me...)

Don't know then...

Can't be sure. I can't understand why I am the only one having this issue.

Is there perhaps any bug introduced lately that is causing that behavior?

If there is anyone with the latest version kind enough to test with a connection faster than 400 Mbits, it would be great.

At least I will be sure that the problem is only on my side (although I can't comprehend why this would happen only to me...)

Do you have ips enabled on the testing rule p, if so what are the settings?
ian

I did not even have a rule in place during the above tests.

But just to make sure I enabled the rule now and got same exact results as the last speedtest screenshot (about 250down)

The rule has no ips configured. It is just a simple rule. Source zones all my internal networks, destination zone WAN and from the rest of the settings I have nothing configured, no app control, no web filter, no ips, just trafiic shaping:

The Home_QoS rule has the below settings:

Perhaps it is worth mentioning that if I change Upload bandwidth to half of the above (3300), the upload speed is limited to half, as it should

So once again the "low" numbers work as they should.

Numbers that offer a download limit above 250Mbits are the ones that don't work

OK, here is mine

Additionally let me share that for download speed in the traffic shaping rule, even if I put the max number 2560000, it still limits to 250Mbits 

It is as if it will discard any number above 30000-35000 for download limit

A suggestion you might like to consider, your hardware is being maxed out. You could try running two or three speed tests at the same time from different devices and look at the diagnostic graphs.

ian

It has crossed my mind and now that you mentioned it, too, I gave it a try

I run a windows app speedtest and a command line speedtest at the same time. During these tests I had the XG console running top.

The tests combined did not exceed  280Mbits down while CPU usage did not exceed 58%.

EDIT : Run again and CPU reached 72%. With only one test running it does not go beyond 60%

Are there any other commands I can issue from the console to possibly get more accurate readings?

The cpu is a 4 core so unless you go into console and run top or similar you will never see what is happening.

i did a deeper investigation into the cou, it is 11 years old model with in built nics, so probably you are seeing the best performance you will get.

ian

ian

Well, top is indeed what I run.

As I mentioned it does not exceed 72% and even that load was only momentarily

Does traffic shaping tax the CPU more?

I assume it does, but still, 72% is fairly low. It still has room to stretch its legs

For what it is worth, here is the CPU graph from the GUI (last 24hours)

Thank you for the graph. What it doesn't show is what the load on each core is? 72% would be the overall load for all cores. Probably means one or more cores is maxed.

Ian

I guess there is always this possibility

I run top from the console but top isn't clear, either.

htop would be a much better option since it distinguishes the CPU usage per core, but it is not installed

Do you happen to know if it is possible to install htop? Is there any possibility at all for sophos to install something additional using a package manager?

EDIT: Well, I found this page https://community.sophos.com/sophos-xg-firewall/f/recommended-reads/134787/sophos-firewall-understanding-top-and-atop-command-in-sophos-firewall-utm#mcetoc_1g4rqfs483

I tried running atop and top with Irix mode on and hit 1 to sort by CPU

While I CPU no CPU going to full utilization I must admit that don't  have a clear picture about it. 

EDIT2: I created a video while running 2 speedtests and showing top in the background. 

Don't know if you can understand more than I do, but if you can spare a minute, please take a look.

EDIT3: I took another video running speedtest more times. Will post it in 5

Here it is/cfs-file/__key/communityserver-discussions-components-files/126/Spedtestvideo.mp4