How can I manually set certain devices as "unlicensed" (not counting towards license)?

I think you are worrying too  much. If a virus etc gets into your internal network then you more than likely have a compromised UTM.

If you do your own scan to find active devices then yes you will have a problem for any new connections, the existing connections will not be affected. The licence count will reset after 14 days of no activity, from memory.

If you want more than "50 IP licence" you will need to pay for it at the going market rate.

There are a number of resellers who frequent these forums that more than likely can help you. I would also pass on the resellers are not being of assistance to you to them.

I think you are worrying too  much. If a virus etc gets into your internal network then you more than likely have a compromised UTM.

If you do your own scan to find active devices then yes you will have a problem for any new connections, the existing connections will not be affected. The licence count will reset after 14 days of no activity, from memory.

If you want more than "50 IP licence" you will need to pay for it at the going market rate.

There are a number of resellers who frequent these forums that more than likely can help you. I would also pass on the resellers are not being of assistance to you to them.

I think you don't understand the problem.

Lets say I have 20 devices on network, where 5 is currently off.

Then someone does a scan. Does not matter, could be a malicious wireless user who have not authenticated to the captive portal, or a virus that manage to get into one of my devices, or even a friend/family member/guest that connect a infected scanning device to the network.

This means that all licenses (devices) gets exhausted, and then those 5 devices that are currently off, will be unable to connect for 14 days. This is a DoS security risk.

What I want to do, is to tell that: "these devices: 192.168.1.21 to 192.168.1.254, should NEVER be able to connect, all traffic from these devices should be ignored, and should NOT be counted on license".

Blocking them in firewall is not enough, as they will still count in licensing, even if 100% of traffic is dropped.

Would it be possible to define a lower subnet to avoid the problem? Like defining the subnet on the interfaces like 255.255.255.240, to restrict that subnet to 14 devices (and then doing this for every segment on network). What happens if devices outside of the subnet attempts to connect? Will they be ignored by licensing (as obviously the firewall will of course ignore traffic from outside its defined subnet) or will it still count in licensing?

I understand the problem and you are worrying too much. If you restart your UTM the licence count is reset from what I can recall.

If one of your managed devices 'gets' a virus then you have bigger problems than just worrying about your licence count.

Why would you let someone access your captive portal if they are not part of your normal family group that lives at your home?

You have to tell the users about your captive portal for them to login to it, you would have to set them up and give them password etc.

You setup two groups using the DHCP on different networks vlans etc one you use for your managed devices and the other for guests. The one for guests does not point at the UTM until you are happy their device meets your security requirements and then you tell them the gateway address or give them a username and password. To me this is way over the top for home security.

Update.

Another approach is to assign fixed IP addresses to your devices, then limit the DHCP server range to 5 to 10 devices.

What I have understand they changed that in one of the later versions so UTM restarts wont reset the license, because of abuse from commercial users to avoid/reset the time limit.

And with captive portal, you have completely misunderstand about what a captive portal is for. A captive portal is designed to let untrusted users to access, so they can authenticate and become trusted. So lets say a neightbour connects to the captive portal, and starts scanning for vulnerable devices or something.

about the DHCP server range and fixed IP adresses, it wont solve the problem, as any device who sends packets, spoofed or not, will count in licensing. Thats what we are discussing.

I wondering, what happend if I assign like 192.168.1.1/28 to the wireless interface, limiting that to 16 devices including reserved adresses. If someone sends packets from outside the subnet, like from 192.168.1.25, will these packets count in licensing?

So you have your own captive portal that is not part of the UTM, so how do they connect to it? You still have to tell them where it is to connect to it? You seem to misunderstand the whole concept of security, if your neighbour has connected then your security has large holes in it, you need to address these first before worrying about IP address usage.

No. Of course I use the captive portal in Sophos UTM.