9.400 bricks connected APs!

Hello Zdenek,

we just did some investigation regarding this regression you mentioned, and indeed there is a regression, but it is a bit different then you think it is. First of all the AP is probably not bricked (even if it looks like). What is broken is the fallback mechanism, which is probably used in your setup as default due to the connecting VLAN being untagged (coming later to it). When you configure your AP with the VLAN tagging it tries to connect the UTM over the specified VLAN, if it can not do so it will after some time fallback to a default LAN behaviour this means it contacts the UTM without using a VLAN tag. And this fallback is currently broken in the 9.400.

If you use this vlantagging option and meanwhile configure your switch to use the specified VLAN with 'untag', the AP will first try to connect the UTM with the specified VLAN. This leads to the packets going out from the AP with the vlantag, the switch will forward them to the UTM, but as soon as the answer from the UTM comes to the switch it will get the vlantag removed. Thus the AP is not able to match the answer (without VLANTAG) to his request (with VLANTAG), then after some time the AP will go to the fallback and work in this fallback mechanism, the bridge 2 VLAN networks are not effected by this fallback, so they still work as expected, it is just the way the AP contacts the UTM.

We are working on the fix for this regression, but for now if you want to get the APs back running which don't show up anymore you could provide the VLANTAG as 'tagged' from the switch to them, then they should come up working again.

Regards,
Emanuel

Hello Emanuel,

thank you for the response. The switch port the AP is connected to is configured with the AP control VLAN being both tagged and untagged on it plus the bridged VLANs are tagged there as well. So just like in past where it works, the AP should be able to reach the UTM via tagged or untagged way.

With AP50 I can see that the AP boots and gets stuck in the infinite loop while trying to get IP from DHCP server (AP sends request, server provides offer but AP never acks it). When I deploy the AP50 without VLAN tagging and using just some testing SSID which bridges the traffic to AP LAN all works fine. AP connects to the UTM, flashes the firmware, reboots and wifi starts working. After I change the configuration to VLAN tagged setup it gets stuck in the DHCP cycle.

However, the AP55 once the VLAN tagging config was pushed to it, it no longer finishes booting and does not even attempt to send DHCP request and just reboots over and over again in random intervals. Unfortunately there is no flash tool for the new APs so right now it is a brick.

Right now I'm not able to do any more tests as I had to rollback the 9.400  back to 9.355 where all works as WiFi with VLANs is critical and I have to keep the business going.

Cheers,

Zdenek

Hello Emanuel,

thank you for the response. The switch port the AP is connected to is configured with the AP control VLAN being both tagged and untagged on it plus the bridged VLANs are tagged there as well. So just like in past where it works, the AP should be able to reach the UTM via tagged or untagged way.

With AP50 I can see that the AP boots and gets stuck in the infinite loop while trying to get IP from DHCP server (AP sends request, server provides offer but AP never acks it). When I deploy the AP50 without VLAN tagging and using just some testing SSID which bridges the traffic to AP LAN all works fine. AP connects to the UTM, flashes the firmware, reboots and wifi starts working. After I change the configuration to VLAN tagged setup it gets stuck in the DHCP cycle.

However, the AP55 once the VLAN tagging config was pushed to it, it no longer finishes booting and does not even attempt to send DHCP request and just reboots over and over again in random intervals. Unfortunately there is no flash tool for the new APs so right now it is a brick.

Right now I'm not able to do any more tests as I had to rollback the 9.400  back to 9.355 where all works as WiFi with VLANs is critical and I have to keep the business going.

Cheers,

Zdenek

Hello Zdenek,

for my understanding, let's say you have the vlantag 500 configured as the AP control VLAN, and the switch has VLAN 500 tagged and untagged on the port the AP is plugged in. Now a packet with VLAN 500 arrives at the switch (which is for the AP) what does/should the switch do? Send the packet to the AP with the VLAN or send it without the VLAN? My expectation would be to send it without the VLAN.

So my understanding is that if you have the switch configured VLAN 500 as untagged and tagged then both is allowed, so when the AP sends a packet with vlantag 500 it gets forwarded to the utm and when the AP sends a packet without any vlantag it also gets forwarded to the utm, but for the answer path there needs to be a decision. So I would expect the behaviour previously described with the fallback. (which is broken since 9.400)

Anyways the problem you describe with the AP55 sounds a bit different since it does not even send the DHCP request on the VLAN, we will continue looking into that one. Was the AP55 also configured in 9.35 and then after update it went in the state, or did you configure it directly in 9.4?

Regards,

Emanuel

Sorry for late reaction., The AP55 was not connected to 9.3. It has been connected directly to 9.400 (it was a fresh AP55 out of the box). It appeared in the pending list, was accepted, firmware update was pushed to it and it never came back up.