IP route not taken out of routing table when interface is disconnected

Hello,

I have this topology where a customer have 2 sites with different ISPs. There also is a wireless link between the 2 sites with OSPF advertising routes for each networks, including the default route to each ISP (with a higher metric).

Here's a topology diagram:

I noticed that the failover is not working when, for example, ISP 1 goes down. Let's say the modem at site 1 is down so the link gets disconnected on the WAN port, the default route to the ISP is still present in the routing table. It seems that if the interface is not "Administratively down", the route is not deleted and failover is not happening.

Now the wireless link interface is not configured as a WAN port (so no default gateway). Instead I am using OSPF to advertise the default route on each side. Manually shutting down the interface in WebAdmin removes that route and failvoer works propoerly.

Is there a way around that and basically remove that route, which is no longer working, from the routing table, and then have the advsertised default route be used instead?

Not sure if I am missing some details. Please let me know if something is not clear.

Thanks