This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

XGS2100 to 19.5 fail- Red Interface- DGD-"stopped"

I started this update this morning around 10am (14 hours ago).  I was running 19.0 with no issues.   I have the XGS2100 with the 4x 10Gb SFP+ Flexi_port module.  I have 2G fiber in my A2 (SFP+) then in Port2 (1G) I have a coax modem as my backup.  A1 (10G) DAC to a mikroTek switch feeding LAN

Everything was working until after applying the 19.5, once it came back up my interfaces all show red. and the DGD is orange.. I have some sort of WAN.  It jumps back and forth between my fiber and coax Public IPs.   I have SIP VIOP phones which can call out with, but when someone calls... it will either goto a random phone/fax or voice mail,  In the SIP PBX the phone say "some  reachable"   and if you hover over it .. they will have multiple MAC address and both public IPs.  Everyone's phone also shows they have voicemails, miscalls, and the wrong date/time.   Rebooting a phone will fix it for 2-5 mins other than the date/time. 

I have noticed that some of the routing info is straight missing... SLA, SD-WAN Route, health checks as well as a IP Sec connection. 

I rolled back to 19.0 and still all my interfaces are red other and DGD is "stopped" .    I have loaded a bunch of restore from backup.. starting with the 8:20am one this morning.  I loaded one from December 4th, 2022 that does turn the interfaces green but still works all the same and DGD is still "stopped".  It also brings the IP Sec info back. I also tried to just unplug my coax, deleted my coax interface, unplug my fiber.  Put state routes in for phones/fax to coax only and SIP ip only. Rebooted every switch/ modem/ phone/ ATA/ door controller. 

In the logs I see each time I reboot

MESSAGE  Jan 03 18:49:26Z

[4760]:  Starting DGD: Tue Jan 3 13:49:26 2023

MESSAGE  Jan 03 18:49:26Z

[4760]:

*****************************************

CRITICAL   Jan 03 18:49:26Z

[4760] line: 84, syntax error

Anyone know how to turn the DGD back on ?



This thread was automatically locked due to age.
Parents
  • The Development Team looked at my appliance and made backend clean up to start DGD and said it was safe to update to 19.5v now.  I clicked on the boot into the 19.5 firmware and I’m exactly where I was before this post. (XGS says both interfaces are offline and DGD is stopped)  I’m almost certain the DGD hasn’t been working right on the XGS since we deployed it because some of the VLANs had to have static routes and static gateways… the phones would flip back and forth from my main ISP and my backup. 

    One of my guys suggest I do a factory reset and load the config from a backup. I’m currently on 19.5… phones have been offline since 8:30pm and the PCs continue to flip from ISP to ISP and if I unplug either ISP I’m completely offline.

  • Hi Charles Cook ,

    I was the engineering who looked at your setup and fixed DB inconsistencies. Subsequent migration to 19.5 should have worked as suggested by us. But it didn't worked as per  your last comment.  I think what went wrong is not doing the migration rather 'booting into 19.5'. This is called "FIRMWARE SWITCHING".  Since your system is in problematic state, I would like to take a look and confirm the same. 

    Sanket has shared me the access-id you pmed to him, I'll take a look, if it can be fixed directly on 19.5

    -HTH

    Moheed

  • i see you already got DGD and the interfaces both showing online.  The steps I took to get move back into 19.5 was from inside the gui-

    Backup & Firmware/ "firmware" tab... then in the listed version- "SFOS 19.5.0 GA-Build197" in the Manage column. i clicked to the boot to firmware image.   Should this firmware switching option not worked ?   

  • Hi Charles,

    Switching back to 19.5 firmware won't work in this case as the migration from 19.0 to 19.5 done earlier would have made the 19.5 DB inconsistent. If it was a migration done again, it would have worked. I've fixed the 19.5 DB inconsistency and everything should be back to normal including DGD/interface issues you were observing. Let us know if anything else is required from our side.

    FYI, Fix for this issue is being tracked via: NC-112370.

    -HTH

    Moheed

     

  • Now that I appear to be up and running normal, Ill add some of the missing SLA, interfaces, routes back in.  I wait until there a patch or a update that fixes the NC-112370 before I do any updates/ FW changes.   

    I also noticed within my SD_WAN routes that one of the routes have an Orange dot beside it.  I don't really see any explanation of what orange means. I don't see any wrong with the SLA strategy which is best quality from latency.  Seems like the color orange within sophos normally means there is a warning.  I know the APs orange means offline. 

  • Hey Charles,

    For the sdwan-route showing Amber is because it has been set with 'Route Only Through Gateway' option.

    This seems a display bug. However, it doesn't have any impact on traffic. It indicates that gateways of that profile had been/are unavailable to route traffic. We are tracking it via NC-113168 to get it fixed ('show correct route-state on gateway up in profile") in future release.

    Work-around: Save the impacted policyroute again from UI.

    I've applied the workaround and SDWAN-route is green. 

    -HTH

    Moheed

  • This SD-Profile is for 2 RED tunnels.  I want it to use the best available circuit according to SLA by pinging/ probing a server at a the main office.   Might be some helpful information, but during the deployment of this XGS we renamed an interface and for whatever reason that crashes the interface.  Ultimately we deleted the interfaces and put them back in with the use of the GUI... i It sounds like might all of these issues are left over from whatever files from the db are not "renamed" when you use the GUI to change the name of an interface.  

  • Thanks Charles Cook  for insight. It echos our findings on NC-112370.

Reply Children
No Data