Thread Info
  • State Not Answered
  • +1 person also asked this people also asked this
  • Locked Locked
  • Replies 5 replies
  • Subscribers 38 subscribers
  • Views 3249 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

RED not working

BeEf

Hello,

we did an firmware update on an XG 550 Cluster last weekend.

Initially the RED 50  was running but after a colleague did some changes on another RED 15W it is disconnected now for unknown reasons.This was on Monday around 16:50. Currently the RED50 is the only device that is connected. Two other devices (one of them the RED 15W mentioned above) are offline.

Below you see the logfile of the device when it disconnected the first time:

Mon Jan 11 16:50:13 2021 REDD INFO command 'PORTSTATE 1E04,1004,1004,1004,1E04'
Mon Jan 11 16:50:13 2021 REDD INFO PORTSTATE LAN1: 1Gb/s,LAN2: Down,LAN3: Down,LAN4: Down
Mon Jan 11 16:50:13 2021 REDD INFO command 'PING 0 uplink=WAN1 uplinkstate=0 wan1_ip=192.168.0.128'
Mon Jan 11 16:50:13 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
Mon Jan 11 16:50:13 2021 REDD INFO PONG local_tx=0
Mon Jan 11 16:50:29 2021 REDD INFO command 'PORTSTATE 1E04,1004,1004,1004,1E04'
Mon Jan 11 16:50:29 2021 REDD INFO A3400E6BF2EA04B/REDs Pilz is now disconnected
Mon Jan 11 16:50:29 2021 REDD INFO device is disconnected.
Mon Jan 11 16:50:35 2021 REDD INFO server: New connection from 84.130.87.219 (cipher ECDHE-RSA-AES256-GCM-SHA384), rev1
Mon Jan 11 16:50:36 2021 REDD INFO connected OK, pushing config
Mon Jan 11 16:50:36 2021 REDD INFO A3400E6BF2EA04B/REDs Pilz is now disconnected
Mon Jan 11 16:50:36 2021 REDD INFO device is disconnected.
Tue Jan 12 09:53:27 2021 REDD ERROR server: generate device config for red A3400E6BF2EA04B failed: Failed to get red device cert pem File /conf/certificate/internalcerts/red_device_A3400E6BF2EA04B.pem does not exists
Tue Jan 12 09:59:42 2021 REDD ERROR server: generate device config for red A3400E6BF2EA04B failed: Failed to get red device cert pem File /conf/certificate/internalcerts/red_device_A3400E6BF2EA04B.pem does not exists
Tue Jan 12 09:59:52 2021 REDD ERROR Failed to register on provisioning service: Failed to register RED device on provisioning server: Curl command to register utm on the provisioning server failed with error 1792
Tue Jan 12 10:00:01 2021 REDD ERROR server: generate device config for red A3400E6BF2EA04B failed: Failed to get red device cert pem File /conf/certificate/internalcerts/red_device_A3400E6BF2EA04B.pem does not exists

After one of my collegues did some troubleshooting we see now this:

Failed to register RED device on provisioning server: Curl command to register utm on the provisioning server failed with error 1792
Tue Jan 12 11:03:12 2021 REDD ERROR server: generate device config for red A3400E6BF2EA04B failed: Failed to get red device cert pem File /conf/certificate/internalcerts/red_device_A3400E6BF2EA04B.pem does not exists
Tue Jan 12 11:03:16 2021 REDD ERROR Failed to delete certificates: Deleting RED device cert from filesystem failed: file /conf/certificate/internalcerts/red_device_A3400E6BF2EA04B.pem does not exist
Tue Jan 12 11:11:10 2021 REDD ERROR server: generate device config for red A3400E6BF2EA04B failed: Failed to get red device cert pem File /conf/certificate/internalcerts/red_device_A3400E6BF2EA04B.pem does not exists
Tue Jan 12 11:11:15 2021 REDD ERROR Failed to register on provisioning service: Failed to register RED device on provisioning server: Curl command to register utm on the provisioning server failed with error 1792
Tue Jan 12 11:11:22 2021 REDD ERROR Failed to delete certificates: Deleting RED device cert from filesystem failed: file /conf/certificate/internalcerts/red_device_A3400E6BF2EA04B.pem does not exist
Tue Jan 12 12:16:36 2021 REDD INFO Forcing regeneration of RED certificates
Tue Jan 12 12:16:37 2021 REDD ERROR Failed to create tunnel: Failed: /bin/red2ctl create 3 reds3
Tue Jan 12 12:17:15 2021 REDD INFO Forcing regeneration of RED certificates
Tue Jan 12 14:18:58 2021 REDD ERROR Failed to notify red service: red_server is not running
Tue Jan 12 14:18:58 2021 REDD ERROR Failed to notify red server (enable_device_event)
Wed Jan 13 09:44:41 2021 REDD ERROR Failed to notify red service: red_server is not running
Wed Jan 13 09:44:41 2021 REDD ERROR Failed to notify red server (enable_device_event)

The Log red.log shows some kind of cycle:

Wed Jan 13 19:26:47 2021 REDD ERROR: DIE --------------------------------------------------------
Wed Jan 13 19:26:47 2021 REDD ERROR: Undefined subroutine &Red::Util::Error called at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 141.

Wed Jan 13 19:26:47 2021 REDD ERROR: Trace begun at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 55
Red::Util::print_stacktrace at /bin/red_server.pl line 55
main::__ANON__('Undefined subroutine &Red::Util::Error called at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 141.^J') called at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 141
Red::Util::mask2bits(undef) called at /lib32/perl/site_perl/5.20.1/Red/CSC.pm line 305
Red::CSC::get_split_networks('AUS_VPN_Pool_10.242.6.0\24') called at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 768
Red::Util::generate_device_config('A360188498A5089') called at /bin/red_server.pl line 157

Undefined subroutine &Red::Util::Error called at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 141.
Wed Jan 13 19:26:47 2021 REDD INFO: client: (Re-)loading device configurations
Wed Jan 13 19:26:47 2021 REDD INFO: client: shutdown requested, killing clients
Wed Jan 13 19:26:47 2021 REDD INFO: client: exiting
Wed Jan 13 19:26:51 2021 REDD INFO: server: Using RED firmware in /content/redfw/
Wed Jan 13 19:26:51 2021 REDD INFO: server: RED15(w) fw version set to 10224
Wed Jan 13 19:26:51 2021 REDD INFO: server: RED20 fw version set to 1-1111-c753026a5-fda4803
Wed Jan 13 19:26:51 2021 REDD INFO: server: RED50 fw version set to 10224
Wed Jan 13 19:26:51 2021 REDD INFO: server: RED60 fw version set to 1-1111-c753026a5-fda4803
Wed Jan 13 19:26:51 2021 REDD INFO: Startup server - waiting 5 seconds ...
Wed Jan 13 19:26:51 2021 REDD INFO: Startup client - waiting 5 seconds ...
Wed Jan 13 19:26:56 2021 REDD INFO: server: Configuration uploader process starting
Wed Jan 13 19:26:56 2021 REDD INFO: server: (Re-)loading device configurations
Wed Jan 13 19:27:04 2021 REDD ERROR: WARN -------------------------------------------------------
Wed Jan 13 19:27:04 2021 REDD ERROR: Use of uninitialized value in concatenation (.) or string at /lib32/perl/site_perl/5.20.1/Red/CSC.pm line 305.

Wed Jan 13 19:27:04 2021 REDD ERROR: Trace begun at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 55
Red::Util::print_stacktrace at /bin/red_server.pl line 61
main::__ANON__('Use of uninitialized value in concatenation (.) or string at /lib32/perl/site_perl/5.20.1/Red/CSC.pm line 305.^J') called at /lib32/perl/site_perl/5.20.1/Red/CSC.pm line 305
Red::CSC::get_split_networks('AUS_VPN_Pool_10.242.6.0\24') called at /lib32/perl/site_perl/5.20.1/Red/Util.pm line 768
Red::Util::generate_device_config('A360188498A5089') called at /bin/red_server.pl line 157

Wed Jan 13 19:27:04 2021 REDD ERROR: DIE --------------------------------------------------------

The device itself is in a boot cycle.

Yesterday my colleague did also an Pattern Update:

Before:



One can clearly see that this was 



What is the issue here? 
What needs to be done to get this up and running again?

Any help appreciated ....



This thread was automatically locked due to age.
  • FormerMember
    0 FormerMember

    Hi ,

    Thank you for reaching out to the Community! 

    Can you tell us more about the changes your colleague made that caused this issue? 

    Did you try to delete the offline RED and reconfigure them on the firewall? 

    Thanks,

  • 0 in reply to FormerMember

    It talked to my colleague this morning. He deleted and recreated the RED on the firewall on Tuesday around 12:17. The firewall was running in split mode before and is now configured in unified mode. 


    Tue Jan 12 12:16:37 2021 REDD ERROR Failed to create tunnel: Failed: /bin/red2ctl create 3 reds3
    Tue Jan 12 12:17:15 2021 REDD INFO Forcing regeneration of RED certificates


    Tue Jan 12 14:18:58 2021 REDD ERROR Failed to notify red service: red_server is not running
    Tue Jan 12 14:18:58 2021 REDD ERROR Failed to notify red server (enable_device_event)

    He also send me some screenshots where one can see that the RED tries to connect to the IP Address the firewall is listening to.

    Is there any need to do a reset on the RED in order to reintialize? If yes what needs to be done?

  • FormerMember
    0 FormerMember in reply to BeEf

    Hi ,

    The logs you provided in the first post indicates the issue with the RED certificate. If you delete the RED configuration and then re-configure it on the firewall will create a new RED certificate. 

    Thanks,

  • 0 in reply to FormerMember

    We alraedy did this 2 days ago.

  • 0 in reply to FormerMember

    The current situation is now: We deleted the configured REDs and reconfigured it.

    1) One of them is here at my home and shows the log entries below. On  this RED 15w the first LEDs are on. Only the Tunnel LED is flashing all the time.

    Thu Jan 14 17:59:01 2021 REDD INFO command 'PING 0 uplink=WAN wan_ip=192.168.178 .82'
    Thu Jan 14 17:59:01 2021 REDD INFO A36027938A84AB4/RED 15w Fiest is now re-conne cted after 11000 ms
    Thu Jan 14 17:59:01 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 17:59:01 2021 REDD INFO PONG local_tx=0
    Thu Jan 14 17:59:17 2021 REDD INFO command 'PING 0 uplink=WAN wan_ip=192.168.178 .82'
    Thu Jan 14 17:59:17 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 17:59:17 2021 REDD INFO PONG local_tx=0
    Thu Jan 14 17:59:31 2021 REDD INFO command 'PING 0 uplink=WAN wan_ip=192.168.178 .82'
    Thu Jan 14 17:59:31 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 17:59:31 2021 REDD INFO PONG local_tx=0
    Thu Jan 14 17:59:32 2021 REDD INFO command 'SYSSTATE unstable peer using stabili zation timeout 30'
    Thu Jan 14 17:59:32 2021 REDD INFO command 'CON_CLOSE reason=no_peer'
    Thu Jan 14 18:00:02 2021 REDD INFO No ping for 30 seconds, exiting.
    Thu Jan 14 18:00:02 2021 REDD INFO A36027938A84AB4/RED 15w Fiest is now disconne cted
    Thu Jan 14 18:00:02 2021 REDD INFO device is disconnected.
    Thu Jan 14 18:00:25 2021 REDD INFO server: New connection from 83.135.154.247 (c ipher ECDHE-RSA-AES256-GCM-SHA384), rev1
    Thu Jan 14 18:00:25 2021 REDD INFO connected OK, pushing config
    Thu Jan 14 18:00:31 2021 REDD INFO command 'UMTS_STATUS value=OK'
    Thu Jan 14 18:00:31 2021 REDD INFO command 'PING 0 uplink=WAN wan_ip=192.168.178 .82'
    Thu Jan 14 18:00:31 2021 REDD INFO A36027938A84AB4/RED 15w Fiest is now re-conne cted after 60000 ms
    Thu Jan 14 18:00:31 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 18:00:31 2021 REDD INFO PONG local_tx=0
    Thu Jan 14 18:00:45 2021 REDD INFO command 'PING 0 uplink=WAN wan_ip=192.168.178 .82'
    Thu Jan 14 18:00:45 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 18:00:45 2021 REDD INFO PONG local_tx=0
      

    2) A second firewall (the RED50 above) at my colleagues home shows on the COM Port  


    Thu Jan 14 18:08:12 2021 REDD INFO command 'PORTSTATE 1E04,1004,1004,1004,1004'
    Thu Jan 14 18:08:12 2021 REDD INFO PORTSTATE LAN1: Down,LAN2: Down,LAN3: Down,LAN4: Down
    Thu Jan 14 18:08:12 2021 REDD INFO command 'PING 0 uplink=WAN1 uplinkstate=0 wan1_ip=192.168.0.128'
    Thu Jan 14 18:08:12 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 18:08:12 2021 REDD INFO PONG local_tx=0
    Thu Jan 14 18:08:27 2021 REDD INFO command 'SYSSTATE unstable peer using stabilization timeout 30'
    Thu Jan 14 18:08:27 2021 REDD INFO command 'CON_CLOSE reason=no_peer'
    Thu Jan 14 18:08:28 2021 REDD INFO server: New connection from 84.130.87.219 (cipher ECDHE-RSA-AES256-GCM-SHA384), rev1
    Thu Jan 14 18:08:28 2021 REDD INFO already connected, releasing old connection.
    Thu Jan 14 18:08:28 2021 REDD INFO A3400E6BF2EA04B/RED 50 Pilz is now disconnected
    Thu Jan 14 18:08:29 2021 REDD INFO device is disconnected.
    Thu Jan 14 18:08:30 2021 REDD INFO connected OK, pushing config
    Thu Jan 14 18:08:33 2021 REDD INFO command 'UMTS_STATUS value=OK'
    Thu Jan 14 18:08:33 2021 REDD INFO command 'PORTSTATE 1E04,1004,1004,1004,1004'
    Thu Jan 14 18:08:33 2021 REDD INFO PORTSTATE LAN1: Down,LAN2: Down,LAN3: Down,LAN4: Down
    Thu Jan 14 18:08:33 2021 REDD INFO command 'PING 0 uplink=WAN1 uplinkstate=0 wan1_ip=192.168.0.128'
    Thu Jan 14 18:08:33 2021 REDD INFO A3400E6BF2EA04B/RED 50 Pilz is now re-connected after 21000 ms
    Thu Jan 14 18:08:33 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 18:08:33 2021 REDD INFO PONG local_tx=0
    Thu Jan 14 18:08:48 2021 REDD INFO command 'PORTSTATE 1E04,1004,1004,1004,1004'
    Thu Jan 14 18:08:48 2021 REDD INFO PORTSTATE LAN1: Down,LAN2: Down,LAN3: Down,LAN4: Down
    Thu Jan 14 18:08:48 2021 REDD INFO command 'PING 0 uplink=WAN1 uplinkstate=0 wan1_ip=192.168.0.128'
    Thu Jan 14 18:08:48 2021 REDD INFO PING remote_tx=0 local_rx=0 diff=0
    Thu Jan 14 18:08:48 2021 REDD INFO PONG local_tx=0
    Thu Jan 14 18:09:03 2021 REDD INFO A3400E6BF2EA04B/RED 50 Pilz transfered bytes TX: 5664 RX: 4080
    Thu Jan 14 18:09:04 2021 REDD INFO command 'SYSSTATE unstable peer using stabilization timeout 30'
    Thu Jan 14 18:09:04 2021 REDD INFO command 'CON_CLOSE reason=no_peer'
    Thu Jan 14 18:09:19 2021 REDD INFO No ping for 30 seconds, exiting.
    Thu Jan 14 18:09:19 2021 REDD INFO A3400E6BF2EA04B/RED 50 Pilz is now disconnected
    Thu Jan 14 18:09:19 2021 REDD INFO device is disconnected.


Unfiltered HTML