This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

XG430 10G Flexi Port link not coming up - replacement of the module

During HA-Rebuild and Upgrade to 19.5.1 I noticed a log in System logs: "PortA4 up" on the peer node at a time when it was unexpected.

I then noticed on the physical interface, that the LED of that Port were off while the other 3 were blinking.

It is 10G Copper with SFP Plus Cable.

I replaced the cable with the same result. After 15 or more minutes I checked the device again and the port was up and running - as can be seen from the below output and screenshots.

I checked the Troubleshooting tests for network issues, but I found no log information or debug procedere else than this one: https://support.sophos.com/support/s/article/KB-000036345?language=en_US

Are there useful logs to analyze this?

This was the port status when it was off:

PortA3    Link encap:Ethernet  HWaddr C8:4F:86:FC:00:0D  
          inet6 addr: fe80::ca4f:86ff:fefc:d/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:772884 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1243396 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:447511125 (426.7 MiB)  TX bytes:782974630 (746.7 MiB)

PortA4    Link encap:Ethernet  HWaddr C8:4F:86:FC:00:0D  
          inet6 addr: fe80::ca4f:86ff:fefc:e/64 Scope:Link
          UP BROADCAST SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

and than when it was on.

--

XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ifconfig PortA4
PortA4    Link encap:Ethernet  HWaddr C8:4F:86:FC:00:0D
          inet6 addr: fe80::ca4f:86ff:fefc:e/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:7259882 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8158434 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:5353399207 (4.9 GiB)  TX bytes:7343072503 (6.8 GiB)

XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ifconfig PortA3
PortA3    Link encap:Ethernet  HWaddr C8:4F:86:FC:00:0D
          inet6 addr: fe80::ca4f:86ff:fefc:d/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:9608423 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6768176 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7102495310 (6.6 GiB)  TX bytes:1499533574 (1.3 GiB)

The 4 Ports are in a LACP LAG.

That is the problematic port:
XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ethtool PortA4
Settings for PortA4:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x0000000f (15)
                               drv probe link timer
        Link detected: yes

One other Port:

XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ethtool PortA3
Settings for PortA3:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x0000000f (15)
                               drv probe link timer
        Link detected: yes

--



This thread was automatically locked due to age.
Parents
  • The ethtool output: was this while it was already online again? 

    BTW is your xmit Hash policy correct? Likely it does not match with your switch. 

    __________________________________________________________________________________________________________________

  • yes. unfortunately I did not collect it while it was down.

    The ethtool output: was this while it was already online again? 

    will check about xmit policy on switch side.

  • it would be OK from switch-side. But that is only loadbalancing - should have nothing to do with the port issue.

    Can you please tell if this is still true?

    NC-94073
    • SFOS 19.0.0 GA-Build317 (19.0.0.317) [Tupai]
    • NoRelease
    • XGS BSP
    XGS 10G interface not working when interface speed is set to Auto-negotiation (Physical or LAG)

    Issue: XGS 10G interface is not working when interface speed is set to Auto-negotiation (Physical or LAG)

    Affected Product: Only XGS hardware with 10G interface

    Set interface speed to Manual 10000 Mbps - Full-Duplex (Applicable for Physical, LAG interfaces).

  • Yes it is still true, but with a firmware update the Link state was fixed to 10 gbe, as you can see above. 

    __________________________________________________________________________________________________________________

  • Why does it autoneg when the interface is set to 10g full? Is that a correct behaviour?

    did a ethtool -r PortA4 and that causes the port to go down for a while.

    G430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool -M PortA4
    Supported interface speed: Supports auto-negotiation:   No
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool -r PortA4
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool -M PortA4
    Supported interface speed:                              1000fd -> 1000baseT/Full


    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA4
    Settings for PortA4:
            Supported ports: [ ]
            Supported link modes:   1000baseT/Full
                                    1000baseKX/Full
                                    10000baseT/Full
                                    1000baseX/Full
                                    10000baseSR/Full
                                    10000baseLR/Full
            Supported pause frame use: Symmetric
            Supports auto-negotiation: Yes
            Supported FEC modes: Not reported
            Advertised link modes:  1000baseT/Full
                                    1000baseKX/Full
                                    10000baseT/Full
                                    1000baseX/Full
                                    10000baseSR/Full
                                    10000baseLR/Full
            Advertised pause frame use: No
            Advertised auto-negotiation: Yes
            Advertised FEC modes: Not reported
            Speed: Unknown!
            Duplex: Unknown! (255)
            Port: Other
            PHYAD: 0
            Transceiver: internal
            Auto-negotiation: off
            Supports Wake-on: g
            Wake-on: g
            Current message level: 0x0000000f (15)
                                   drv probe link timer
            Link detected: no

    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA1 | grep uto-neg
            Supports auto-negotiation: No
            Advertised auto-negotiation: No
            Auto-negotiation: off
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA2 | grep uto-neg
            Supports auto-negotiation: No
            Advertised auto-negotiation: No
            Auto-negotiation: off
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA3 | grep uto-neg
            Supports auto-negotiation: No
            Advertised auto-negotiation: No
            Auto-negotiation: off
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA4 | grep uto-neg
            Supports auto-negotiation: Yes
            Advertised auto-negotiation: Yes
            Auto-negotiation: off

    the switching team reported the port is set to auto neg on switch side. as this is a bad setting in this situation they'll try to change it. looks like the port group needs to be rebuild for that as the port speed cannot be changed in a port group.

  • So the colleagues have changed the port group on the switch to use 10G full static and still this one flexi Port out of four is doing some auto negotiate in  opposite to the adapter setting: auto neg: off

    Can we convince PortA4 to do no auto negotiation?


    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA4 | grep -i auto-neg
            Supports auto-negotiation: Yes
            Advertised auto-negotiation: Yes
            Auto-negotiation: off

  • I am mixing something up: The bug only involved the XGS Flexiport Module of 1U Appliances. You are not affected by the bug. The Bug was due the firmware of the Flexi Port itself, which is not applicable to your scenario, as you have an other Flexi Port module. 

    __________________________________________________________________________________________________________________

  • our XG430 is a 1U machine.

    ethtool -i PortA4
    driver: i40e
    version: 2.10.19.82
    firmware-version: 5.05 0x800028ac 0.0.0
    expansion-rom-version:
    bus-info: 0000:02:00.3
    supports-statistics: yes
    supports-test: yes
    supports-eeprom-access: yes
    supports-register-dump: yes
    supports-priv-flags: yes

    do we have other tools to check the firmware version you're pointing at?

    With v18 we started having issues with lag members in auto speed mode that we did not have before. I mentioned in the link to my old post above. That's when we changed it for the firewall side but forgot to change the switches to match that setting.

    Or do you suggest opening a case? If not a software bug, I suspect something faulty on this port.

    btpw: that is the status now after some minutes in faulty state:

    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA4
    Settings for PortA4:
            Supported ports: [ FIBRE ]
            Supported link modes:   10000baseT/Full
            Supported pause frame use: Symmetric
            Supports auto-negotiation: No
            Supported FEC modes: Not reported
            Advertised link modes:  10000baseT/Full
            Advertised pause frame use: No
            Advertised auto-negotiation: No
            Advertised FEC modes: Not reported
            Speed: 10000Mb/s
            Duplex: Full
            Port: Direct Attach Copper
            PHYAD: 0
            Transceiver: internal
            Auto-negotiation: off
            Supports Wake-on: g
            Wake-on: g
            Current message level: 0x0000000f (15)
                                   drv probe link timer
            Link detected: yes

Reply
  • our XG430 is a 1U machine.

    ethtool -i PortA4
    driver: i40e
    version: 2.10.19.82
    firmware-version: 5.05 0x800028ac 0.0.0
    expansion-rom-version:
    bus-info: 0000:02:00.3
    supports-statistics: yes
    supports-test: yes
    supports-eeprom-access: yes
    supports-register-dump: yes
    supports-priv-flags: yes

    do we have other tools to check the firmware version you're pointing at?

    With v18 we started having issues with lag members in auto speed mode that we did not have before. I mentioned in the link to my old post above. That's when we changed it for the firewall side but forgot to change the switches to match that setting.

    Or do you suggest opening a case? If not a software bug, I suspect something faulty on this port.

    btpw: that is the status now after some minutes in faulty state:

    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA4
    Settings for PortA4:
            Supported ports: [ FIBRE ]
            Supported link modes:   10000baseT/Full
            Supported pause frame use: Symmetric
            Supports auto-negotiation: No
            Supported FEC modes: Not reported
            Advertised link modes:  10000baseT/Full
            Advertised pause frame use: No
            Advertised auto-negotiation: No
            Advertised FEC modes: Not reported
            Speed: 10000Mb/s
            Duplex: Full
            Port: Direct Attach Copper
            PHYAD: 0
            Transceiver: internal
            Auto-negotiation: off
            Supports Wake-on: g
            Wake-on: g
            Current message level: 0x0000000f (15)
                                   drv probe link timer
            Link detected: yes

Children
  • Again: It is only affected the particular SKU, which only the XGS 1U supports. 

    This particular module has a firmware bug, which does not support Auto-Neg. 

    You are not effected by this problem, as you cannot have the affected hardware in the first place. 

    __________________________________________________________________________________________________________________

  • ok - I did not notice the S in you writing XGS. Thanks for your help so far!

  • The question is likely: Is only one of both appliances affected? If yes, likely the hardware is defect (port on XG, cable or Port on Switch). 
    If both appliances have the same problem with PortA4, it could be a software problem. 

    The KIL you mentioned describe the XGS problem as well. 

    __________________________________________________________________________________________________________________

  • used a different switch port.

    Still the interface autonegotiating a long time. will try on the other node.

    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool lag0
    Settings for lag0:
            Supported ports: [ ]
            Supported link modes:   Not reported
            Supported pause frame use: No
            Supports auto-negotiation: No
            Supported FEC modes: Not reported
            Advertised link modes:  Not reported
            Advertised pause frame use: No
            Advertised auto-negotiation: No
            Advertised FEC modes: Not reported
            Speed: 30000Mb/s
            Duplex: Full
            Port: Other
            PHYAD: 0
            Transceiver: internal
            Auto-negotiation: off
            Link detected: yes
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Auxiliary# ethtool PortA4
    Settings for PortA4:
            Supported ports: [ ]
            Supported link modes:   1000baseT/Full
                                    1000baseKX/Full
                                    10000baseT/Full
                                    1000baseX/Full
                                    10000baseSR/Full
                                    10000baseLR/Full
            Supported pause frame use: Symmetric
            Supports auto-negotiation: Yes
            Supported FEC modes: Not reported
            Advertised link modes:  1000baseT/Full
                                    1000baseKX/Full
                                    10000baseT/Full
                                    1000baseX/Full
                                    10000baseSR/Full
                                    10000baseLR/Full
            Advertised pause frame use: No
            Advertised auto-negotiation: Yes
            Advertised FEC modes: Not reported
            Speed: Unknown!
            Duplex: Unknown! (255)
            Port: Other
            PHYAD: 0
            Transceiver: internal
            Auto-negotiation: off
            Supports Wake-on: g
            Wake-on: g
            Current message level: 0x0000000f (15)
                                   drv probe link timer
            Link detected: no

    console> show network interface-link lag0
    lag0: 10000Mb/s  duplex: Full  link: yes  auto-negotiation: No fecmode: Off

    console> show network interfaces

    ...
    PortA1           Zonetype:LAN  MAC Address:7C:5A:1C:5B:15:F2  MTU:1500
                     Speed:10000Mb/s Full Duplex
                     UP BROADCAST RUNNING SLAVE MULTICAST
                     RX State: packets:259419 bytes:27036459 (25.7 MiB)
                               errors:0 dropped:0 overruns:0 frame:0
                     TX State: packets:8281 bytes:1026780 (1002.7 KiB)
                               errors:0 dropped:0 overruns:0 carrier:0


    PortA2           Zonetype:LAN  MAC Address:7C:5A:1C:5B:15:F2  MTU:1500
                     Speed:10000Mb/s Full Duplex
                     UP BROADCAST RUNNING SLAVE MULTICAST
                     RX State: packets:781504 bytes:77017349 (73.4 MiB)
                               errors:0 dropped:0 overruns:0 frame:0
                     TX State: packets:8283 bytes:1027040 (1002.9 KiB)
                               errors:0 dropped:0 overruns:0 carrier:0


    PortA3           Zonetype:LAN  MAC Address:7C:5A:1C:5B:15:F2  MTU:1500
                     Speed:10000Mb/s Full Duplex
                     UP BROADCAST RUNNING SLAVE MULTICAST
                     RX State: packets:1185291 bytes:81866988 (78.0 MiB)
                               errors:0 dropped:0 overruns:0 frame:0
                     TX State: packets:8398 bytes:1039658 (1015.2 KiB)
                               errors:0 dropped:0 overruns:0 carrier:0


    PortA4           Zonetype:LAN  MAC Address:7C:5A:1C:5B:15:F2  MTU:1500
                     Speed:-1Mb/s Full Duplex
                     UP BROADCAST SLAVE MULTICAST
                     RX State: packets:269692 bytes:23651709 (22.5 MiB)
                               errors:0 dropped:0 overruns:0 frame:0
                     TX State: packets:7678 bytes:952072 (929.7 KiB)
                               errors:0 dropped:0 overruns:0 carrier:0

    ...

  • We could not get PortA4 online without issues on one allied telesis x950 switch on it's integrated SFP Ports while PortA3 is attached to the same switch but on an other integrated SFP Port. Tested 5 different cables and works fine.

    We got it now working on an other x950 switch in the same stack but there on a SFP Plug-in Module. It worked immediately there.

    that switch model did not work with Port 4

    works with SFP addon module

    Other Question:

    Is it possible to know what the blue LED should show? Usually it is "speed".

    Green ports are blinking, so I did not catch them all on at the same time on the foto.

    All ports are at 10G FDX. Should'nt all Ports have the blue LED lit on?

    ethtool PortA1 |grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s
    ethtool PortA2 |grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s
     ethtool PortA3 |grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s
    ethtool PortA4 |grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s

  • German docs: 

    __________________________________________________________________________________________________________________

  • Thank you. We'll discuss internally if we should open a RMA case.

  • the other HA node (primary) - that is running at a different location also has this issue with the blue LED.

    3 out of 4 are lit on while on the auxiliary node are only 2 out of 4 lit on.

    Primary: here PortA2 had it off while connected and running at full speed.

    The following output is equal to the other node.

    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ethtool PortA1 | grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ethtool PortA2 | grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ethtool PortA3 | grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s
    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ethtool PortA4 | grep "10000"
            Supported link modes:   10000baseT/Full
            Advertised link modes:  10000baseT/Full
            Speed: 10000Mb/s

    XG430_WP02_SFOS 19.5.1 MR-1-Build278 HA-Primary# ethtool -i PortA2
    driver: i40e
    version: 2.10.19.82
    firmware-version: 5.05 0x800028ac 0.0.0
    expansion-rom-version:
    bus-info: 0000:02:00.1
    supports-statistics: yes
    supports-test: yes
    supports-eeprom-access: yes
    supports-register-dump: yes
    supports-priv-flags: yes

  • Hey  ,

    Do you require a RMA for Sophos 4 port 10GbE SFP+ FleXi Port module right ?
    The one which is mentioned here - https://www.connection.com/product/sophos-4-port-10gbe-sfp-flexi-port-module/sgiztchf4/41043540 | If yes could you please double check the serial no by Sophos mentioned in here - https://support.sophos.com/support/s/article/KB-000037169?language=en_US 

    Thanks & Regards,
    _______________________________________________________________

    Vivek Jagad | Team Lead, Global Support & Services 

    Log a Support Case | Sophos Service Guide
    Best Practices – Support Case


    Sophos Community | Product Documentation | Sophos Techvids | SMS
    If a post solves your question please use the 'Verify Answer' button.

  • Yes, Vivek, they are SGIZTCHF4

    I have posted a picture of the shipping boxes with SN to the case.