Open IPv6 Issues / questions

- will the fix for issue NUTM-7187 be included with 9.5?

- is there a fix in the works for IPv6 Connections where the WAN Port is supposed to use an address out of the delegated prefix? Currently users of such ISPs do not get any IPv6 address. (for esxample KPN netherlands)

- what about the ability to change/edit the UID for IPv6 Delegation Requests?

- what about long standing feature requests such as 6tunnel integration, lets encrypt - is that on the roadmap? Users, myself included had high hopes for 9.5 but this seems to be more than a maintance release.

 

thank you in advance.

  • In reply to SanderRutten:

    I have not seen the "Installing default route" in my logs.

    Attached are my PPPoE and IPv6 log files.

    logfiles_20170420085819.zip

  • In reply to rklomp:

    Checked my logs as well, and they are quite the same.

    So.. I decided to start over. No factory reset yet, just config-wise.
    Removed all my IPv6 settings on my interfaces, deleted Prefix for my LAN, disabled IPv6.
    Re-applied the patch (I know, shouldn't matter), restarted the watchdog service, and reconnected the PPPoE connection.

    Gave my LAN a static address 2001:981:9D6E:1::1
    Re-created the Prefix Advertisement for my LAN.
    Enabled the "IPv6 Default GW" on my WAN. (2017:04:20-12:10:38 router ipv6_watchdog[10859]: Installing default route via fe80::2a0:a50f:fc78:5530 for interface ppp0(ifidx 19))
    WAN interface shows: DEFAULT GW 194.109.5.175 | ::
    After reconnecting the WAN: DEFAULT GW 194.109.5.175 | fe80::2a0:a50f:fc78:5530

    No working connection :(
    So, why not try a reboot, solves everything right? Appearantly it does :O After a reboot I could imediately ping and surf to IPv6 hosts.

    I saw something new in the ipv6.log: Started dhclient6 -P
    Let's hope it keeps working now!
    This is also in René's logfile, but I didn't see it in mine before. Attached are my ipv6.log and pppoe.log. Everything before 12:15 is before the reboot, and after 12.15 is after the reboot.

    There is one thing I already noticed, my clients are using my LAN's address 2001:981:9D6E:1::1 as outgoing address. Is this expected behavior? Security wise? Or should a client always use their temporary privacy extension address? 

    Logs.zip

  • In reply to SanderRutten:

    I tried rebooting, disabling IPv6 etc. but no matter what I try I am not seeing the following lines in my log:

    2017:04:20-12:18:18 router ipv6_watchdog[4332]: Installing default route via fe80::2a0:a50f:fc78:5530 for interface ppp0(ifidx 10)
    2017:04:20-12:18:18 router ipv6_watchdog[4332]: JSON "ppp0", "unknown", "up", { "gateway6": "fe80::2a0:a50f:fc78:5530", "network6":"1"}
    2017:04:20-12:18:18 router ipv6_watchdog[4332]: RA flags changed for interface ppp0(ifidx 10): SENT,READY -> SENT,RCVD,READY

    Somehow the creation of the default route does not work for me.

    If you do a reconnect, will it keep working? 

  • In reply to rklomp:

    You made me break my IPv6 :( :+
    It was working untill I reconnected my PPPoE. A reboot didn't solve it this time, and right now deleting my IPv6 settings and re-applying them didn't solve it either. 
    So.. eh, weird. Maybe more time this weekend.

  • In reply to SanderRutten:

    i can only advice to record connects with tcpdump for later wireshark dissection, was able to find problems really fast through that. 

    say your pppoe interface is eth1, 

    # tcpdump -i eth1 pppoes and ip6 -v -w /home/login/eth1-pppoes-datestamp.pcap

    i did that with all patches and before so i could see the changes.

     

    edit: fixed a typo in the tcpdump, its "ip6" not "ipv6"

  • In reply to SanderRutten:

    Somehow it also a relief that you can reproduce my problem ;)

    If you just add the default route manually it should work for now:
    # route add -A inet6 default gw <remote LL> dev ppp0

     

    Regarding the issue that everything was using 2001:981:9D6E:1::1. Are you doing some NAT translation?

  • In reply to rklomp:

    Hi All,

       My name is Duc Le. I am a new member of NSG.

       I've been working with Prakash on this issue for a bit of time before his vacation.

       Since I am a newbie on this issue, I just wonder if you (rklomp and SanderRutten)

       can do the followings (A):

       1) Running the system (UTM with Prakash patch) as vanilla as possible. This is to reduce noise.

           Only concentrate on IPv6 address problem. (this is only a reminder, please ignore if you already. Thanks)

       2) Capture tcpdump

       3) Capture UTM log files (ipv6 and pppoe) and other logs if you think they are needed

       4) Capture system log (dmesg)

       5) Capture these: ifconfig -a, route table info, ps -elaf

       6) Misc Info as you think needed

     

        I suspected there might be some sort of race condition as to sometimes you see "Installing default route"

        and sometimes not.

     

       That's all for now. Please let me know your comments, ideas or questions. Thanks.

     

    Regards

    Le

    P.S. I prefer to be called Le. Thanks.

  • In reply to Duc Le:

    Hi Ben, rklomp & SanderRutten,

     

    Thank you for your continued efforts to help us troubleshoot & test this issue.

    Le is another member on our team, and he'll be taking over this issue from Prakash since Prakash went on holiday earlier this week. Prakash handed over the issue before he left, but please forgive us if Le asks for some redundant information as he gets up to speed on the issue.

    If I followed the thread correctly, I think Ben's issues are completely resolved with Prakash's latest patch, but rklomp & SanderRutten still has the issue of missing default route?

    Thanks again for all your time & effort in helping us improve the product!

  • In reply to bobbylam:

    Hi Bobby,

    Thank you for taking the time to troubleshoot and help on this issue so quickly!
    You are correct, for SanderRutten and me the only issue left is the missing default route.

     I am away this weekend, but will try to make some more captures next week. See my previous post for pppoe and ipv6 logs.

    Thanks for the efforts!

    Regards,

    René

  • In reply to Duc Le:

    Hello Le,

    I'm writing down every steps I did, just to make sure I didn't do something wrong. This will result in a long text, sorry :)

    • Factory reset 9.5 beta (9.470-14)
    • Performed the basic system setup
      • Changed LAN to 10.0.0.0/16, enabled DHCP
      • WAN setup, configured as DSL PPPoE
    • Created 1 firewall rule: Any source using any protocol to any destination. Keep it simple for now :P
    • Enabled SSH
      • Uploaded ep-ipv6-watchdog-9.40-4.gce64053.i686.rpm, and installed it (rpm -Uhv --force ep-ipv6-watchdog-9.40-4.gce64053.i686.rpm)
      • Restarted watchdog: /var/mdw/scripts/ipv6_watchdog restart
      • Reconnect PPPoE interface

    Wanted to run the tcpdump before enabling IPv6, the command Ben posted gave a syntax error. Appearantly the filter option wasn't ipv6, but ip6.
    tcpdump -i eth1 pppoes and ip6 -v -w /home/login/eth1-pppoes-datestamp.pcap

    • Interfaces & Routing
      • IPv6: Enabled IPv6.
    • Interfaces & Routing > Interfaces: Gave my LAN interface the address 2001:981:9D6E:1::1/64
    • Interfaces & Routing > IPv6: Created a LAN IPv6 Prefix Advertisement.
    • Interfaces & Routing > Interfaces: Enabled the option "IPv6 Default GW" for the WAN interface.
      Interface status now shows:
      External (WAN) on eth1 [80.100.129.64/32]
      MTU 1492 · DEFAULT GW 194.109.5.175 | ::

    Seems that there is no IPv6 gateway, but the ipv6.log shows:
    Installing default route via fe80::2a0:a50f:fc78:5530 for interface ppp0(ifidx 13)

    router:/home/login # ip -6 route
    2001:981:9d6e:1::/64 dev eth0 proto kernel metric 256
    fe80::/64 dev eth1 proto kernel metric 256
    fe80::/64 dev eth0 proto kernel metric 256
    fe80::/10 dev ppp0 metric 1
    fe80::/10 dev ppp0 proto kernel metric 256
    default via fe80::2a0:a50f:fc78:5530 dev ppp0 proto ra metric 1024 expires 1727sec hoplimit 64

    I'm not able to ping google.com from client or UTM over IPv6.

    • Reconnected my PPPoE connection

    The interface status now shows:
    External (WAN) on eth1 [80.100.129.64/32]
    MTU 1492 · DEFAULT GW 194.109.5.175 | fe80::2a0:a50f:fc78:5530

    Although it now shows my linklocal address, I'm not able to ping. At this point I stopped tcpdump and rebooted the UTM.
    Started tcpdump again (eth1-pppoes-datestamp2.pcap), after the reboot.

    router:/home/login # ip -6 route
    2001:981:9d6e:1::/64 dev eth0 proto kernel metric 256
    fe80::/64 dev eth0 proto kernel metric 256
    fe80::/64 dev eth1 proto kernel metric 256
    fe80::/10 dev ppp0 metric 1
    fe80::/10 dev ppp0 proto kernel metric 256
    default via fe80::2a0:a50f:fc78:5530 dev ppp0 proto ra metric 1024 expires 1559sec hoplimit 64

    Still looks the same as before. But just like a previous attempt, since the reboot I can connect to external IPv6 hosts. (As mentioned before: My clients are using the LAN's IPv6 address 2001:981:9D6E:1::1 and not their own address)

    Interface & Routing > IPv6: The connectivity shows:
    Native over External (WAN)
    Delegated Prefix: 2001:981:9d6e::/48

    I don't believe I have seen this before. Of course I forgot to check earlier.

    Time to break stuff. Started tcpdump again (eth1-pppoes-datestamp3.pcap)
    Reconnected my PPPoE interface, and ip -6 route output is the same as above. But I cannot ping anymore.
    Rebooted the UTM again, and IPv6 is still broken. And so far I have not been able to get IPv6 up and running again since I have no clue what triggers something to get it working :)

    Went back to Interface & Routing > IPv6: The connectivity still shows: Delegated Prefix: 2001:981:9d6e::/48

    Summary:
    IPv6 assigned by ISP: 2001:981:9D6E::/48
    LAN prefix: 2001:981:9D6E:1::/64
    Remote linklocal address fe80::2a0:a50f:fc78:5530 (Genexis fiber modem)

    Attached are the tcpdump files and logfiles (pppoe.log, ipv6.log, boot.log, kernel.log, system.log).

    Logfiles:
    eth1-pppoes-datestamp.pcap = Fresh enabled IPv6, but not working
    eth1-pppoes-datestamp2.pcap = IPv6 Working, capture started after a reboot
    eth1-pppoes-datestamp3.pcap = IPv6 stopped working after reconnecting PPPoE

    ifconfig.txt: Output of ifconfig -a. Included it only once, there was no difference (Except packets) during working or broken IPv6.
    ps.txt: Output of ps -elaf when IPv6 was working.
    ps_broken_ipv6.txt: Output after reconnecting PPPoE.

    1781.Logs.zip

  • In reply to bobbylam:

    Hi Bobby,

    my problem is completly solved, thank you again for the swift responses of you and your team :) 

    there are some improvement that could be made to ipv6, lots of great suggestions on the feature request section, as i do not know how much effort you can currently put in please allow me to just name a few

    - Sophos RED Server IPv6 to IPv6

    - "6tunnel" integration to translate incoming WAN IPv6 to IPv4 internally (For the WAF for example)

    - NatPDv6 (translate IPv6 subnets to internal IPv6 subnets via NAT)

    - adjust the licence count on IPv6 Adresses. Most OSs pull more than one single IPv6 and use up the 50 IPs addresses really fast, it makes it REALLY hard to test IPv6 or use on a day to day basis without having licence alerts all the time ;)

     

    final big question: will the XG gain from all these improvements also? Last time i tried iPv6 was completly broken on the XG in this scenario.

  • In reply to SanderRutten:

    I think the last line in your route table shows your default route:

    default via fe80::2a0:a50f:fc78:5530 dev ppp0 proto ra metric 1024 expires 1559sec hoplimit 64

    So it seems it should be working for you.

  • In reply to rklomp:

    There is the magic word: Should :) It should indeed work. Except it isn't :( Or at least not always, it breaks very easily. 
    If I get it working again I will keep my hands of the settings for a while, and see if it keeps working. Or maybe it breaks anyway and has the reconnect/reboot no influence at all.

  • In reply to SanderRutten:

    @SanderRutten: Thanks so much for your help and log files. I am looking at it and will update ASAP. Thanks.

    @Ben: Thanks

  • In reply to Duc Le:

    Quick summary of findings in the SanderRutten latest logs:

     

    1) The default route over the ppp0/eth1 is there in the route table. So it is OK for now.

    My view: As IPv6 allows multiple default gateways. This is OK even though there is no RA.

    TBD: Keep an eye if the default route for ppp0 is missing. This is a no-no.

     

    2) Ping from 2001:981:9d6e:1::1 to 2a00:1450:400e:801::200e (I assumed this is the ping that you are concerned about):

    i) In the very first capture, packet# 304 failed to get ping-reply   // This is changed configuration

    ii) In the second capture (datestamp2), the packet#620 and packet #621 succeeds   // This is reboot

    iii) In the third capture. packet#189 failed to get a reply // After reconnect PPPoE

    My view: This looks quite strange since most of the stuff (system/UTM) looked OK

    To Do: See (3)

     

    3) Also, I saw in the log the following message (system log): dhclient6: send_packet6: Operation not permitted

    My view: more than likely it is iptables rule. This might impact the (2) above as well

    To Do: SanderRutten

    a) Please dump out the iptables when it is working and when it is not working and include them in this thread.

    Sorry that I forgot about the iptables stuff. I made the mistake of making the assumption that it is OK. I just want to make sure this time

    b) When ping fails, can you check the stattisctis of sending and receiving on the eth1 to see they are going up

     

    4) Rebind issue 

    My view: to be deal with once 2 is out of the way

     

    5) The ps look fines

     

    Thanks a lot SanderRutten for your help and sorry for my mistake of overlooking the iptables stuff.

    Also please let me know your ideas, comments or questions. Thanks again.