Hurricane Electric's IPv6 Tunnel Broker Forums

General IPv6 Topics => IPv6 on Routing Platforms => Topic started by: growse on January 31, 2010, 01:20:49 PM

Title: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on January 31, 2010, 01:20:49 PM
I've got a tunnel set up on my Cisco 877 and the tunnel seems to work great. However, the stability of the connectivity of autoconfigured clients (laptops) behind the router seems to vary quite significantly. Connections to certain sites work, whereas other sites don't (most of the time). For example:

ping6 ipv6.google.com
PING ipv6.google.com(2a00:1450:8001::67) 56 data bytes
*times out*


Sometimes the above works, and I get a response.

whereas...


ing6 www.kame.net
PING www.kame.net(orange.kame.net) 56 data bytes
64 bytes from orange.kame.net: icmp_seq=1 ttl=52 time=282 ms
64 bytes from orange.kame.net: icmp_seq=2 ttl=52 time=294 ms


seems to mostly work all the time.

Pinging the above hosts work from both my router and static-addressed hosts. The router can ping static-addressed internal hosts but not autoconfigured hosts. No internal hosts can ping the router on the assigned address in the subnet, but everything can ping the router's end of the tunnel address.

Here's the interfaces in the router config:


interface Vlan1
description $ETH-SW-LAUNCH$$INTF-INFO-HWIC 4ESW$$ES_LAN$$FW_INSIDE$
ip address 192.168.0.1 255.255.255.0
no ip redirects
no ip unreachables
no ip proxy-arp
ip flow ingress
ip nat inside
ip virtual-reassembly
ip tcp adjust-mss 1452
ipv6 address 2001:470:1F09:784::1/64
ipv6 nd prefix 2001:470:1F09:784::/64
ipv6 nd ra lifetime 180
ipv6 nd ra interval 60
hold-queue 100 out
end
!
interface Tunnel0
description Hurricane Electric IPv6 Tunnel Broker
no ip address
ipv6 address 2001:470:1F08:784::2/64
ipv6 enable
tunnel source 93.97.176.164
tunnel destination 216.66.80.26
tunnel mode ipv6ip
end


What on earth is going on?
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: cholzhauer on January 31, 2010, 01:26:37 PM
Hmm...what does your routing table look like on the router?  What about one of the RA-enabled hosts?
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on February 01, 2010, 09:32:33 AM
Right now, the ping connectivity to ipv6.google.com is there. What's never there is ping connectivity to the router.

Ra-host:


Kernel IPv6 routing table
Destination                    Next Hop                   Flag Met Ref Use If
2001:470:1f09:784::/64         ::                         UAe  256 0 10675 wlan0
2a01:348:18a::/64              ::                         UAe  256 0  1629 wlan0
fe80::/64                      ::                         U    256 0     0 wlan0
::/0                           fe80::223:4ff:fe11:98bd    UGDAe 1024 1 30980 wlan0
::/0                           ::                         !n   -1  1 49065 lo
::1/128                        ::                         Un   0   3    36 lo
2001:470:1f09:784:221:5cff:fe08:7291/128 ::                         Un   0   1 76795 lo
fe80::221:5cff:fe08:7291/128   ::                         Un   0   1   976 lo
ff00::/8                       ::                         U    256 0     0 wlan0
::/0                           ::                         !n   -1  1 49065 lo


Router


IPv6 Routing Table - Default - 6 entries
Codes: C - Connected, L - Local, S - Static, U - Per-user Static route
       B - BGP, M - MIPv6, R - RIP, D - EIGRP
       EX - EIGRP external
       O - OSPF Intra, OI - OSPF Inter, OE1 - OSPF ext 1, OE2 - OSPF ext 2
       ON1 - OSPF NSSA ext 1, ON2 - OSPF NSSA ext 2
S   ::/0 [1/0]
     via Tunnel0, directly connected
C   2001:470:1F08:784::/64 [0/0]
     via Tunnel0, directly connected
L   2001:470:1F08:784::2/128 [0/0]
     via Tunnel0, receive
C   2001:470:1F09:784::/64 [0/0]
     via Vlan1, directly connected
L   2001:470:1F09:784::1/128 [0/0]
     via Vlan1, receive
L   FF00::/8 [0/0]
     via Null0, receive
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on February 01, 2010, 02:05:49 PM
I have a Cisco 871W with almost the same problem, sometimes I cannot connect to a IPv6 host and a few minutes later I can connect without any problem. The strange thing is that I have a Windows Vista laptop and a Ubuntu 9.10 laptop, I only have these problems on the Vista laptop.

I'm using c870-advipservicesk9-mz.150-1.M.bin on the router (a version from 01/Oct/2009).
I already turned of the Windows Vista firewall, turned off the virus scanner and re-installed the IPv6 stack. Non seems to work :(
For now I have disabled IPv6 on the Vista laptop because I didn't want to wast any more time on the problem.

Can you tell what Cisco IOS version, DNS addresses (I only use the HE DNS, no secondary), POP location (mine is Amsterdam) and operating systems you are using?
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: cholzhauer on February 01, 2010, 06:15:13 PM
What's the output of "ipconfig /all" on your windows machine?
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on February 01, 2010, 06:51:53 PM
Try doing some packet captures with Wireshark (http://www.wireshark.org/) or whatever when this problem is happening to get an idea what's failing.  
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on February 02, 2010, 02:09:59 AM
I've got a static-addressed windows machine as well as an autoconfigured one. I'll grab the routing table and ipconfig output from the windows autoconfigured one later tonight.

I've been on IOS 12.4 for a long time and had ipv6 work perfectly. When I upgraded to IOS 15 I started to notice problems, but have since downgraded back to 12.4 and still have them, so I think the IOS version is coincidental.

I started to experience this back when I tunnelled via Sixxs - had the same issue there, so don't think it's a PoP / infrastructure problem.

I'll try and grab some packet captures later tonight as well. I know that the ping connectivity to the router never works, so I'll grab some captures of a ping from an autoconfigured linux host to the router.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on February 02, 2010, 10:18:17 AM
Right, on my static-allocated windows box which can currently not ping either the router or google.



Pinging ipv6.l.google.com [2a00:1450:8001::6a] with 32 bytes of data:
Destination host unreachable.

Ping statistics for 2a00:1450:8001::6a:
    Packets: Sent = 1, Received = 0, Lost = 1 (100% loss),
Control-C
^C
C:\Users\andrew>route print
===========================================================================
Interface List
34...00 22 15 79 d2 1c ......TEAM: Goteam
23...00 50 56 c0 00 01 ......VMware Virtual Ethernet Adapter for VMnet1
24...00 50 56 c0 00 08 ......VMware Virtual Ethernet Adapter for VMnet8
  1...........................Software Loopback Interface 1
32...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
13...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
14...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #3
15...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #4
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.19    261
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
      169.254.0.0      255.255.0.0         On-link     192.168.137.1    296
  169.254.255.255  255.255.255.255         On-link     192.168.137.1    276
      192.168.0.0    255.255.255.0         On-link      192.168.0.19    261
     192.168.0.19  255.255.255.255         On-link      192.168.0.19    261
    192.168.0.255  255.255.255.255         On-link      192.168.0.19    261
     192.168.65.0    255.255.255.0         On-link      192.168.65.1    276
     192.168.65.1  255.255.255.255         On-link      192.168.65.1    276
   192.168.65.255  255.255.255.255         On-link      192.168.65.1    276
    192.168.137.0    255.255.255.0         On-link     192.168.137.1    276
    192.168.137.1  255.255.255.255         On-link     192.168.137.1    276
  192.168.137.255  255.255.255.255         On-link     192.168.137.1    276
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link      192.168.0.19    261
        224.0.0.0        240.0.0.0         On-link     192.168.137.1    276
        224.0.0.0        240.0.0.0         On-link      192.168.65.1    276
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link      192.168.0.19    261
  255.255.255.255  255.255.255.255         On-link     192.168.137.1    276
  255.255.255.255  255.255.255.255         On-link      192.168.65.1    276
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0      192.168.0.1  Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination      Gateway
34    261 ::/0                     2001:470:1f09:784::1
34    261 ::/0                     fe80::223:4ff:fe11:98bd
  1    306 ::1/128                  On-link
34    261 2001:470:1f09:784::/64   On-link
34    261 2001:470:1f09:784::19/128
                                    On-link
34    261 2001:470:1f09:784:4486:916a:11a6:5746/128
                                    On-link
34    261 2001:470:1f09:784:8952:9e94:2200:a49/128
                                    On-link
34    261 fe80::/64                On-link
23    276 fe80::/64                On-link
24    276 fe80::/64                On-link
23    276 fe80::884:a1ab:55db:73c3/128
                                    On-link
34    261 fe80::4486:916a:11a6:5746/128
                                    On-link
24    276 fe80::6915:11b0:f0b:8fd9/128
                                    On-link
  1    306 ff00::/8                 On-link
34    261 ff00::/8                 On-link
23    276 ff00::/8                 On-link
24    276 ff00::/8                 On-link
===========================================================================
Persistent Routes:
If Metric Network Destination      Gateway
  0 4294967295 ::/0                     2001:470:1f09:784::1
===========================================================================

ipconfig:

Windows IP Configuration

   Host Name . . . . . . . . . . . . : andrew-PC
   Primary Dns Suffix  . . . . . . . :
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : i.growse.com
                                       growse.com

Ethernet adapter Local Area Connection 4:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : TEAM: Goteam
   Physical Address. . . . . . . . . : 00-22-15-79-D2-1C
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv6 Address. . . . . . . . . . . : 2001:470:1f09:784::19(Preferred)
   IPv6 Address. . . . . . . . . . . : 2001:470:1f09:784:4486:916a:11a6:5746(Preferred)
   Temporary IPv6 Address. . . . . . : 2001:470:1f09:784:b03e:dcc:a450:63be(Preferred)
   Link-local IPv6 Address . . . . . : fe80::4486:916a:11a6:5746%34(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.0.19(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 2001:470:1f09:784::1
                                       fe80::223:4ff:fe11:98bd%34
                                       192.168.0.1
   DNS Servers . . . . . . . . . . . : 2001:470:1f09:784::13
                                       192.168.0.13
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet1
   Physical Address. . . . . . . . . : 00-50-56-C0-00-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::884:a1ab:55db:73c3%23(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.137.1(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 536891478
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-10-E4-4E-00-22-15-79-D2-1C
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet8:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet8
   Physical Address. . . . . . . . . : 00-50-56-C0-00-08
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::6915:11b0:f0b:8fd9%24(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.65.1(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 553668694
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-10-E4-4E-00-22-15-79-D2-1C
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter isatap.{33B17797-2238-4B5E-92B9-18E969C93152}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 12:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter isatap.{1E73E28C-333C-4CFA-86C7-D5E321851A3D}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #3
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter isatap.{840D305C-7E8E-4DCA-BB4C-90CE1AE4B83F}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #4
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes


And it started working before I could get a wireshark dump. Grrrr.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on February 02, 2010, 10:34:40 AM
What Windows version(s) are you using and what is your laptop manufacture (mine Dell Inspiron). I can not check if the wireless interface is a Intel or Broadcom right now, but maybe it's a bug in the wireless driver...
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on February 02, 2010, 11:50:08 AM
The problem is the same on an Asus Eee901 with an intel 4965 wlan card running Ubuntu 9.10, a Dell Inspiron 15 (whatever that has) running windows 7 64 bit, my desktop (Asus motherboard, broadcom nics in LACP trunk) running Windows 7 64 bit.

Nothing can ping the router, and all get random dropouts for external connectivity.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: cholzhauer on February 02, 2010, 12:21:55 PM
Does the router always work?  Are you running some sort of firewall or virus scanner on your windows machines?  If the firewall/virus scanner doesn't know what IPv6 is, it will often silently discard that traffic.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on February 02, 2010, 12:44:31 PM
There's the default firewall running on the windows machines, but that wouldn't explain why the linux clients have the same issue. The router works well in all other ways. Here's a pcap of what happens when I try to ping the router's VLAN ipv6 address - http://filebin.ca/sptkp/router-fail.pcap

Note that I can still, and always have been able to ping the two tunnel IP addresses, both the remote and local side. I just can't ping the IPv6 address assigned to the internal interface on the router.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on February 02, 2010, 01:07:30 PM
Oke, so its probably not the client or the IOS version, so perhaps a config setting on the Cisco.
I can't test my Cisco right now, but maybe you can disable all access-lists and firewall rules for both IPv4 and IPv6. Maybe that will point to the right solution.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on February 02, 2010, 07:23:14 PM
Quote from: growse on February 02, 2010, 12:44:31 PM
There's the default firewall running on the windows machines, but that wouldn't explain why the linux clients have the same issue. The router works well in all other ways. Here's a pcap of what happens when I try to ping the router's VLAN ipv6 address - http://filebin.ca/sptkp/router-fail.pcap

Note that I can still, and always have been able to ping the two tunnel IP addresses, both the remote and local side. I just can't ping the IPv6 address assigned to the internal interface on the router.
Looking at this cap, the router isn't answering neighbor solicitations.  Not sure why that'd be.  When the neighbor solicitation goes out, you should see the router answer with a neighbor advertisement.  

Maybe you should check out your switch to see if it's not doing multicast correctly or something?

I also noticed the lack of "ipv6 enable" command on the vlan1 interface.  Not sure if this is required or not.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: cholzhauer on February 02, 2010, 07:38:10 PM
Now that you mention it jimb, broquea had posted a command that needed to be in the cisco config to make it work properly.  For the life of me though, I can't find it in the forums.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on February 02, 2010, 08:02:58 PM
Since I can't see the whole config, I don't know what he has.  I think you need "ipv6 unicast routing" or something like that.

From the packet sniffs, it looks like he has some HP Procurve switches.  From glancing at the docs, it looks like they do multicast forwarding by default, but didn't give it a good read, just a glance.

@growse: you may want to take a gander at this: http://cdn.procurve.com/training/Manuals/2900-3500-5400-6200-8200-IPv6-Jan08-7-MLD.pdf (http://cdn.procurve.com/training/Manuals/2900-3500-5400-6200-8200-IPv6-Jan08-7-MLD.pdf)

Not sure what's plugged into what though.  I presume the router has a built in switch?  If so, are the affected machines plugged into the built in?  Into another switch connected to it?  Are the switches on the right vlan (I imagine so since it sometimes works).  Anywayz, at this point it looks like a layer-2 problem, unless the router is actually receiving the mcast neighbor solicitations and ignoring them for some reason.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: cholzhauer on February 03, 2010, 04:57:39 AM
I'm running a bunch of procurve switches here at work and they're working great with IPv6.  If you are indeed using Procurve switches, make sure you're running at least a 13.xx firmware, or better yet, the 14.xx
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on February 05, 2010, 05:32:30 PM
I fixed it for my situation. It was indeed the problem with the "Router Advertisement" like jimb pointed out. I opened a Wireshark on both my Windows laptop and on my Ubuntu laptop. I noticed that on every "Neighbor Solicitation" Ubuntu sent a "Router Advertisement" was returned by the Cisco router. This was not the case on the Windows laptop that did sent out Neighbor Solicitation but not always received a Router advertisement (just like you can see in the pcap from growse), it only received a Router advertisement once in like 2 minutes and then I could use IPv6 for a little wile.
I entered the following command on Vlan 1 of the Cisco router:

ipv6 nd ra interval 4

This will force the router to sent a Router advertisement every 4 seconds. For now it is (temporarily)fixed for me, but I keep finding it strange that this behavior does not occur on the Ubuntu system. The systems are connected directly to the Cisco router which has a four port switch, I also swapped the Ethernet cables.
growse can you tell if this also works for you?

Btw, is there a default for the Router Advertisement interval?
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on February 05, 2010, 07:05:55 PM
Quote from: chiel on February 05, 2010, 05:32:30 PM
I fixed it for my situation. It was indeed the problem with the "Router Advertisement" like jimb pointed out. I opened a Wireshark on both my Windows laptop and on my Ubuntu laptop. I noticed that on every "Neighbor Solicitation" Ubuntu sent a "Router Advertisement" was returned by the Cisco router. This was not the case on the Windows laptop that did sent out Neighbor Solicitation but not always received a Router advertisement (just like you can see in the pcap from growse), it only received a Router advertisement once in like 2 minutes and then I could use IPv6 for a little wile.
I entered the following command on Vlan 1 of the Cisco router:

ipv6 nd ra interval 4

This will force the router to sent a Router advertisement every 4 seconds. For now it is (temporarily)fixed for me, but I keep finding it strange that this behavior does not occur on the Ubuntu system. The systems are connected directly to the Cisco router which has a four port switch, I also swapped the Ethernet cables.
growse can you tell if this also works for you?

Btw, is there a default for the Router Advertisement interval?
That's very odd.  Neighbor Solicitations should be answered by Neighbor Announcements.  Router Advertisements, as far as I understand, are just to announce the presence of a router on the link, and is also used for stateless autoconfig.

One test would be to swap ports on the ubuntu and windows systems.  If the ubuntu sys starts behaving like the windows system, it's probably the configuration of your ports/VLANs etc.  I'm wondering if the ports are all members of the VLAN which the router interface is on, although I don't know how it'd ever work if it wasn't.  It's almost as if the router isn't hearing the multicast neighbor solicitations, or the windows box isn't hearing the replies. 

You also may want to turn on ND debugging on the Cisco and see if you can see the solicitations when they are sent out by the windows box.  (debug ipv6 nd ... ND stuff should then show up on the console and in the log file).

For an example of what it should look like, here's some txt of ND/NA captures on my LAN:

No.     Time        Source                Destination           Protocol Info
      2 26.139155   2001:db8:1234:0:213:ceff:fe9b:2d70 ff02::1:ff6d:5aa4     ICMPv6   Neighbor solicitation

Frame 2 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70), Dst: IPv6mcast_ff:6d:5a:a4 (33:33:ff:6d:5a:a4)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 135 (Neighbor solicitation)
    Code: 0
    Checksum: 0x77f6 [correct]
    Target: 2001:db8:1234:0:201:2ff:fe6d:5aa4 (2001:db8:1234:0:201:2ff:fe6d:5aa4)
    ICMPv6 Option (Source link-layer address)
        Type: Source link-layer address (1)
        Length: 8
        Link-layer address: 00:13:ce:9b:2d:70

No.     Time        Source                Destination           Protocol Info
      3 26.140036   2001:db8:1234:0:201:2ff:fe6d:5aa4 2001:db8:1234:0:213:ceff:fe9b:2d70 ICMPv6   Neighbor advertisement

Frame 3 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: 3com_6d:5a:a4 (00:01:02:6d:5a:a4), Dst: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 136 (Neighbor advertisement)
    Code: 0
    Checksum: 0x0863 [correct]
    Flags: 0x60000000
    Target: 2001:db8:1234:0:201:2ff:fe6d:5aa4 (2001:db8:1234:0:201:2ff:fe6d:5aa4)
    ICMPv6 Option (Target link-layer address)
        Type: Target link-layer address (2)
        Length: 8
        Link-layer address: 00:01:02:6d:5a:a4

No.     Time        Source                Destination           Protocol Info
     10 31.140460   fe80::201:2ff:fe6d:5aa4 2001:db8:1234:0:213:ceff:fe9b:2d70 ICMPv6   Neighbor solicitation

Frame 10 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: 3com_6d:5a:a4 (00:01:02:6d:5a:a4), Dst: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 135 (Neighbor solicitation)
    Code: 0
    Checksum: 0x7479 [correct]
    Target: 2001:db8:1234:0:213:ceff:fe9b:2d70 (2001:db8:1234:0:213:ceff:fe9b:2d70)
    ICMPv6 Option (Source link-layer address)
        Type: Source link-layer address (1)
        Length: 8
        Link-layer address: 00:01:02:6d:5a:a4

No.     Time        Source                Destination           Protocol Info
     11 31.140491   2001:db8:1234:0:213:ceff:fe9b:2d70 fe80::201:2ff:fe6d:5aa4 ICMPv6   Neighbor advertisement

Frame 11 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70), Dst: 3com_6d:5a:a4 (00:01:02:6d:5a:a4)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 136 (Neighbor advertisement)
    Code: 0
    Checksum: 0x736c [correct]
    Flags: 0x60000000
    Target: 2001:db8:1234:0:213:ceff:fe9b:2d70 (2001:db8:1234:0:213:ceff:fe9b:2d70)
    ICMPv6 Option (Target link-layer address)
        Type: Target link-layer address (2)
        Length: 8
        Link-layer address: 00:13:ce:9b:2d:70

No.     Time        Source                Destination           Protocol Info
     12 71.851767   2001:db8:1234:0:213:ceff:fe9b:2d70 ff02::1:ff00:1        ICMPv6   Neighbor solicitation

Frame 12 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70), Dst: IPv6mcast_ff:00:00:01 (33:33:ff:00:00:01)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 135 (Neighbor solicitation)
    Code: 0
    Checksum: 0x3118 [correct]
    Target: 2001:db8:1234::1 (2001:db8:1234::1)
    ICMPv6 Option (Source link-layer address)
        Type: Source link-layer address (1)
        Length: 8
        Link-layer address: 00:13:ce:9b:2d:70

No.     Time        Source                Destination           Protocol Info
     13 71.853007   2001:db8:1234::1      2001:db8:1234:0:213:ceff:fe9b:2d70 ICMPv6   Neighbor advertisement

Frame 13 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: 3com_53:65:64 (00:50:da:53:65:64), Dst: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 136 (Neighbor advertisement)
    Code: 0
    Checksum: 0x618f [correct]
    Flags: 0xe0000000
    Target: 2001:db8:1234::1 (2001:db8:1234::1)
    ICMPv6 Option (Target link-layer address)
        Type: Target link-layer address (2)
        Length: 8
        Link-layer address: 00:50:da:53:65:64

No.     Time        Source                Destination           Protocol Info
     18 76.851330   fe80::250:daff:fe53:6564 2001:db8:1234:0:213:ceff:fe9b:2d70 ICMPv6   Neighbor solicitation

Frame 18 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: 3com_53:65:64 (00:50:da:53:65:64), Dst: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 135 (Neighbor solicitation)
    Code: 0
    Checksum: 0xae8d [correct]
    Target: 2001:db8:1234:0:213:ceff:fe9b:2d70 (2001:db8:1234:0:213:ceff:fe9b:2d70)
    ICMPv6 Option (Source link-layer address)
        Type: Source link-layer address (1)
        Length: 8
        Link-layer address: 00:50:da:53:65:64

No.     Time        Source                Destination           Protocol Info
     19 76.851354   2001:db8:1234:0:213:ceff:fe9b:2d70 fe80::250:daff:fe53:6564 ICMPv6   Neighbor advertisement

Frame 19 (86 bytes on wire, 86 bytes captured)
Ethernet II, Src: IntelCor_9b:2d:70 (00:13:ce:9b:2d:70), Dst: 3com_53:65:64 (00:50:da:53:65:64)
Internet Protocol Version 6
Internet Control Message Protocol v6
    Type: 136 (Neighbor advertisement)
    Code: 0
    Checksum: 0x9076 [correct]
    Flags: 0x60000000
    Target: 2001:db8:1234:0:213:ceff:fe9b:2d70 (2001:db8:1234:0:213:ceff:fe9b:2d70)
    ICMPv6 Option (Target link-layer address)
        Type: Target link-layer address (2)
        Length: 8
        Link-layer address: 00:13:ce:9b:2d:70


- Jim
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on February 14, 2010, 04:47:29 AM
Hello,

I did some more testing today. I first remove the configured "ipv6 nd ra interval 4" command, because this was just a temporarily fix.

I start doing some test on Windows Vista using a Ethernet cable directly attached to the Cisco 871W router. I opened Wireshark I noticed the following behavior:

1. I opened a ping to Google: "ping ipv6.google.com".
2. Neighbor Solicitations are sent from the Vista host using the IPV6 Link-Local address to a IPv6 multicast. Messages are not seen by the router using "debug ipv6 nd".
3. No answer is received from the Neighbor Solicitation, resulting in a ping timeout.
4. Once in a while a Router Advertisement is sent from the Cisco 781W router and received by the Windows Vista host.

router#show ipv6 neighbors
IPv6 Address                              Age Link-layer Addr State Interface
FE80::3143:BC3F:CC37:84AD                   2 0023.ae16.5735  STALE Vl1
2001:470:1F15:14AB:4886:CA6A:F752:BEFC      2 0023.ae16.5735  STALE Vl1

router#
*Nov  6 2009 21:30:23 UTC: ICMPv6-ND: Request to send RA for FE80::21C:F6FF:FE8C:5632
*Nov  6 2009 21:30:23 UTC: ICMPv6-ND: Sending RA from FE80::21C:F6FF:FE8C:5632 to FF02::1 on Vlan1
*Nov  6 2009 21:30:23 UTC: ICMPv6-ND:     MTU = 1500
*Nov  6 2009 21:30:23 UTC: ICMPv6-ND:     prefix = 2001:470:1F15:14AB::/64 onlink autoconfig
*Nov  6 2009 21:30:23 UTC: ICMPv6-ND:        2592000/604800 (valid/preferred)

router#show ipv6 neighbors
IPv6 Address                              Age Link-layer Addr State Interface
FE80::3143:BC3F:CC37:84AD                   0 0023.ae16.5735  REACH Vl1
2001:470:1F15:14AB:4886:CA6A:F752:BEFC      0 0023.ae16.5735  REACH Vl1


5. The ping to ipv6.google.com starts working.
6. And Neighbor Solicitations and Neighbor Advertisement messages are transported. This time using a Link-Local addresses instead of Global and Mulicast as in step 2.
7. After a while I guess the Router Advertisement that was received in step 4 times out and we are back at step 1.

No traffic is blocked on the Windows Vista host, I checked it on the Ethernet wire using a hub with Wireshark.

I did the same with a Ubuntu 9.10 host using the same wire (and thus port/vlan etc).

1. I opened a ping to Google: "ping ipv6.google.com".
2. Router Solicitations are sent from the Ubuntu host. Messages are not seen by the router using "debug ipv6 nd".
3. No answer is received from the Router Solicitations, resulting in a ping timeout.
4. Once in a while a Router Advertisement is sent from the Cisco 781W router and received by the Ubuntu host.
5. The ping to ipv6.google.com start working.
6. Neighbor Solicitations are sent from the router using the Link-Local address to the Ubuntu host on the Link-Global address.
   Neighbor Advertisements are sent from the Ubuntu host using the Link-Global address to the router on the Link-Local address.
7. IPv6 keeps working, I guess that the Router Advertisement stays active and doesn't time out like it did on the Windows Vista host.


My questions:
- It seems that Neighbor Solicitations are only for a while being processed by the Cisco router after a Router Advertisement has been sent by the router itself. How to fix this?
- Why does Windows Vista sent Neighbor Solicitations and Ubuntu Router Solicitations. Which OS is doing it wrong?
- Must Ubuntu not time out the Router Advertisement that it received in step 7

Attached are the pcap of the Windows host and the Ubuntu hosts, also the router config is attached. Open the pcaps using Wireshark and put in "icmpv6" as display filter.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on February 15, 2010, 04:19:58 PM
So the router is not seeing the ND packets, since you're not seeing them in the debug output.  Not sure why this is happening, but perhaps, as I mentioned before, it's some issue with the multicasts not making it to the router?

May want to try hooking one of the problem machines and the router interface to a difference switch, or a hub to see if it starts working.  If so then your switch isn't forwarding the mcast ND requests for whatever reason.

Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: mrtwrx on March 02, 2010, 07:14:54 PM
Seems quite similar to my issue in

http://www.tunnelbroker.net/forums/index.php?topic=818.msg4182#new

Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: mrtwrx on March 02, 2010, 07:17:59 PM
Interesting that your tuning of the router advertisement interval fixes it, the default lifetime value is 30 minutes!!

Windows must not be honoring this.

More later after ive checked further.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on March 04, 2010, 03:28:36 PM
mrtwrx, did you manage to get more information regarding this problem? I know that once I once got "native" IPv6 connectivity to that Windows machine I removed/disabled all Teredo/ISATAP stuff that provided IPv6 tunnels, I only noticed the problems a few months after that. Maybe it has something to do with that and that Windows also disables some other critical components once you disable/remove Teredo/ISATAP stuff.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on March 04, 2010, 04:16:06 PM
The more I see this stuff, the more I'm thinking this particular device or the IOS version people are using on them have some sort of bug in the ND area, or multicast processing area.

My theory is that shortening the RA period only masks the problems since the devices on the LAN learn the hardware address of the router via these RAs, while NA/ND is still failing.  If you make the RA interval short enough, that is, shorter than the expiry time on whatever hosts you are running IPv6 on, the MAC will stay in the neighbor table continually.

So this is just masking the real issue.

Now, if you can ping other devices on the same LAN without issue, it's definitely a problem with the router.  If you also have problems pinging other devices, then it points to some sort of layer 2 issue, or perhaps a problem with Windows itself.

Anyway, that's my additional $.02 on this issue.   :)
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: cholzhauer on March 04, 2010, 04:20:25 PM
Interesting.  You mean in this case, it might actually be a good thing that all my ASA's don't run IOS?

;)
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on March 04, 2010, 04:30:58 PM
Or at least that version of IOS on that hardware. 

I have no similar problems with a Cisco 7200 w/ IOS 12.4 ... OK.  I don't really have one.  It's dynamips.  But it works just fine.  :P

Also, I'd think that the possibility of failure would even be higher on this dynamips virtual cisco, seeing that it's essentially a VM, and it happens to be running under Windows XP (and using cygwin).  :P

Also, you guys may want to post up this question on somewhere with a wider audience, such as the IPv6-ops (http://lists.cluenet.de/mailman/listinfo/ipv6-ops) list, or maybe even NANOG (http://www.nanog.org/mailinglist/howtojoin/).
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on March 14, 2010, 03:15:47 PM
Hi guys,

Apologies for abandoning this slightly, I'll try to answer some of the points raised. It's good to see that others have raised this issue.

1) I've just enabled nd debugging. Will see what happens.

2) There is indeed a procurve in this network. I'll do my best to try and explain the topology: ADSL connection terminates on Cisco 877 running IOS12.4 with HE tunnel. HP Procurve 1800-8G is then plugged into one of the switchports on the router. Most other hosts are then plugged into this switch, two are LACP enabled (just in case that's throwing a wobbly in somewhere). HP firmware was previously PA02.09, just upgraded to PA03.09.

3) I just enabled ipv6 unicast-routing. Will see what happens.

4) I can always ping other hosts on the network. This is a router issue

5) I had this issue with IOS15 as well. This is what prompted me to figure that IOS15 was buggy, but I've got the same issue in 12.4

6) In the time that I've typed this, I've noticed I've received a lot of link-local RA debug messages. Is this normal? Here's a sample:

Mar 14 22:14:33 talkbot 7684: talkbot: 007680: *Dec 21 14:56:37.423 UTC: ICMPv6-ND: PROBE -> REACH: 2001:470:1F09:784:20C:29FF:FEA1:150B
Mar 14 22:14:36 talkbot 7685: talkbot: 007681: *Dec 21 14:56:39.855 UTC: ICMPv6-ND: Request to send RA for FE80::223:4FF:FE11:98BD
Mar 14 22:14:36 talkbot 7686: talkbot: 007682: *Dec 21 14:56:39.855 UTC: ICMPv6-ND: Sending RA from FE80::223:4FF:FE11:98BD to FF02::1 on Vlan1
Mar 14 22:14:36 talkbot 7687: talkbot: 007683: *Dec 21 14:56:39.855 UTC: ICMPv6-ND:     MTU = 1500
Mar 14 22:14:36 talkbot 7688: talkbot: 007684: *Dec 21 14:56:39.855 UTC: ICMPv6-ND:     prefix = 2001:470:1F09:784::/64 onlink autoconfig
Mar 14 22:14:36 talkbot 7689: talkbot: 007685: *Dec 21 14:56:39.855 UTC: ICMPv6-ND: #011     2592000/604800 (valid/preferred)
Mar 14 22:14:37 talkbot 7690: talkbot: 007686: *Dec 21 14:56:41.751 UTC: ICMPv6-ND: DELAY -> PROBE: FE80::20C:29FF:FEA1:150B
Mar 14 22:14:37 talkbot 7691: talkbot: 007687: *Dec 21 14:56:41.751 UTC: ICMPv6-ND: Sending NS for FE80::20C:29FF:FEA1:150B on Vlan1
Mar 14 22:14:37 talkbot 7692: talkbot: 007688: *Dec 21 14:56:41.755 UTC: ICMPv6-ND: Received NA for FE80::20C:29FF:FEA1:150B on Vlan1 from FE80::20C:29FF:FEA1:150B
Mar 14 22:14:37 talkbot 7693: talkbot: 007689: *Dec 21 14:56:41.755 UTC: ICMPv6-ND: PROBE -> REACH: FE80::20C:29FF:FEA1:150B
Mar 14 22:14:39 talkbot 7694: talkbot: 007690: *Dec 21 14:56:43.292 UTC: ICMPv6-ND: Request to send RA for FE80::223:4FF:FE11:98BD
Mar 14 22:14:39 talkbot 7695: talkbot: 007691: *Dec 21 14:56:43.292 UTC: ICMPv6-ND: Sending RA from FE80::223:4FF:FE11:98BD to FF02::1 on Vlan1
Mar 14 22:14:39 talkbot 7696: talkbot: 007692: *Dec 21 14:56:43.292 UTC: ICMPv6-ND:     MTU = 1500
Mar 14 22:14:39 talkbot 7697: talkbot: 007693: *Dec 21 14:56:43.292 UTC: ICMPv6-ND:     prefix = 2001:470:1F09:784::/64 onlink autoconfig
Mar 14 22:14:39 talkbot 7698: talkbot: 007694: *Dec 21 14:56:43.292 UTC: ICMPv6-ND: #011     2592000/604800 (valid/preferred)
Mar 14 22:14:39 talkbot 7699: talkbot: 007695: *Dec 21 14:56:43.420 UTC: ICMPv6-ND: STALE -> DELAY: 2001:470:1F09:784:94A9:11DE:AE:B2CF
Mar 14 22:14:42 talkbot 7700: talkbot: 007696: *Dec 21 14:56:46.340 UTC: %SEC-6-IPACCESSLOGDP: list INTERNET-IN denied icmp 74.55.143.114 -> 93.97.176.164 (0/0), 1 packet
Mar 14 22:14:43 talkbot 7701: talkbot: 007697: *Dec 21 14:56:47.328 UTC: ICMPv6-ND: Request to send RA for FE80::223:4FF:FE11:98BD
Mar 14 22:14:43 talkbot 7702: talkbot: 007698: *Dec 21 14:56:47.328 UTC: ICMPv6-ND: Sending RA from FE80::223:4FF:FE11:98BD to FF02::1 on Vlan1
Mar 14 22:14:43 talkbot 7703: talkbot: 007699: *Dec 21 14:56:47.328 UTC: ICMPv6-ND:     MTU = 1500
Mar 14 22:14:43 talkbot 7704: talkbot: 007700: *Dec 21 14:56:47.328 UTC: ICMPv6-ND:     prefix = 2001:470:1F09:784::/64 onlink autoconfig
Mar 14 22:14:43 talkbot 7705: talkbot: 007701: *Dec 21 14:56:47.328 UTC: ICMPv6-ND: #011     2592000/604800 (valid/preferred)
Mar 14 22:14:43 talkbot 7706: talkbot: 007702: *Dec 21 14:56:47.652 UTC: ICMPv6-ND: REACH -> STALE: 2001:470:1F09:784:20C:29FF:FE7F:19E5
Mar 14 22:14:43 talkbot 7707: talkbot: 007703: *Dec 21 14:56:48.212 UTC: ICMPv6-ND: Received NS for FE80::223:4FF:FE11:98BD on Vlan1 from FE80::F880:F7D2:A3DF:7BE6
Mar 14 22:14:44 talkbot 7708: talkbot: 007704: *Dec 21 14:56:48.212 UTC: ICMPv6-ND: Sending NA for FE80::223:4FF:FE11:98BD on Vlan1
Mar 14 22:14:44 talkbot 7709: talkbot: 007705: *Dec 21 14:56:48.216 UTC: ICMPv6-ND: STALE -> DELAY: FE80::F880:F7D2:A3DF:7BE6
Mar 14 22:14:44 talkbot 7710: talkbot: 007706: *Dec 21 14:56:48.420 UTC: ICMPv6-ND: DELAY -> PROBE: 2001:470:1F09:784:94A9:11DE:AE:B2CF
Mar 14 22:14:44 talkbot 7711: talkbot: 007707: *Dec 21 14:56:48.420 UTC: ICMPv6-ND: Sending NS for 2001:470:1F09:784:94A9:11DE:AE:B2CF on Vlan1
Mar 14 22:14:44 talkbot 7712: talkbot: 007708: *Dec 21 14:56:48.424 UTC: ICMPv6-ND: Received NA for 2001:470:1F09:784:94A9:11DE:AE:B2CF on Vlan1 from 2001:470:1F09:784:94A9:11DE:AE:B2CF
Mar 14 22:14:44 talkbot 7713: talkbot: 007709: *Dec 21 14:56:48.424 UTC: ICMPv6-ND: PROBE -> REACH: 2001:470:1F09:784:94A9:11DE:AE:B2CF
Mar 14 22:14:47 talkbot 7714: talkbot: 007710: *Dec 21 14:56:51.433 UTC: ICMPv6-ND: Request to send RA for FE80::223:4FF:FE11:98BD
Mar 14 22:14:47 talkbot 7715: talkbot: 007711: *Dec 21 14:56:51.433 UTC: ICMPv6-ND: Sending RA from FE80::223:4FF:FE11:98BD to FF02::1 on Vlan1
Mar 14 22:14:47 talkbot 7716: talkbot: 007712: *Dec 21 14:56:51.433 UTC: ICMPv6-ND:     MTU = 1500
Mar 14 22:14:47 talkbot 7717: talkbot: 007713: *Dec 21 14:56:51.433 UTC: ICMPv6-ND:     prefix = 2001:470:1F09:784::/64 onlink autoconfig
Mar 14 22:14:47 talkbot 7718: talkbot: 007714: *Dec 21 14:56:51.433 UTC: ICMPv6-ND: #011     2592000/604800 (valid/preferred)
Mar 14 22:14:47 talkbot 7719: talkbot: 007715: *Dec 21 14:56:52.145 UTC: ICMPv6-ND: REACH -> STALE: FE80::20C:29FF:FE7F:19E5
Mar 14 22:14:48 mailbot spamd[23942]: 190.120.128.47: connected (1/0)
Mar 14 22:14:49 talkbot 7720: talkbot: 007716: *Dec 21 14:56:53.217 UTC: ICMPv6-ND: DELAY -> PROBE: FE80::F880:F7D2:A3DF:7BE6
Mar 14 22:14:49 talkbot 7721: talkbot: 007717: *Dec 21 14:56:53.217 UTC: ICMPv6-ND: Sending NS for FE80::F880:F7D2:A3DF:7BE6 on Vlan1
Mar 14 22:14:49 talkbot 7722: talkbot: 007718: *Dec 21 14:56:53.233 UTC: ICMPv6-ND: Received NA for FE80::F880:F7D2:A3DF:7BE6 on Vlan1 from FE80::F880:F7D2:A3DF:7BE6
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on March 14, 2010, 08:12:42 PM
From looking at the logs, it looks like ND is actually working.  At least the router is sending NS requests and getting NA replies from at least one host on your global IPv6.

I'm not sure what's going on at this point.  May want to turn on more debugging.  It could just be a bug in multiple versions of IOS on that hardware.  May want to put a TAC call in to Cisco or something.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on March 15, 2010, 02:53:08 AM
Hmmm.

I know for certain it worked about 2 years ago at a different house. Same router. Now, the main things that would have been different then would be 1) IOS version (probably on 12.3 back then) and 2) client OS and network topology. I've got two theories. Either there's a bug in the router that manifests itself in 15 and 12.4. Or, there's a conflict between something new on the network and the router - obvious candidate is Windows 7.

I'll run it with the above changes for a while and then may roll back to either 12.2 or 12.3 and see if that resolves the issue.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on March 15, 2010, 01:35:45 PM
I have no issues with Win7 on my network.  Works fine, just like XP, Linux, FreeBSD.  I had no issue with a cisco 7200 emulator running 12.4 running under Dynamamips.

It does look like the router is receiving neighbor solicitations since I do see one in the logs.  Although maybe something funny is going on and the switches are filtering (or router ignoring) certain multicasts or something like that.

You may want to mirror the routers port and listen using wirehshark to see if the switches are properly delivering NS packets.  Do some pings from the boxes and that should generate NS packets.  If you see them, and especially if you don't see them in the debug output of the cisco, then the problem is definitely on the router.

If you don't see them on the switchport, then maybe you need to play with MLD snooping or other multicast features of the switches.  I vaguely remember from reading about MLD briefly that MLD snooping ignores multicast addresses in the range of neighbor solicitations, so it should always flood those anyway.  I'll have to revisit the docs on that though to be sure.  :P

EDIT: Yep.  I misremembered that.  MLD snooping would apply in the case of the solicited node multicasts used as destination addresses in neighbor solicitations.  So if it's goofing up or whatever, it could be preventing those packets from reaching the router.  I'd set up the wireshark snoop and see for sure if the router is receiving the NS multicasts.  A proper one (taken with tshark here) should look like this:

Frame 2 (86 bytes on wire, 86 bytes captured)
   Arrival Time: Mar 15, 2010 13:45:16.082192000
   [Time delta from previous captured frame: 68.048419000 seconds]
   [Time delta from previous displayed frame: 68.048419000 seconds]
   [Time since reference or first frame: 68.048419000 seconds]
   Frame Number: 2
   Frame Length: 86 bytes
   Capture Length: 86 bytes
   [Frame is marked: False]
   [Protocols in frame: eth:ipv6:icmpv6]
Ethernet II, Src: 3com_6d:5a:a4 (00:01:02:6d:5a:a4), Dst: IPv6mcast_ff:32:00:76 (33:33:ff:32:00:76)
   Destination: IPv6mcast_ff:32:00:76 (33:33:ff:32:00:76)
       Address: IPv6mcast_ff:32:00:76 (33:33:ff:32:00:76)
       .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
       .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
   Source: 3com_6d:5a:a4 (00:01:02:6d:5a:a4)
       Address: 3com_6d:5a:a4 (00:01:02:6d:5a:a4)
       .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
       .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
   Type: IPv6 (0x86dd)
Internet Protocol Version 6
   0110 .... = Version: 6
       [0110 .... = This field makes the filter "ip.version == 6" possible: 6]
   .... 0000 0000 .... .... .... .... .... = Traffic class: 0x00000000
   .... .... .... 0000 0000 0000 0000 0000 = Flowlabel: 0x00000000
   Payload length: 32
   Next header: ICMPv6 (0x3a)
   Hop limit: 255
   Source: 2001:db8:1234:0:201:2ff:fe6d:5aa4 (2001:db8:1234:0:201:2ff:fe6d:5aa4)
   Destination: ff02::1:ff32:76 (ff02::1:ff32:76)
Internet Control Message Protocol v6
   Type: 135 (Neighbor solicitation)
   Code: 0
   Checksum: 0x47d2 [correct]
   Target: 2001:db8:1234:0:211:25ff:fe32:76 (2001:db8:1234:0:211:25ff:fe32:76)
   ICMPv6 Option (Source link-layer address)
       Type: Source link-layer address (1)
       Length: 8
       Link-layer address: 00:01:02:6d:5a:a4


If you see packets like this arriving on the switchport the router is plugged into, then you know it's not a layer-2 problem, but a problem on the router.  Actually, you should see such packets on any port, unless MLD snooping is causing the solicited node address for the router to only be forwarded to the router (which would be proper behavior if you have the switches configured to not flood all mutlicast traffic and IGMP and/or MLD snooping is turned on).
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: mrtwrx on March 19, 2010, 08:32:52 PM
I don't have any switches in the way of my devices communicating.

The router works fine with all other O/S's except for Windows7. I have the same problem with with an 1841 and a 2821 that I borrowed from my lab.

How are you actually testing the dynamips instance providing reachability? Do you have another instance or two and are pinging a remote loopback?

I haven't gotten to the point of dumping packets using Wireshark, i'll have to set apart some time to compare timestamped debugs with wireshark outputs from both Windows7 and non windows7 hosts.

I actually get destination host unreachable from the ping attempts in the cmd prompt when its not working.
Funny how the neighbor discovery timeout on all other hosts/platforms is a lot longer then the 3 seconds it appears to be windows 7 and it completely ignores the default value of 30 minutes.

Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on March 19, 2010, 08:50:37 PM
I'm doing things like pinging the dynamips 7200 and SSHing into it from windows boxes, including windows 7 boxes.

Note that it's not acting as a router, it's only running IPv6.  But the ND problems you're having should still apply.  I have also set parts of my /48 on loopbacks on the Cisco and have been able to ping those too.

Not sure what the deal is, but from the sounds of it now (I didn't understand that it was only Win7 having issues), it sounds like something on your win7 box.

BTW, are your win7 boxes standalone boxes or VMs.  'Cause a VM would complicate things.  Make sure the interface is in bridged mode if using a VM.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on March 23, 2010, 05:01:25 AM
Ok, I *think* I'm an idiot.

I was messing around with a completely different problem involving ipv6 routing, and I noticed on my router config that I had the following statement:

ipv6 route 2000::/3 2a01:348:18a:6::1

This is fairly obviously a carry over from when I used to have a Sixxs tunnel configured. It's bad because it's trying to route an entire chunk of the ipv6 address space over to an ip that probably isn't expecting it.

I've removed that line and added:

ipv6 route ::/0 Tunnel 0

So far, from inside the network, I can now ping the router ipv6 address perfectly. I'll do some more testing when I get home to make sure that connectivity to the outside world is as I expect it.

Lessons learned: Check, recheck, recheck, and then recheck your configs. If it's a choice between finding a bug in IOS and the config errors, it's a config error.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on March 23, 2010, 09:03:03 AM
Ugh.  I should have asked for a complete config dump from the getgo.  But then that route wouldn't explain not being able to get to configured interfaces with more specific routes.  Even with a bad default route, pinging the routers interface should have still worked.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: cholzhauer on March 23, 2010, 09:04:43 AM
QuoteEven with a bad default route, pinging the routers interface should have still worked.

That's what I thought too; I was going to post a response, but didn't know if there was some cisco trick that I was missing.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: growse on March 23, 2010, 09:06:19 AM
It's weird, but I'm willing to put it down to "one of those things". I'm not completely confident it's problem-free, so if I see any similar issues, I'll post back.

Thanks all.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jimb on March 23, 2010, 09:25:41 AM
Maybe there's more left-over junk in the config. 

erase startup-config
reload

start over maybe?   ;D
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: jporcutt on May 29, 2010, 08:04:44 AM
@growse: Indeed it is "one of those things". This is a documented bug in the IOS 15 release notes. It was said to have been resolved in IOS 15.0(1)M2, but I'm still having this issue in 15.0(1)M2. It worked for a while after upgrading, but now today the symptom is back.

bummer  ::)

Bug reference number CSCte23299, IOS 15.0(1) release notes.
http://www.cisco.com/en/US/docs/ios/15_0/release/notes/150MCAVS.html
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on May 30, 2010, 02:12:53 PM
@jporcutt, thanks for that link. When I was looking at it I saw that the newest IOS is 15.1.1T(ED). I will give that a try next week.
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: chiel on June 05, 2010, 03:59:14 AM
Just to let you know, I upgraded to 15.1.1T(ED) and that finally fixed the problem for me! :)
Title: Re: IPv6 on Cisco 877 and clients - strange intermittent connectivity
Post by: mrtwrx on June 27, 2010, 05:24:53 AM
Quote from: chiel on June 05, 2010, 03:59:14 AM
Just to let you know, I upgraded to 15.1.1T(ED) and that finally fixed the problem for me! :)

Hah!

Nice, i'll have to try this.