IPv6 on Cisco 877 and clients - strange intermittent connectivity

[growse]

I've got a tunnel set up on my Cisco 877 and the tunnel seems to work great. However, the stability of the connectivity of autoconfigured clients (laptops) behind the router seems to vary quite significantly. Connections to certain sites work, whereas other sites don't (most of the time). For example:

Code Select Expand

ping6 ipv6.google.com
PING ipv6.google.com(2a00:1450:8001::67) 56 data bytes
*times out*


Sometimes the above works, and I get a response.

whereas...

Code Select Expand


ing6 www.kame.net
PING www.kame.net(orange.kame.net) 56 data bytes
64 bytes from orange.kame.net: icmp_seq=1 ttl=52 time=282 ms
64 bytes from orange.kame.net: icmp_seq=2 ttl=52 time=294 ms


seems to mostly work all the time.

Pinging the above hosts work from both my router and static-addressed hosts. The router can ping static-addressed internal hosts but not autoconfigured hosts. No internal hosts can ping the router on the assigned address in the subnet, but everything can ping the router's end of the tunnel address.

Here's the interfaces in the router config:

Code Select Expand


interface Vlan1
description $ETH-SW-LAUNCH$$INTF-INFO-HWIC 4ESW$$ES_LAN$$FW_INSIDE$
ip address 192.168.0.1 255.255.255.0
no ip redirects
no ip unreachables
no ip proxy-arp
ip flow ingress
ip nat inside
ip virtual-reassembly
ip tcp adjust-mss 1452
ipv6 address 2001:470:1F09:784::1/64
ipv6 nd prefix 2001:470:1F09:784::/64
ipv6 nd ra lifetime 180
ipv6 nd ra interval 60
hold-queue 100 out
end
!
interface Tunnel0
description Hurricane Electric IPv6 Tunnel Broker
no ip address
ipv6 address 2001:470:1F08:784::2/64
ipv6 enable
tunnel source 93.97.176.164
tunnel destination 216.66.80.26
tunnel mode ipv6ip
end


What on earth is going on?

[cholzhauer]

Hmm...what does your routing table look like on the router?  What about one of the RA-enabled hosts?

[growse]

Right now, the ping connectivity to ipv6.google.com is there. What's never there is ping connectivity to the router.

Ra-host:

Code Select Expand


Kernel IPv6 routing table
Destination                    Next Hop                   Flag Met Ref Use If
2001:470:1f09:784::/64         ::                         UAe  256 0 10675 wlan0
2a01:348:18a::/64              ::                         UAe  256 0  1629 wlan0
fe80::/64                      ::                         U    256 0     0 wlan0
::/0                           fe80::223:4ff:fe11:98bd    UGDAe 1024 1 30980 wlan0
::/0                           ::                         !n   -1  1 49065 lo
::1/128                        ::                         Un   0   3    36 lo
2001:470:1f09:784:221:5cff:fe08:7291/128 ::                         Un   0   1 76795 lo
fe80::221:5cff:fe08:7291/128   ::                         Un   0   1   976 lo
ff00::/8                       ::                         U    256 0     0 wlan0
::/0                           ::                         !n   -1  1 49065 lo


Router

Code Select Expand


IPv6 Routing Table - Default - 6 entries
Codes: C - Connected, L - Local, S - Static, U - Per-user Static route
       B - BGP, M - MIPv6, R - RIP, D - EIGRP
       EX - EIGRP external
       O - OSPF Intra, OI - OSPF Inter, OE1 - OSPF ext 1, OE2 - OSPF ext 2
       ON1 - OSPF NSSA ext 1, ON2 - OSPF NSSA ext 2
S   ::/0 [1/0]
     via Tunnel0, directly connected
C   2001:470:1F08:784::/64 [0/0]
     via Tunnel0, directly connected
L   2001:470:1F08:784::2/128 [0/0]
     via Tunnel0, receive
C   2001:470:1F09:784::/64 [0/0]
     via Vlan1, directly connected
L   2001:470:1F09:784::1/128 [0/0]
     via Vlan1, receive
L   FF00::/8 [0/0]
     via Null0, receive


[chiel]

I have a Cisco 871W with almost the same problem, sometimes I cannot connect to a IPv6 host and a few minutes later I can connect without any problem. The strange thing is that I have a Windows Vista laptop and a Ubuntu 9.10 laptop, I only have these problems on the Vista laptop.

I'm using c870-advipservicesk9-mz.150-1.M.bin on the router (a version from 01/Oct/2009).
I already turned of the Windows Vista firewall, turned off the virus scanner and re-installed the IPv6 stack. Non seems to work :(
For now I have disabled IPv6 on the Vista laptop because I didn't want to wast any more time on the problem.

Can you tell what Cisco IOS version, DNS addresses (I only use the HE DNS, no secondary), POP location (mine is Amsterdam) and operating systems you are using?

[cholzhauer]

What's the output of "ipconfig /all" on your windows machine?

[jimb]

Try doing some packet captures with Wireshark or whatever when this problem is happening to get an idea what's failing.  

[growse]

I've got a static-addressed windows machine as well as an autoconfigured one. I'll grab the routing table and ipconfig output from the windows autoconfigured one later tonight.

I've been on IOS 12.4 for a long time and had ipv6 work perfectly. When I upgraded to IOS 15 I started to notice problems, but have since downgraded back to 12.4 and still have them, so I think the IOS version is coincidental.

I started to experience this back when I tunnelled via Sixxs - had the same issue there, so don't think it's a PoP / infrastructure problem.

I'll try and grab some packet captures later tonight as well. I know that the ping connectivity to the router never works, so I'll grab some captures of a ping from an autoconfigured linux host to the router.

[growse]

Right, on my static-allocated windows box which can currently not ping either the router or google.

Code Select Expand



Pinging ipv6.l.google.com [2a00:1450:8001::6a] with 32 bytes of data:
Destination host unreachable.

Ping statistics for 2a00:1450:8001::6a:
    Packets: Sent = 1, Received = 0, Lost = 1 (100% loss),
Control-C
^C
C:\Users\andrew>route print
===========================================================================
Interface List
34...00 22 15 79 d2 1c ......TEAM: Goteam
23...00 50 56 c0 00 01 ......VMware Virtual Ethernet Adapter for VMnet1
24...00 50 56 c0 00 08 ......VMware Virtual Ethernet Adapter for VMnet8
  1...........................Software Loopback Interface 1
32...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
13...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
14...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #3
15...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #4
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      192.168.0.1     192.168.0.19    261
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
      169.254.0.0      255.255.0.0         On-link     192.168.137.1    296
  169.254.255.255  255.255.255.255         On-link     192.168.137.1    276
      192.168.0.0    255.255.255.0         On-link      192.168.0.19    261
     192.168.0.19  255.255.255.255         On-link      192.168.0.19    261
    192.168.0.255  255.255.255.255         On-link      192.168.0.19    261
     192.168.65.0    255.255.255.0         On-link      192.168.65.1    276
     192.168.65.1  255.255.255.255         On-link      192.168.65.1    276
   192.168.65.255  255.255.255.255         On-link      192.168.65.1    276
    192.168.137.0    255.255.255.0         On-link     192.168.137.1    276
    192.168.137.1  255.255.255.255         On-link     192.168.137.1    276
  192.168.137.255  255.255.255.255         On-link     192.168.137.1    276
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link      192.168.0.19    261
        224.0.0.0        240.0.0.0         On-link     192.168.137.1    276
        224.0.0.0        240.0.0.0         On-link      192.168.65.1    276
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link      192.168.0.19    261
  255.255.255.255  255.255.255.255         On-link     192.168.137.1    276
  255.255.255.255  255.255.255.255         On-link      192.168.65.1    276
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0      192.168.0.1  Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination      Gateway
34    261 ::/0                     2001:470:1f09:784::1
34    261 ::/0                     fe80::223:4ff:fe11:98bd
  1    306 ::1/128                  On-link
34    261 2001:470:1f09:784::/64   On-link
34    261 2001:470:1f09:784::19/128
                                    On-link
34    261 2001:470:1f09:784:4486:916a:11a6:5746/128
                                    On-link
34    261 2001:470:1f09:784:8952:9e94:2200:a49/128
                                    On-link
34    261 fe80::/64                On-link
23    276 fe80::/64                On-link
24    276 fe80::/64                On-link
23    276 fe80::884:a1ab:55db:73c3/128
                                    On-link
34    261 fe80::4486:916a:11a6:5746/128
                                    On-link
24    276 fe80::6915:11b0:f0b:8fd9/128
                                    On-link
  1    306 ff00::/8                 On-link
34    261 ff00::/8                 On-link
23    276 ff00::/8                 On-link
24    276 ff00::/8                 On-link
===========================================================================
Persistent Routes:
If Metric Network Destination      Gateway
  0 4294967295 ::/0                     2001:470:1f09:784::1
===========================================================================

ipconfig:

Code Select Expand

Windows IP Configuration

   Host Name . . . . . . . . . . . . : andrew-PC
   Primary Dns Suffix  . . . . . . . :
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : i.growse.com
                                       growse.com

Ethernet adapter Local Area Connection 4:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : TEAM: Goteam
   Physical Address. . . . . . . . . : 00-22-15-79-D2-1C
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv6 Address. . . . . . . . . . . : 2001:470:1f09:784::19(Preferred)
   IPv6 Address. . . . . . . . . . . : 2001:470:1f09:784:4486:916a:11a6:5746(Preferred)
   Temporary IPv6 Address. . . . . . : 2001:470:1f09:784:b03e:dcc:a450:63be(Preferred)
   Link-local IPv6 Address . . . . . : fe80::4486:916a:11a6:5746%34(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.0.19(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 2001:470:1f09:784::1
                                       fe80::223:4ff:fe11:98bd%34
                                       192.168.0.1
   DNS Servers . . . . . . . . . . . : 2001:470:1f09:784::13
                                       192.168.0.13
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet1
   Physical Address. . . . . . . . . : 00-50-56-C0-00-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::884:a1ab:55db:73c3%23(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.137.1(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 536891478
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-10-E4-4E-00-22-15-79-D2-1C
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet8:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet8
   Physical Address. . . . . . . . . : 00-50-56-C0-00-08
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::6915:11b0:f0b:8fd9%24(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.65.1(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 553668694
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-12-10-E4-4E-00-22-15-79-D2-1C
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter isatap.{33B17797-2238-4B5E-92B9-18E969C93152}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 12:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter isatap.{1E73E28C-333C-4CFA-86C7-D5E321851A3D}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #3
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter isatap.{840D305C-7E8E-4DCA-BB4C-90CE1AE4B83F}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #4
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes


And it started working before I could get a wireshark dump. Grrrr.

[chiel]

What Windows version(s) are you using and what is your laptop manufacture (mine Dell Inspiron). I can not check if the wireless interface is a Intel or Broadcom right now, but maybe it's a bug in the wireless driver...

[growse]

The problem is the same on an Asus Eee901 with an intel 4965 wlan card running Ubuntu 9.10, a Dell Inspiron 15 (whatever that has) running windows 7 64 bit, my desktop (Asus motherboard, broadcom nics in LACP trunk) running Windows 7 64 bit.

Nothing can ping the router, and all get random dropouts for external connectivity.

[cholzhauer]

Does the router always work?  Are you running some sort of firewall or virus scanner on your windows machines?  If the firewall/virus scanner doesn't know what IPv6 is, it will often silently discard that traffic.

[growse]

There's the default firewall running on the windows machines, but that wouldn't explain why the linux clients have the same issue. The router works well in all other ways. Here's a pcap of what happens when I try to ping the router's VLAN ipv6 address - http://filebin.ca/sptkp/router-fail.pcap

Note that I can still, and always have been able to ping the two tunnel IP addresses, both the remote and local side. I just can't ping the IPv6 address assigned to the internal interface on the router.

[chiel]

Oke, so its probably not the client or the IOS version, so perhaps a config setting on the Cisco.
I can't test my Cisco right now, but maybe you can disable all access-lists and firewall rules for both IPv4 and IPv6. Maybe that will point to the right solution.

[jimb]

There's the default firewall running on the windows machines, but that wouldn't explain why the linux clients have the same issue. The router works well in all other ways. Here's a pcap of what happens when I try to ping the router's VLAN ipv6 address - http://filebin.ca/sptkp/router-fail.pcap

Note that I can still, and always have been able to ping the two tunnel IP addresses, both the remote and local side. I just can't ping the IPv6 address assigned to the internal interface on the router.

Looking at this cap, the router isn't answering neighbor solicitations.  Not sure why that'd be.  When the neighbor solicitation goes out, you should see the router answer with a neighbor advertisement.  

Maybe you should check out your switch to see if it's not doing multicast correctly or something?

I also noticed the lack of "ipv6 enable" command on the vlan1 interface.  Not sure if this is required or not.

[cholzhauer]

Now that you mention it jimb, broquea had posted a command that needed to be in the cisco config to make it work properly.  For the life of me though, I can't find it in the forums.