Strange IPv6 behavior through tunnel

[sblan]

Been running a tunnel for a couple years with no issues.  Then sometime in early March I noticed that some sites stopped loading via IPv6, while others work fine.  The symptom is the browser will connect, but then hang with no data received.  This happens on facebook and weatherunderground most often.  Google seems to work without issue over the tunnel, as do many other random (infrequently visited) sites.  Some of the random, infrequent sites exhibit the same behavior as facebook and wunderground which are consistent in not working.  If I take the tunnel down and connect with IPv4, the sites load correctly.  I've tried lowering the MSS but that didn't seem to help.  Using tcpdump, I can see IPv6 packets going to/from sites that are working and IPv4 packets going to other sites that are working (presumably they don't have IPv6 addresses yet), but on the ones that don't work, the initial handshake goes out and the connection is established, but then nothing else comes through.  I can't figure out what to tweak or try next so any ideas, info, wild *** guesses are welcome.

[cholzhauer]

Does browser matter? 

[sblan]

No, browser does not matter. OS doesn't matter either.  See it on both Linux and Windows.

[Hello71]

try ping.

attach a tcpdump.

[snarked]

Check for this:  Some sites have different hosts that serve their auxiliary web stuff like images and CSS descriptions.  Is it possible that you've encountered a situation where their primary web server has IPv6 connectivity and their auxiliary functions host(s) do not?

[PatrickDickey]

I'm running into this same issue. I updated an earlier post of mine (IPv6 sites time out) because of it. In March, I found that my IPv4 endpoint at Tunnelbroker changed (without my knowing it). That fixed the issue for a while, but about a month ago it came back.

So, I'm eagerly hoping for a resolution to this issue as well.

Have a great day.:)
Patrick.

P.S. If we're both on the same IPv4 endpoint server, I'd wonder if there isn't an issue with that specific one. My endpoint is 184.105.253.14.

[jstark]

I'm seeing this issue as well.  Like the first poster I first noticed the issue back in March, however at the time I was too busy to properly deal with it.  The tunnel had been working well for for me, for a number of years, until this new behaviour began to occur in March.  I find that IPV6 HTTP connections to google always work, while IPV6 HTTP connections to almost anywhere else (i.e. facebook, wikipedia, he.net) usually timeout.  The behaviour seems to be intermittent, meaning that IPV6 connections to the above sites occasionally work.  I should point out that my experience has been that either they all work, or none of them (except google) work.

I'm including the output of tcpdump -vvv at the end of this post.  The output that I'm including here was run on the client as the web browser attempted to make a HTTP connection to he.net over IPV6.  Running tcpdump on the tunnel interface directly produces output that is almost identical.  From the output below we see that the tcp handshake completes successfully (first 3 packets), and the client issues the HTTP GET request which the server acknowledges (next 2 packets).  After that is a bunch of garbage (the remaining 6) packets, followed by nothing.   The garbage packets might be a part of the response to the HTTP GET request, but I'm just guessing here.

HTTP requests to the other sites listed (i.e. facebook) show the same behaviour as the one to he.net.  I can successfully ping and traceroute to all of these sites over IPV6, it's only HTTP that seems to fail.  Google (which works) seems to be about two hops closer to me than the other sites (which fail).  I rebooted my gateway machine, which has the tunnel connection, a few days ago.  After that reboot everything worked until today when my IP changed (I'm on DSL).  That might be related or it might not.  I manually rebuilt the IPV6 tunnel, but that had no effect.

Any idea about what is happening here, and what to do about it would be appreciated.

Code Select Expand


17:16:17.608402 IP6 (hlim 64, next-header TCP (6) payload length: 40) 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272 > 2001:470:0:76::2.80: Flags [S], cksum 0x71b0 (incorrect -> 0xdde7), seq 3902521298, win 28800, options [mss 1440,sackOK,TS val 982948 ecr 0,nop,wscale 7], length 0
17:16:17.680390 IP6 (hlim 56, next-header TCP (6) payload length: 40) 2001:470:0:76::2.80 > 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272: Flags [S.], cksum 0x171b (correct), seq 1150038101, ack 3902521299, win 14080, options [mss 1420,sackOK,TS val 1871780829 ecr 982948,nop,wscale 7], length 0
17:16:17.680418 IP6 (hlim 64, next-header TCP (6) payload length: 32) 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272 > 2001:470:0:76::2.80: Flags [.], cksum 0x71a8 (incorrect -> 0x7bcc), seq 1, ack 1, win 225, options [nop,nop,TS val 982966 ecr 1871780829], length 0
17:16:17.680455 IP6 (hlim 64, next-header TCP (6) payload length: 333) 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272 > 2001:470:0:76::2.80: Flags [P.], cksum 0x72d5 (incorrect -> 0xd5ef), seq 1:302, ack 1, win 225, options [nop,nop,TS val 982966 ecr 1871780829], length 301
17:16:17.757738 IP6 (hlim 56, next-header TCP (6) payload length: 32) 2001:470:0:76::2.80 > 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272: Flags [.], cksum 0x7af5 (correct), seq 1, ack 302, win 119, options [nop,nop,TS val 1871780849 ecr 982966], length 0
17:16:17.765292 IP6 (hlim 56, next-header TCP (6) payload length: 1405) 2001:470:0:76::2.80 > 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272: Flags [P.], cksum 0x315e (correct), seq 7041:8414, ack 302, win 119, options [nop,nop,TS val 1871780850 ecr 982966], length 1373
17:16:17.765316 IP6 (hlim 64, next-header TCP (6) payload length: 44) 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272 > 2001:470:0:76::2.80: Flags [.], cksum 0x71b4 (incorrect -> 0x1e26), seq 302, ack 1, win 248, options [nop,nop,TS val 982987 ecr 1871780849,nop,nop,sack 1 {7041:8414}], length 0
17:16:17.765319 IP6 (hlim 56, next-header TCP (6) payload length: 43) 2001:470:0:76::2.80 > 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272: Flags [P.], cksum 0xcdc8 (correct), seq 8414:8425, ack 302, win 119, options [nop,nop,TS val 1871780850 ecr 982966], length 11
17:16:17.765323 IP6 (hlim 64, next-header TCP (6) payload length: 44) 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272 > 2001:470:0:76::2.80: Flags [.], cksum 0x71b4 (incorrect -> 0x1e13), seq 302, ack 1, win 256, options [nop,nop,TS val 982987 ecr 1871780849,nop,nop,sack 1 {7041:8425}], length 0
17:16:32.784983 IP6 (hlim 56, next-header TCP (6) payload length: 32) 2001:470:0:76::2.80 > 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272: Flags [F.], cksum 0x4b4c (correct), seq 8425, ack 302, win 119, options [nop,nop,TS val 1871784604 ecr 982987], length 0
17:16:32.785008 IP6 (hlim 64, next-header TCP (6) payload length: 44) 2001:470:b008:0:f66d:4ff:fe25:7e8c.57272 > 2001:470:0:76::2.80: Flags [.], cksum 0x71b4 (incorrect -> 0x0f67), seq 302, ack 1, win 256, options [nop,nop,TS val 986742 ecr 1871780849,nop,nop,sack 1 {7041:8426}], length 0


[PatrickDickey]

Can you post the results of pings with various sized packets (ping -l in Windows, I'm not sure how to do it in Linux)? Something like ping -l 1480 ipv6.google.com, ping -l 1400 ipv6.google.com, and so on? See what the maximum sized packet that goes through is, and add 48 to that. Then check to see if the MTU (Advanced Settings on your tunnel configuration page) is less than or equal to that number. In my case, my maximum packet was 1424, but my MTU was set for 1432 (MTU was 1480, which is 1432 + 48). The 48 is the size in bytes for the header information.

Hope this helps a bit, and have a great day.:)
Patrick.

[dtaubert]

The problem I'm seeing is that HE's side of the tunnel interface isn't sending ICMPv6 Packet Too Big messages when it should.  This causes ipv6 MTU discovery to not work properly for inbound traffic, so all packets above the tunnel's MTU but below the rest of the network's MTU are dropped.  Hence, tcp sessions appear to hang when the window gets big enough.

For example:

Code Select Expand

[2001:420:302:1003:fe4d:d4ff:fe3d:de22]% ping6 -c 3 -s 1400 -M do 2001:470:1f04:1490::2
PING 2001:470:1f04:1490::2(2001:470:1f04:1490::2) 1400 data bytes
1408 bytes from 2001:470:1f04:1490::2: icmp_seq=1 ttl=51 time=19.2 ms
1408 bytes from 2001:470:1f04:1490::2: icmp_seq=2 ttl=51 time=19.9 ms
1408 bytes from 2001:470:1f04:1490::2: icmp_seq=3 ttl=51 time=18.2 ms

--- 2001:470:1f04:1490::2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2021ms
rtt min/avg/max/mdev = 18.214/19.110/19.911/0.705 ms
[2001:420:302:1003:fe4d:d4ff:fe3d:de22]% ping6 -c 3 -s 1452 -M do 2001:470:1f04:1490::2
PING 2001:470:1f04:1490::2(2001:470:1f04:1490::2) 1452 data bytes

--- 2001:470:1f04:1490::2 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 11999ms

HE's tunnel interface (2001:470:0:45::2) should have responded with ICMPv6 Packet Too Big for those last 3 packets.

Clearly this was a relatively recent change by HE.  It worked properly for years before this...

Derek

[Hello71]

you need to set the mtu in advanced.