Packets <MTU cause huge packet loss

[jorisclaassen]

I'm running a tunnel from my OpenWRT box (MTU 1280) to ams1 tunnelbroker endpoint, all worked well for quite some weeks. I'm experiencing some weird packet loss, for example to facebook. A quick trace and ping gives me back these results:

Code Select Expand

PS C:\Users\Joris> tracert facebook.com

Tracing route to facebook.com [2a03:2880:2130:cf05:face:b00c:0:1]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  2001:470:1f15:af4::1
  2    11 ms    10 ms    10 ms  jorisclaassen-2.tunnel.tserv11.ams1.ipv6.he.net [2001:470:1f14:b07::1]
  3    17 ms     8 ms    15 ms  v213.core1.ams1.he.net [2001:470:0:7d::1]
  4    16 ms    22 ms    24 ms  100ge9-1.core1.lon2.he.net [2001:470:0:2d0::1]
  5    82 ms    81 ms    87 ms  100ge1-1.core1.nyc4.he.net [2001:470:0:2cf::2]
  6    85 ms    92 ms    93 ms  100ge11-1.core1.ash1.he.net [2001:470:0:299::1]
  7    90 ms    87 ms    86 ms  facebook-as32934.10gigabitethernet6.switch3.ash1.he.net [2001:470:0:1bf::2]
  8   116 ms   118 ms   116 ms  be11.bb02.iad3.tfbnw.net [2620:0:1cff:dead:beef::e2]
  9   100 ms    99 ms    99 ms  ae13.bb04.frc3.tfbnw.net [2620:0:1cff:dead:beef::12a6]
10   121 ms   121 ms   119 ms  ae63.dr02.frc3.tfbnw.net [2620:0:1cff:dead:beef::615]
11   122 ms   125 ms   121 ms  po1021.csw12c.frc3.tfbnw.net [2620:0:1cff:dead:beef::25d]
12     *        *        *     Request timed out.
13   101 ms   100 ms    98 ms  edge-star6-shv-12-frc3.facebook.com [2a03:2880:2130:cf05:face:b00c:0:1]

Trace complete.
PS C:\Users\Joris> ping facebook.com -n 10

Pinging facebook.com [2a03:2880:2130:cf05:face:b00c:0:1] with 32 bytes of data:
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=100ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=101ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=99ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=101ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=98ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=100ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=101ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=98ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=99ms
Reply from 2a03:2880:2130:cf05:face:b00c:0:1: time=98ms

Ping statistics for 2a03:2880:2130:cf05:face:b00c:0:1:
    Packets: Sent = 10, Received = 10, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 98ms, Maximum = 101ms, Average = 99ms


Code Select Expand


PS C:\Users\Joris> ping facebook.com -n 10 -l 1500

Pinging facebook.com [2a03:2880:2130:cf05:face:b00c:0:1] with 1500 bytes of data:
Request timed out.
Any ideas?

The second box was of course stupid. Facebook just blocks these bigger ping packets. Still - the time before a page (facebook, google, youtube) loads is so high (20+ seconds) that I've unchecked IPv6 on my Windows box ATM. http://ipv6-test.com/pingtest/ gives ~60-80% packet loss to all servers.

Once again, any ideas?

[broquea]

You're on a tunnel trying packets that size? You are on (at the most) a tunnel with MTU 1480, and if you connect to your ISP over PPPOE, your tunnel needs to be set to 1472. You should be getting Packet too big errors reported:

Code Select Expand

~$ ping6 2001:470:1f14:af4::2 -s 1500
PING 2001:470:1f14:af4::2(2001:470:1f14:af4::2) 1500 data bytes
From 2001:470:0:7d::2 icmp_seq=1 Packet too big: mtu=1480
1508 bytes from 2001:470:1f14:af4::2: icmp_seq=2 ttl=54 time=242 ms
1508 bytes from 2001:470:1f14:af4::2: icmp_seq=3 ttl=54 time=242 ms
1508 bytes from 2001:470:1f14:af4::2: icmp_seq=4 ttl=54 time=242 ms
^C
--- 2001:470:1f14:af4::2 ping statistics ---
4 packets transmitted, 3 received, +1 errors, 25% packet loss, time 3003ms


[evantkh]

Some ICMPv6 types must not/should not be blocked.

https://www.ietf.org/rfc/rfc4890.txt

[jorisclaassen]

All ICMPv6 traffic to my host is allowed. I'm running a tunnel with MTU 1280, on a FTTH solution in the Netherlands.

I think this might be the same issue as https://forums.he.net/index.php?topic=3428.0

To illustrate the issue, see this image:

[evantkh]

Are you sure that the mtu is configured on both sodes, i.e. HE website and your client?

[jorisclaassen]

Double checked, changed MTU in OpenWRT from 1280 to 1480, reflecting the tunnelbroker.net side.

Then verified the MTU in my Windows box:

Code Select Expand


PS C:\Users\Joris> netsh interface ipv6 show subinterfaces

   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295                1          0     212840  Loopback Pseudo-Interface 1
  1480                1      66965     122638  Ethernet
  1500                1          0     160152  VMware Network Adapter VMnet1
  1500                1          0     160064  VMware Network Adapter VMnet8


Afterwards trying to ping google as an example

Code Select Expand


PS C:\WINDOWS\system32> ping google.com

Pinging google.com [2a00:1450:4013:c00::8b] with 32 bytes of data:
Request timed out.
Reply from 2a00:1450:4013:c00::8b: time=11ms
Reply from 2a00:1450:4013:c00::8b: time=13ms
Reply from 2a00:1450:4013:c00::8b: time=13ms

Ping statistics for 2a00:1450:4013:c00::8b:
    Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),
Approximate round trip times in milli-seconds:
    Minimum = 11ms, Maximum = 13ms, Average = 12ms

The ipv6-test website still states ~80% packet loss on ipv6 only.

[evantkh]

But you don't need to change the MTU of your ethernet connection to 1480, just on the tunnel only.

[jorisclaassen]

It is set on the tunnel as well, so there should not be any problem there, or am I mistaken?

[evantkh]

It is set on the tunnel as well, so there should not be any problem there, or am I mistaken?

The MTU of your own Ethernet connection should match with that of your router.
The MTU of the tunnel configuration of your router should match with the setting on tunnelbroker.net.

If it is still not working, try another device. If trying another device does not work as well, email ipv6@he.net.

[jorisclaassen]

After testing my laptop (running Fedora) had no issues, I resinstalled.

Wanted to reinstall that box for quite some time, and frankly - it worked.

Thanks for the assistance anyways!