High packet Loss Issue with tserv13.ash1

[NavankurIT]

I have a tunnel whose server is tserv13.ash1 (216.66.22.2)
with server IPv6 address of 2001:470:x:y::1/64.

Everything was working fine till last night. However, recently, I am
getting huge packet loss over IPv6.

Example (from a connected host):

$ ping -6 2001:470:x:y::1
PING 2001:470:x:y::1(2001:470:x:y::1) 56 data bytes
64 bytes from 2001:470:x:y::1: icmp_seq=11 ttl=63 time=12.3 ms
64 bytes from 2001:470:x:y::1: icmp_seq=15 ttl=63 time=9.68 ms
64 bytes from 2001:470:x:y::1: icmp_seq=22 ttl=63 time=9.85 ms
...
^C
--- 2001:470:x:y::1 ping statistics ---
124 packets transmitted, 20 received, 83% packet loss, time 124629ms
rtt min/avg/max/mdev = 8.236/12.530/57.814/10.507 ms

From the router (Asus RT-N66U running latest version of AsusWRT Merlin
fork):

# ping6 2001:470:x:y::1
PING 2001:470:x:y::1 (2001:470:x:y::1): 56 data bytes
64 bytes from 2001:470:x:y::1: seq=1 ttl=64 time=9.045 ms
64 bytes from 2001:470:x:y::1: seq=5 ttl=64 time=14.029 ms
64 bytes from 2001:470:x:y::1: seq=8 ttl=64 time=10.578 ms
64 bytes from 2001:470:x:y::1: seq=12 ttl=64 time=8.693 ms
64 bytes from 2001:470:x:y::1: seq=19 ttl=64 time=9.672 ms
64 bytes from 2001:470:x:y::1: seq=26 ttl=64 time=8.345 ms
64 bytes from 2001:470:x:y::1: seq=33 ttl=64 time=9.525 ms


--- 2001:470:x:y::1 ping statistics ---
59 packets transmitted, 12 packets received, 79% packet loss
round-trip min/avg/max = 7.821/9.460/14.029 ms

I have another HE.net tunnel exiting at tserv15.lax1 whose client node I
am using as an OpenVPN server. Pinging the node
www.facebook.com(edge-star-mini6-shv-01-ort2.facebook.com
(2a03:2880:f127:83:face:b00c:0:25de)), I get:

$ ping -6 www.facebook.com
PING www.facebook.com(edge-star-mini6-shv-01-ort2.facebook.com
(2a03:2880:f127:83:face:b00c:0:25de)) 56 data bytes
64 bytes from edge-star-mini6-shv-01-ort2.facebook.com
(2a03:2880:f127:83:face:b00c:0:25de): icmp_seq=1 ttl=53 time=117 ms
64 bytes from edge-star-mini6-shv-01-ort2.facebook.com
(2a03:2880:f127:83:face:b00c:0:25de): icmp_seq=2 ttl=53 time=117 ms
64 bytes from edge-star-mini6-shv-01-ort2.facebook.com
(2a03:2880:f127:83:face:b00c:0:25de): icmp_seq=3 ttl=53 time=120 ms
...
^C
--- www.facebook.com ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9010ms
rtt min/avg/max/mdev = 116.914/118.369/121.678/1.463 ms

Doing a traceroute:

sachin@triveni:~$ sudo traceroute -6 www.facebook.com
traceroute to www.facebook.com (2a03:2880:f127:83:face:b00c:0:25de), 30
hops max, 80 byte packets
1  2001:470::x:y:z:1 (2001:470:d:76:67:d:2525:1)  70.807 ms
70.821 ms  70.820 ms
2  xyz.tunnel.tserv15.lax1.ipv6.he.net (2001:470:x:y::1)
83.297 ms  88.606 ms  96.264 ms
3  10ge9-12.core1.lax1.he.net (2001:470:0:9d::1)  101.638 ms  101.657
ms  101.663 ms
4  asn-qwest-us-as209.10gigabitethernet5-5.core1.lax1.he.net
(2001:470:0:2c0::2)  96.819 ms  96.778 ms  96.805 ms
5  2001:428::205:171:200:225 (2001:428::205:171:200:225)  144.178 ms
144.136 ms  144.162 ms
6  2001:428:1802:10:0:13e:0:2 (2001:428:1802:10:0:13e:0:2)  142.919 ms
118.623 ms  125.931 ms
7  po131.asw04.ord3.tfbnw.net (2620:0:1cff:dead:beef::16ca)  126.414 ms
po131.asw01.ord3.tfbnw.net (2620:0:1cff:dead:beef::16c2)  125.124 ms
po131.asw04.ord3.tfbnw.net (2620:0:1cff:dead:beef::16ca)  124.858 ms
8  po221.psw01a.ort2.tfbnw.net (2620:0:1cff:dead:beef::16f5)  125.050
ms  124.863 ms  124.862 ms
9  po1.msw1aa.01.ort2.tfbnw.net (2a03:2880:f00b:ffff::101)  124.760 ms
po1.msw1an.01.ort2.tfbnw.net (2a03:2880:f00b:ffff::11b)  117.197 ms
po3.msw1af.01.ort2.tfbnw.net (2a03:2880:f00b:ffff::173)  116.177 ms
10  edge-star-mini6-shv-01-ort2.facebook.com
(2a03:2880:f127:83:face:b00c:0:25de)  116.744 ms  121.266 ms  121.149 ms


This looks fine ...

Pinging over IPv4 to the tunnel server:

$ ping tserv13.ash1.ipv6.he.net
PING tserv13.ash1.ipv6.he.net (216.66.22.2) 56(84) bytes of data.
64 bytes from tserv2.ash1.he.net (216.66.22.2): icmp_seq=1 ttl=58
time=9.16 ms
64 bytes from tserv2.ash1.he.net (216.66.22.2): icmp_seq=2 ttl=58
time=8.46 ms
64 bytes from tserv2.ash1.he.net (216.66.22.2): icmp_seq=3 ttl=58
time=9.94 ms
64 bytes from tserv2.ash1.he.net (216.66.22.2): icmp_seq=4 ttl=58
time=7.23 ms
64 bytes from tserv2.ash1.he.net (216.66.22.2): icmp_seq=5 ttl=58
time=12.7 ms
64 bytes from tserv2.ash1.he.net (216.66.22.2): icmp_seq=6 ttl=58
time=7.80 ms
64 bytes from tserv2.ash1.he.net (216.66.22.2): icmp_seq=7 ttl=58
time=7.98 ms
...
^C
--- tserv13.ash1.ipv6.he.net ping statistics ---
183 packets transmitted, 183 received, 0% packet loss, time 195281ms
rtt min/avg/max/mdev = 7.174/9.580/32.977/2.682

[kriteknetworks]

if you want help, don't obfuscate your ips.