Question regarding tserv5.lon1

[mhisani]

I see it's currently down on the service status. I've been having packet loss and intermittent connectivity for a few days now. Is the downtime related to that? Are there issues? Sorry to trouble you all but i'd like to find the cause for my issues that have been fine up until a little bit ago..

[kasperd]

I see it's currently down on the service status. I've been having packet loss and intermittent connectivity for a few days now. Is the downtime related to that? Are there issues? Sorry to trouble you all but i'd like to find the cause for my issues that have been fine up until a little bit ago..

I looked at http://tunnelbroker.net/status.php and it showed all tunnel servers as up. However in the past when I have found a tunnel server to be down, it also showed as up on that status page.

I did a few traceroutes to your IP. I did notice a few packets getting lost and high latency. Multiple times I saw a roundtrip time of one second or more.

Maybe the downtime you saw was due to the tunnel server being restarted, after somebody else affected by this problem contacted ipv6@he.net. But I have no way of knowing. The symptoms I saw are gone now, and I don't know why.

[cholzhauer]

I just tried to access this thread and was unable to because my tunnel wasn't working (ironic) 

In the past though, I've seen glitches that don't make it to the tunnel status page.

[kasperd]

I just tried to access this thread and was unable to because my tunnel wasn't working (ironic)

But this thread is about the London tunnel server, and as far as I can tell you are using the New York tunnel server. Another thread may be relevant to you. I don't know if twinbee reported the problem. But I have kept pinging occasionally, and the problem isn't gone yet.

[cholzhauer]

Yeah I know, my post wasn't to say that my tunnel was down, but to say that some glitches with the tunnel servers don't make it to the server status page.

[kasperd]

Yeah I know, my post wasn't to say that my tunnel was down, but to say that some glitches with the tunnel servers don't make it to the server status page.

Point taken.

[mhisani]

I'm still baffled at whats going on. The tunnel works perfectly but say I leave a ping going it will randomly stop for sometimes a few seconds.. sometimes a a few minutes then start again.. Totally losing ipv6 connectivity. Router still have it's ipv6 address as does my person pc but dead to the world for a bit.. I'd feel better if I knew it was the tunnel server doing it but obviously as it stands it niggles at me. Ideas / Things to try please?

[kasperd]

Did you create another tunnel since last I checked? It looks like you currently have two tunnels on that tunnel server, and only one of them is responding.

This is what a traceroute looks like on the tunnel, that is up:

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  0.509 ms  22.846 ms  23.852 ms
2  2001:470:27:940::1  40.112 ms  33.458 ms  43.644 ms
3  2001:470:0:11e::1  48.363 ms  36.567 ms  41.121 ms
4  2001:470:0:22f::1  71.260 ms  76.101 ms  65.785 ms
5  2001:470:0:3f::1  66.608 ms  73.892 ms  75.250 ms
6  2001:470:0:67::2  68.426 ms  108.981 ms  108.850 ms
7  2001:470:1f08:1d8::2  127.598 ms  96.176 ms  96.857 msI'll try to run that traceroute periodically such that I hopefully can provide you with an example of what it looks like from the other end, when the connection goes down.

[kasperd]

I'll try to run that traceroute periodically such that I hopefully can provide you with an example of what it looks like from the other end, when the connection goes down.

At 1:24 CEST there was a short outage lasting at least 10 seconds:

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  0.510 ms  8.970 ms  9.566 ms
2  2001:470:27:940::1  31.375 ms  27.377 ms  30.613 ms
3  2001:470:0:11e::1  36.020 ms  38.328 ms  38.870 ms
4  2001:470:0:22f::1  58.305 ms  62.172 ms  63.782 ms
5  2001:470:0:3f::1  76.606 ms  71.681 ms  76.912 ms
6  *  *  *
7  *  *  *
8  *  *  *
9  *  *  *
10  *  *  *
11  *  *  *
12  *  *  *
13  *  *  *
14  *  *  *
15  *  *  *
16  *  *  *
17  *  *  *
18  *  *  *
19  *  *  *
20  *  *  *
21  *  *  *
22  *  *  *
23  *  *  *
24  *  *  *
25  *  *  *
26  *  *  *
27  *  *  *
28  *  *  *
29  *  *  *
30  *  *  *

At 2:10:50 there was a short outage lasting at least 5 seconds:

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  0.518 ms  8.745 ms  9.341 ms
2  2001:470:27:940::1  56.063 ms  52.459 ms  55.119 ms
3  2001:470:0:11e::1  67.108 ms  44.914 ms  48.076 ms
4  2001:470:0:22f::1  76.570 ms  77.426 ms  76.999 ms
5  2001:470:0:3f::1  88.699 ms  77.522 ms  78.114 ms
6  2001:470:0:67::2  59.117 ms  62.425 ms  62.133 ms
7  *  *  *
8  *  *  *
9  *  *  *
10  *  *  *
11  *  *  *
12  *  *  *
13  *  *  *
14  *  *  *
15  *  *  *
16  *  *  *
17  *  *  *
18  *  *  *
19  *  *  *
20  *  *  2001:470:1f08:1d8::2  80.903 ms

At 2:26:12 there was a short outage lasting at least 10 seconds:

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  0.518 ms  9.025 ms  9.620 ms
2  2001:470:27:940::1  31.265 ms  27.120 ms  33.762 ms
3  2001:470:0:11e::1  34.121 ms  26.308 ms  26.875 ms
4  2001:470:0:22f::1  50.273 ms  61.416 ms  55.155 ms
5  2001:470:0:3f::1  62.845 ms  66.341 ms  70.169 ms
6  2001:470:0:67::2  61.303 ms  65.275 ms  69.398 ms
7  *  *  *
8  *  *  *
9  *  *  *
10  *  *  *
11  *  *  *
12  *  *  *
13  *  *  *
14  *  *  *
15  *  *  *
16  *  *  *
17  *  *  *
18  *  *  *
19  *  *  *
20  *  *  *
21  *  *  *
22  *  *  *
23  *  *  *
24  *  *  *
25  *  *  *
26  *  *  *
27  *  *  *
28  *  *  2001:470:1f08:1d8::2  89.308 ms

At 9:34:29 there was an outage lasting at least 25 seconds:

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  0.560 ms  8.338 ms  8.990 ms
2  2001:470:27:940::1  61.356 ms  57.341 ms  60.470 ms
3  2001:470:0:11e::1  63.518 ms  21.001 ms  32.069 ms
4  2001:470:0:22f::1  53.394 ms  54.198 ms  63.375 ms
5  2001:470:0:3f::1  66.825 ms  74.776 ms  75.582 ms
6  *  *  2001:470:0:67::2  1878.428 ms
7  *  *  *
8  *  *  *
9  *  *  *
10  *  *  *
11  *  *  *
12  *  *  *
13  *  *  *
14  *  *  *
15  *  *  *
16  *  *  *
17  *  *  *
18  *  *  *
19  *  *  *
20  *  *  *
21  *  *  *
22  *  *  *
23  *  *  *
24  *  *  *
25  *  *  *
26  *  *  *
27  *  *  *
28  *  *  *
29  *  *  *
30  *  *  *

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  0.544 ms  14.111 ms  14.590 ms
2  2001:470:27:940::1  55.138 ms  45.240 ms  48.532 ms
3  2001:470:0:11e::1  51.965 ms  23.266 ms  27.209 ms
4  2001:470:0:22f::1  54.968 ms  58.350 ms  62.156 ms
5  2001:470:0:3f::1  80.421 ms  77.639 ms  78.054 ms
6  *  *  *
7  *  2001:470:1f08:1d8::2  2604.667 ms  2615.972 ms

Around 9:53 there was a longer outage lasting 15 minutes:

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  11.655 ms  1.374 ms  5.781 ms
2  2001:470:27:940::1  27.729 ms  31.485 ms  31.770 ms
3  2001:470:0:11e::1  34.312 ms  49.743 ms  53.846 ms
4  2001:470:0:22f::1  78.585 ms  91.300 ms  87.745 ms
5  2001:470:0:3f::1  94.181 ms  91.760 ms  92.394 ms
6  *  *  *
7  *  *  *
8  *  *  *
9  *  *  *
10  *  *  *
11  *  *  *
12  *  *  *
13  *  *  *
14  *  *  *
15  *  *  *
16  *  *  *
17  *  *  *
18  *  *  *
19  *  *  *
20  *  *  *
21  *  *  *
22  *  *  *
23  *  *  *
24  *  *  *
25  *  *  *
26  *  *  *
27  *  *  *
28  *  *  *
29  *  *  *
30  *  *  *The last thirty seconds of that outage there wasn't complete packet loss but just extremely high latency

Code Select Expand

traceroute to mhisani-3-pt.tunnel.tserv5.lon1.ipv6.he.net (2001:470:1f08:1d8::2), 30 hops max, 80 byte packets
1  2001:470:28:940:5d75:c1f4:e0a0:f8ec  0.528 ms  8.778 ms  9.379 ms
2  2001:470:27:940::1  32.172 ms  28.674 ms  31.604 ms
3  2001:470:0:11e::1  34.435 ms  65.829 ms  71.432 ms
4  2001:470:0:22f::1  99.056 ms  99.707 ms  97.204 ms
5  2001:470:0:3f::1  115.883 ms  113.525 ms  114.548 ms
6  2001:470:0:67::2  633.935 ms  637.145 ms  650.200 ms
7  2001:470:1f08:1d8::2  1269.933 ms  1260.103 ms  1295.334 ms

Notice that many of these outages appear on the IPv6 side of the tunnel server, so cannot be caused by the IPv4 link between tunnel server and user. The last one which ended with a period of high latency apparently added about 500ms of extra latency on both IPv4 and IPv6 side of the tunnel server. A few of them appear to be only on the IPv4 side of the tunnel server. Based on the above I say it is reasonable to assume the problem is on the tunnel server.

[mhisani]

http://www.thinkbroadband.com/ping/share/ca4583e711ddc7417993372a6951808f.html

Yeah. Think I should email them about it?

Also I made a new tunnel just to full start a scratch.

[kasperd]

That looks like a neat monitoring service. But why are users required to make up a UK postal code in order to use it?

Here is a graph from one of my tunnels: http://www.thinkbroadband.com/ping/share/083c2a45828ae66069c1eb577cf72c73.html

Mabye I should reconsider my choice to write a message on stdout for every ping I receive, at one per second that does make the output a bit noisy. I'm curious how their service will react to that particular IP address sending three replies to every ping it receives.

And a monitor on another one of my tunnels: http://www.thinkbroadband.com/ping/share/39ab03b5cd01faeb355e1ac99826bb35.html

[mhisani]

Namely as that site is basically for UK Broadband interests :)

I'm still getting the drop outs. :/

Perhaps I could trouble you to setup a london tunnel and monitor it yourself if you can. Do you think I should email he.net and bring it to their attention?

[kasperd]

Perhaps I could trouble you to setup a london tunnel and monitor it yourself if you can.

I already have five tunnels, so I can't create another.

Do you think I should email he.net and bring it to their attention?

Yes.

[kasperd]

Mabye I should reconsider my choice to write a message on stdout for every ping I receive, at one per second that does make the output a bit noisy. I'm curious how their service will react to that particular IP address sending three replies to every ping it receives.

They were dealing just fine with the multiple replies. But I found a starvation bug in my own code. In one rare corner case my code would only retransmit a query after 10 seconds had passed with no packets received at all. Receiving a ping per second meant that retransmission would never happen. Oops  :-[