routing problems within he.net?

[noahm]

Hi all. I've got two tunnels, one on the east coast (v6 and v4 addrs, respectively, are 2001:470:7:152::2/64 and 128.31.0.48) and one on the west coast (2001:470:1f04:9f0::2/64 and 69.181.139.255).  Sometime yesterday, it looks like something in Hurricane Electric's core stopped routing between the two.  Some traceroutes:

(from west coast)
frodo@shelob:~$ traceroute6 -s 2001:470:1f04:9f0::2 2001:470:7:152::2
traceroute to 2001:470:7:152::2 (2001:470:7:152::2), 30 hops max, 40 byte packets
1  tomkao-2.tunnel.tserv3.fmt2.ipv6.he.net (2001:470:1f04:9f0::1)  33.400 ms  40.620 ms  48.299 ms
2  gige-g3-20.core1.fmt2.ipv6.he.net (2001:470:0:45::1)  48.432 ms  48.441 ms  48.397 ms
3  10gigabitethernet1-2.core1.pao1.ipv6.he.net (2001:470:0:30::2)  57.137 ms  57.499 ms  57.494 ms
4  10gigabitethernet2-4.core1.ash1.ipv6.he.net (2001:470:0:35::2)  126.326 ms  126.279 ms  126.269 ms
5  gige-gbge0.tserv13.ash1.ipv6.he.net (2001:470:0:90::2)  124.610 ms  125.347 ms  125.744 ms
6  * * *
7  * * *
8  * * *
9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

(from east coast)
minas:~$ traceroute6 -m 20 -s 2001:470:7:152::2 2001:470:1f06:57d::2
traceroute to 2001:470:1f06:57d::2 (2001:470:1f06:57d::2), 20 hops max, 40 byte packets
1  2001:610:188:140:7268:7275:6c7a:4142 (2001:610:188:140:7268:7275:6c7a:4142)  88.355 ms  88.305 ms  88.595 ms
2  2001:610:188:140:145:100:188:1 (2001:610:188:140:145:100:188:1)  89.326 ms  89.302 ms  89.279 ms
3  * * *
4  * * *
5  * * *
6  * * *
7  * * *
8  * * *
9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *

I also note that the looking glass at http://lg.he.net/ can't reach my east coast endpoint's v6 address from Fremont 1, but it can reach my v4 address, so I don't think there's a v4 routing issue.

Thanks in advance for any clues, and as always, don't hesitate to ask for more details.

[broquea]

Not really seeing a problem tracing with the tunnel-servers or the looking glass.
Also appears your second traceroute was to 1f06 space instead of 1f04, and wasn't over the tunnel.

FMT->ASH:
1  gige-g3-20.core1.fmt2.ipv6.he.net (2001:470:0:45::1)  0.245 ms  0.14 ms  0.138 ms
2  10gigabitethernet1-2.core1.pao1.ipv6.he.net (2001:470:0:30::2)  3.221 ms  0.759 ms  0.738 ms
3  10gigabitethernet2-4.core1.ash1.ipv6.he.net (2001:470:0:35::2)  76.855 ms  86.709 ms  76.177 ms
4  gige-gbge0.tserv13.ash1.ipv6.he.net (2001:470:0:90::2)  77.282 ms  76.488 ms  76.996 ms
5  noahm-3-pt.tunnel.tserv13.ash1.ipv6.he.net (2001:470:7:152::2)  80.766 ms  81.341 ms  81.094 ms

ASH->FMT:
1  gige-g4-12.core1.ash1.he.net (2001:470:0:90::1)  0.17 ms  6.18 ms  0.136 ms
2  10gigabitethernet1-2.core1.nyc4.ipv6.he.net (2001:470:0:36::2)  9.813 ms  16.656 ms  6.376 ms
3  10gigabitethernet5-3.core1.lax1.he.net (2001:470:0:10e::1)  67.422 ms  67.425 ms  67.519 ms
4  10gigabitethernet1-3.core1.pao1.ipv6.he.net (2001:470:0:34::1)  75.204 ms  85.113 ms  75.25 ms
5  10gigabitethernet1-4.core1.fmt2.ipv6.he.net (2001:470:0:30::1)  78.096 ms  75.863 ms  75.903 ms
6  gige-gbge0.tserv3.fmt2.ipv6.he.net (2001:470:0:45::2)  80.639 ms  80.494 ms  86.666 ms
7  noahm-1-pt.tunnel.tserv3.fmt2.ipv6.he.net (2001:470:1f04:9f0::2)  89.818 ms  90.828 ms  91.169 ms

[noahm]

Yup, things are OK now.  But really, they were broken for ~12 hours with no changes on my end.  I'll try to get details posted quicker if I see it happen again.  Thanks.

[broquea]

You should actually be emailing ipv6@he.net since that opens a trouble ticket, and the forums are only browsed occasionally.