BGP Peering (IPv6) - Peering point for North American traffic in London?

[snarked]

Some screwy things are going on with HE and Cogent.  Maybe this is something that HE would like to fix.  It looks as if domestic (within North America) traffic has to get routed to London (in Europe) to cross the peering bridge between the two:

From Columbus, Ohio to Los Angeles, CA:

1  core1.fe4-0-0.columbus.glorb.com (2607:fc18:dead:beef::1)  0.735 ms 0.683 ms  0.878 ms
2  2001:550:2:12::5 (2001:550:2:12::5)  154.252 ms  154.250 ms  154.237 ms
3  2001:550::139 (2001:550::139)  187.161 ms  187.150 ms  187.345 ms
4  2001:550::166 (2001:550::166)  162.839 ms  162.839 ms  162.829 ms
5   (2001:550:2:17::1)  205.984 ms  206.219 ms  206.213 ms
6  2001:550::164 (2001:550::164)  67.991 ms  66.542 ms  66.420 ms
7  2001:550::67 (2001:550::67)  134.055 ms  67.881 ms  67.826 ms
8  * * *
9  2001:978:2:22::1 (2001:978:2:22::1)  333.461 ms  313.348 ms  313.226 ms
10  2001:978:3::1a (2001:978:3::1a)  102.270 ms  102.062 ms  102.237 ms
11  10gigabitethernet1-1.core1.lon1.he.net (2001:7f8:4::1b1b:1)  91.963 ms 91.969 ms  92.228 ms
12  10gigabitethernet2-3.core1.ny4.ipv6.he.net (2001:470:0:3e::1)  92.160 ms  96.964 ms  91.851 ms
13  10gigabitethernet5-3.core1.lax1.he.net (2001:470:0:10e::1)  153.235 ms 153.160 ms  153.108 ms
14  gige-gbge0.tserv15.lax1.ipv6.he.net (2001:470:0:9d::2)  158.010 ms 160.450 ms  155.411 ms
15  me.

2001:550::/32 Is Cogent North America
2001:978::/32 is Cogent Europe (Germany).

The traceroute was done by an adminstrator at Glorb Internet Services (from his end).

[broquea]

So either someone is leaking us to Cogent, or Cogent takes IPv6 transit from someone to get to us now and had them not announce Cogent. (basically this is probably a leak)

However what it really is, is that Glorb.com isn't preferring routes learned over their BGP tunnel with us for our own address space. That would be something they need to fix.

Our return path:

Code Select Expand

core1.lax2.he.net> traceroute ipv6 2607:fc18:dead:beef::1

Tracing the route to IPv6 node  from 1 to 30 hops

 1    <1 ms   <1 ms   <1 ms 10gigabitethernet2-1.core1.lax1.he.net [2001:470:0:72::1]
 2    66 ms   61 ms   63 ms 10gigabitethernet4-3.core1.nyc4.he.net [2001:470:0:10e::2]
 3    67 ms   67 ms   72 ms 10gigabitethernet2-3.core1.ash1.ipv6.he.net [2001:470:0:36::1]
 4    68 ms   68 ms   68 ms 100m-0-0.ash.ipv6.he.net [2001:470:0:40::1]
 5   157 ms  153 ms  153 ms core1.fe4-0-0.columbus.glorb.com [2607:fc18:dead:beef::1]# Entry cached for another 56 seconds.

[snarked]

OK.  I'll let them know.  Glad to see it wasn't an issue under HE's control.

[broquea]

Probably some sort of leak, since:

Code Select Expand


core1.lon1.he.net> show ipv6 bgp 2001:0978::/32
BGP4 : None of the BGP4 routes match the display condition

Code Select Expand

core1.lon1.he.net> traceroute ipv6 2607:fc18:dead:beef::1

Tracing the route to IPv6 node  from 1 to 30 hops

  1    79 ms   74 ms   75 ms 10gigabitethernet2-3.core1.ny4.ipv6.he.net [2001:470:0:3e::1]
  2    74 ms   74 ms   74 ms 10gigabitethernet2-3.core1.ash1.ipv6.he.net [2001:470:0:36::1]
  3    75 ms   75 ms   75 ms 100m-0-0.ash.ipv6.he.net [2001:470:0:40::1]
  4    94 ms  100 ms  100 ms core1.fe4-0-0.columbus.glorb.com [2607:fc18:dead:beef::1]

London definitely knows that Glorb is in Ashburn.

[jimb]

FWIW your route server doesn't show anything for either of those Cogent nets either.

[snarked]

Thank you for your side of the traceroutes.  We (really meaning the Glorb Admin) found a bad leak (from Cogent) and were able to override it.  The path is now 7 hops (6 to the LA tunnel server), which makes more sense considering the 5-hop path from the tunnel server.  Thanks.

[broquea]

Glad Glorb could fix it for you!

[jimb]

Thank you for your side of the traceroutes.  We (really meaning the Glorb Admin) found a bad leak (from Cogent) and were able to override it.  The path is now 7 hops (6 to the LA tunnel server), which makes more sense considering the 5-hop path from the tunnel server.  Thanks.

Curious.  What exactly was causing HE's /32 (presumably) route from Cogent to be preferred over the route from HE ?

[snarked]

I don't exactly know, but glorb was set up to prefer Cogent (including the default route), and somehow, this yielded a result where Cogent's route to HE was considered "lower cost" than the BGP tunnel to HE even for HE itself (at least, that's my understanding - as this is second hand).  What wasn't predicted is that in this case, Cogent's lowest cost peering point with HE (per Cogent) turned out to be London instead of a point within North America.  This has been manually overridden - HE's /32 net will always take HE's tunnel now (regardless of cost).

[jimb]

Ah.  Prob local preference or something like that.

[broquea]

Well, the real underlying issue here is someone was leaking our prefix, and maybe others. We've determined what happened, and are working with them to resolve their misconfiguration.

[snarked]

That's my understanding as well.  Glorb fixed the problem at their router manually, but who knows where else it got leaked....