Hello,
I have the following config:
2 public IPv4 addresses used for 2 HE tunnel (2 accounts) which reside on a VPS (KVM)
These are my two configs for the interfaces:
/etc/network/interfaces.d/he01
auto he01
iface he01 inet6 v4tunnel
address 2001:470:1f1c:da5::2
netmask 64
endpoint 216.66.88.98
local 164.132.192.71
ttl 255
up ip -6 rule add from 2001:470:1f1c:da5::2 table he1
up ip -6 route add default via 2001:470:1f1c:da5::1 dev he01 table he1
down ip -6 rule del table he1
down ip -6 route flush table he1
/etc/network/interfaces.d/he02
auto he02
iface he02 inet6 v4tunnel
address 2001:470:6c:f4::2
netmask 64
endpoint 216.66.86.114
local 178.33.37.66
ttl 255
up ip -6 rule add from 2001:470:6c:f4::2 dev he02 table he2
up ip -6 route add default via 2001:470:6c:f4::1 table he2
down ip -6 rule del table he2
down ip -6 route flush table he2
I added 2 routing tables for this:
/etc/iproute2/rt_tables
#
# reserved values
#
100 he1
101 he2
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
but now I hit a brick wall. :-\
Depending on which interface was started first, the second one cant route into the internet. So he01 goes up first, he02 is pingable from the outside, can ping the HE gateway but can't ping ipv6.google.com
Ping via he01:
ping6 ipv6.google.com -I he01
PING ipv6.google.com(par10s33-in-x0e.1e100.net (2a00:1450:4007:816::200e)) from 2001:470:1f1c:da5::2 he01: 56 data bytes
64 bytes from par10s33-in-x0e.1e100.net (2a00:1450:4007:816::200e): icmp_seq=1 ttl=55 time=22.5 ms
64 bytes from par10s33-in-x0e.1e100.net (2a00:1450:4007:816::200e): icmp_seq=2 ttl=55 time=22.5 ms
Ping via he02:
ping6 ipv6.google.com -I he02
connect: Network is unreachable
Ping gateway of he02:
ping6 2001:470:6c:f4::1 -I he02
PING 2001:470:6c:f4::1(2001:470:6c:f4::1) from 2001:470:6c:f4::2 he02: 56 data bytes
64 bytes from 2001:470:6c:f4::1: icmp_seq=1 ttl=64 time=23.1 ms
64 bytes from 2001:470:6c:f4::1: icmp_seq=2 ttl=64 time=23.1 ms
If I ifdown he01, he02 works fine and vice versa. What am I missing?
Here are some diagnostics, if you need anything else let me know :)
netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 164.132.192.1 0.0.0.0 UG 0 0 0 ens3
164.132.192.1 0.0.0.0 255.255.255.255 UH 0 0 0 ens3
178.33.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ens3
ip -6 ro
2001:470:6c:f4::/64 dev he02 proto kernel metric 256 pref medium
2001:470:1f1c:da5::/64 dev he01 proto kernel metric 256 pref medium
fe80::/64 dev ens3 proto kernel metric 256 pref medium
fe80::/64 dev he01 proto kernel metric 256 pref medium
fe80::/64 dev he02 proto kernel metric 256 pref medium
route -6n
Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
::/0 2001:470:1f1c:da5::1 UG 1024 1 18 he01
::/0 :: !n -1 1 1078 lo
::/0 2001:470:6c:f4::1 UG 1024 0 0 he02
::/0 :: !n -1 1 1078 lo
2001:470:6c:f4::/64 :: Un 256 0 1 he02
2001:470:1f1c:da5::/64 :: Un 256 0 1 he01
fe80::/64 :: U 256 0 0 ens3
fe80::/64 :: Un 256 0 0 he01
fe80::/64 :: Un 256 0 0 he02
::/0 :: !n -1 1 1078 lo
::1/128 :: Un 0 2 82 lo
2001:470:6c:f4::2/128 :: Un 0 1 0 lo
2001:470:1f1c:da5::2/128 :: Un 0 2 8 lo
fe80::a484:c047/128 :: Un 0 1 0 lo
fe80::b221:2542/128 :: Un 0 1 0 lo
fe80::f816:3eff:fe26:fb1c/128 :: Un 0 1 0 lo
ff00::/8 :: U 256 0 0 ens3
ff00::/8 :: U 256 0 0 he01
ff00::/8 :: U 256 0 0 he02
::/0 :: !n -1 1 1078 lo
ifconfig he01
he01: flags=209<UP,POINTOPOINT,RUNNING,NOARP> mtu 1480
inet6 2001:470:1f1c:da5::2 prefixlen 64 scopeid 0x0<global>
inet6 fe80::a484:c047 prefixlen 64 scopeid 0x20<link>
sit txqueuelen 1 (IPv6-in-IPv4)
RX packets 117 bytes 221114 (215.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 110 bytes 8327 (8.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ifconfig he02
he02: flags=209<UP,POINTOPOINT,RUNNING,NOARP> mtu 1480
inet6 2001:470:6c:f4::2 prefixlen 64 scopeid 0x0<global>
inet6 fe80::b221:2542 prefixlen 64 scopeid 0x20<link>
sit txqueuelen 1 (IPv6-in-IPv4)
RX packets 2 bytes 208 (208.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2 bytes 208 (208.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Did this used to work and broke, or is this a new setup?
This a new setup, before I only had only 1 tunnel interface per server
1) Look carefully at your "up ip -6" rules. They are not parallel for the two interfaces. The explicit interface declaration "dev xxx" isn't similar - it hops between the "rule" and "route" subcommands.
2) If you want packets to go out via both interfaces, you need to do some sort of multi-routing. This may entail running a routing protocol (BGP, OSPF, etc), or enabling multiple equal path routing in the kernel. You have multiple default routes, so only the first one found in the routing table will be used in the absence of multi-routing.
You'll also need to make sure packets from a range of IPs go out their associated tunnel interface. We do drop spoofed traffic across tunnels, such as would be the case if tunnel1's IPs send traffic out tunnel2.
1) I suggest avoiding /etc/iproute2/rt_tables and ip rule altogether. These are for when you want grossly different routing based on some conditions. On the other hand, you do actually have such a condition configured in this case. but...
2) If you actually want to split things over two tunnels, what you should actually be doing is getting your own AS, two BGP tunnels, and a full routing daemon to manage them.
3) With the rule setup, I think it is not sufficient to specify "-I he02" on ping. Specify "-I 2001:470:1f1c:da5::2" or "-I 2001:470:6c:f4::2". The problem is that the source address gets chosen in the table, which is after the rule.
4) Showing "ip -6 ro" isn't enough. You need "ip -6 rule" and "ip -6 route list table he1", etc...