Create my own broker server

[liquidcz]

Is there anyone who can help me to build my own tunnel broker server?

I have my own server with nativ IPV6 connectivity with my own /48 subnet. Now i want to connect my office and home to ipv6 thru my own server.

Haw can i do that?

Servers os is Linux Ubuntu 8.04 LTS.

Many thanks.

[kristiankrohn]

You could configure standard 6in4 tunnels (basically what you use to connect to HE), but that requires static IPv4 addresses (or dynamic updates of the tunnel configuration), doesn't play well with NAT and offers no encryption.
If you want to do it this way, just look at your HE configuration and setup additional devices based on it.

I would recommend to build a VPN using either OpenVPN or tinc:

* Install the openvpn/tinc software on all nodes.
* Setup the tunnels.
* Make sure routing is enabled where necessary and IPv4 & IPv6 firewalls don't block anything important.

You have to make sure to use a VPN configuration that allows IPv6 (e.g. something that emulates ethernet).
For OpenVPN I use 'dev-type tun-ipv6' and for tinc I use 'Mode = switch'.

Sorry for not describing the tunnel setup, routing etc. in detail, but if you have specific questions I - and surely others - will gladly help.

[liquidcz]

Thank you for your fast respond.

Im thinking about tinc, with this similar solution. http://www.tinc-vpn.org/examples/ipv6-network

BUT, im using RouterBoard at home and office, and im not sure if i can connect RouterOS (Mikrotik RouterBoard) to this type od VPN.

Do you know i it is possible? If not, i will prefer to do it by tunnelbroker style.

If i describe my situation detailed, are you unhesitating to help configure it?

[kristiankrohn]

I have never heard of MikroTik before. But a quick google search seems to indicate that OpenVPN is supported, while tinc is not.

But the main problem seems to be: This RouterOS thingy doesn't seem to support IPv6 at all. (At least I didn't find anything usable.)
You might want to look into this...

Edit: Ok, it does support IPv6. (Their wiki search just doesn't work & I didn't really google it.)

[liquidcz]

It does, because im using HE tunnel at this time from home.

Well, you suggest me to use OpenVPN solution then using tunnels?

Im running this configuration.

MyISP -> My Server - eth0 - IPV4/IPV6 + sit1 tunnel connected with my home

Im able to ping from home to my server via sit1 interface, but GW of MyISP is unreachable.

[kristiankrohn]

Assuming you have public, static IPv4 addresses for your MikroTik devices and don't need the encryption, I would suggest to use 6in4 tunnels.
This seems to be setup very easily: http://wiki.mikrotik.com/wiki/Setting_up_an_IPv6_tunnel_via_a_tunnel_broker

If you allready set this up and just can't reach anything beyond your server, you should check if the server is routing IPv6 packets:

Code Select Expand

cat /proc/sys/net/ipv6/conf/*/forwarding

If you see zeros, just put a one into the 'all' file:

Code Select Expand

echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

[liquidcz]

I have allready set it up on Mikrotik side.

Linux server (broker):
root@hankey:~# cat /proc/sys/net/ipv6/conf/*/forwarding
1
1
1
1
1
1
1

It looks like i allready have forwarding enabled. But still no answer from GW od MyISP.

[liquidcz]

Hi, thanks for helping me. This is my configuration of broker server:

I have this subnet:
2a01:5f0:100e::/48

GW on MyISP side is: 2a01:5f0:100e::1

interfaces:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 87.236.194.11
        netmask 255.255.254.0
        broadcast 87.236.195.255
        gateway 87.236.194.1

auto eth0:0
iface eth0:0 inet static
        address 87.236.194.12
        netmask 255.255.254.0
        broadcast 87.236.195.255
        gateway 78.236.194.1

auto eth0:1
iface eth0:1 inet static
        address 87.236.194.13
        netmask 255.255.254.0
        broadcast 87.236.195.255
        gateway 78.236.194.1

iface eth0 inet6 static
        address 2a01:5f0:100e::3
        netmask 64
        gateway 2a01:5f0:100e::1
        up ip -6 addr add 2a01:5f0:100e::4/64 dev eth0
        up ip -6 addr add 2a01:5f0:100e::5/64 dev eth0
        up ip -6 addr add 2a01:5f0:100e::6/64 dev eth0
        up ip -6 addr add 2a01:5f0:100e::7/64 dev eth0
        up ip -6 addr add 2a01:5f0:100e::8/64 dev eth0
        up ip -6 addr add 2a01:5f0:100e::9/64 dev eth0
        up ip -6 addr add 2a01:5f0:100e::2/128 dev eth0

auto sit1
iface sit1 inet6 v4tunnel
        address 2a01:5f0:100e:2::1
        netmask 64
        endpoint 88.102.7.4
        local 87.236.194.13
        ttl 255
        up ip link set mtu 1280 dev sit1
        up ip -6 route add 2a01:5f0:100e:3::/64 via 2a01:5f0:100e:2::2 dev sit1

ip -6 route

2a01:5f0:100e::2 dev eth0  metric 256  expires -3875sec mtu 1500 advmss 1440 hoplimit 4294967295
2a01:5f0:100e::/64 dev eth0  metric 256  expires -3875sec mtu 1500 advmss 1440 hoplimit 4294967295
2a01:5f0:100e:2::/64 via :: dev sit1  metric 256  expires -3875sec mtu 1280 advmss 1220 hoplimit 4294967295
2a01:5f0:100e:3::/64 via 2a01:5f0:100e:2::2 dev sit1  metric 1024  expires -3874sec mtu 1280 advmss 1220 hoplimit 4294967295
fe80::/64 dev vmnet8  metric 256  expires -76572sec mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev eth0  metric 256  expires -3875sec mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 via :: dev sit1  metric 256  expires -3874sec mtu 1280 advmss 1220 hoplimit 4294967295
default via 2a01:5f0:100e::1 dev eth0  metric 1  expires -3874sec mtu 1500 advmss 1440 hoplimit 4294967295


Thak you for your help.

[kristiankrohn]

This looks good so far, but I have a few questions:

* Is 2a01:5f0:100e::3 the correct address for eth0?
(It's just that I would expect 2a01:5f0:100e::2 to be the "normal" address. Also I can ping the ...::2 but not ...::3.)

* Can you confirm that there are no firewall rules that might block wanted v6 traffic? ("ip6tables -n -v -L")

* What addresses can you reach from the Mikrotik device? (2a01:5f0:100e:2::1? 2a01:5f0:100e::3?)

* How is the Mikrotik configured? (especially the default route)

FYI: I also cannot ping 2a01:5f0:100e:2::1
FYI2: Maybe tcpdump can help debugging stuff? E.g. "tcpdump -nqi sit1 ip6"

[liquidcz]

ad 1) Yes, it is corect IP for eth0. I will explain. I found trouble with IPV6 aliases on Ubuntu/Debian distro. I cannot ping primary IP, and the last IP on the same interface must be /128 to work right. If i add IP aliases without the last IP /128, a cannot ping MyISP GW at all. :-(

ad 2) I use ip6tables, but when i --flush ip6tables = same situation

Code Select Expand


root@hankey:~# ip6tables -n -v -L
Chain INPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
1642  164K ACCEPT     all      lo     *       ::/0                 ::/0
  382 86376 ACCEPT     all      eth0   *       ::/0                 ::/0               state RELATED,ESTABLISHED
   78  5714 ACCEPT     icmpv6    eth0   *       ::/0                 ::/0
   20  1444 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::7/128tcp dpt:80
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::7/128tcp dpt:443
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::7/128tcp dpt:20
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::7/128tcp dpt:21
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::7/128tcp dpt:22
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::6/128tcp dpt:80
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::6/128tcp dpt:443
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::6/128tcp dpt:53
    0     0 ACCEPT     udp      eth0   *       ::/0                 2a01:5f0:100e::6/128udp dpt:53
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::4/128tcp dpt:80
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::4/128tcp dpt:443
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::4/128tcp dpt:25
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::4/128tcp dpt:110
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::4/128tcp dpt:143
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::4/128tcp dpt:993
    0     0 ACCEPT     tcp      eth0   *       ::/0                 2a01:5f0:100e::4/128tcp dpt:995
   15   858 ACCEPT     all      sit1   *       ::/0                 ::/0
    0     0 ACCEPT     all      sit1   *       ::/0                 ::/0               state RELATED,ESTABLISHED
    0     0 ACCEPT     icmpv6    sit1   *       ::/0                 ::/0
    0     0 ACCEPT     all      sit2   *       ::/0                 ::/0
    0     0 ACCEPT     all      sit2   *       ::/0                 ::/0               state RELATED,ESTABLISHED
    0     0 ACCEPT     icmpv6    sit2   *       ::/0                 ::/0
    0     0 LOG        all      eth0   *       ::/0                 ::/0               LOG flags 0 level 4
    0     0 DROP       all      eth0   *       ::/0                 ::/0

Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
    3   221 ACCEPT     all      sit1   eth0    ::/0                 ::/0
    0     0 ACCEPT     all      eth0   sit1    ::/0                 ::/0
    0     0 ACCEPT     all      sit2   eth0    ::/0                 ::/0
    0     0 ACCEPT     all      eth0   sit2    ::/0                 ::/0

Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
1642  164K ACCEPT     all      *      lo      ::/0                 ::/0
  538  417K ACCEPT     all      *      eth0    ::/0                 ::/0               state NEW,RELATED,ESTABLISHED
   66  4864 ACCEPT     icmpv6    *      eth0    ::/0                 ::/0
   15   858 ACCEPT     all      *      sit1    ::/0                 ::/0
    0     0 ACCEPT     all      *      sit1    ::/0                 ::/0               state NEW,RELATED,ESTABLISHED
    0     0 ACCEPT     icmpv6    *      sit1    ::/0                 ::/0
    0     0 ACCEPT     all      *      sit2    ::/0                 ::/0
    0     0 ACCEPT     all      *      sit2    ::/0                 ::/0               state NEW,RELATED,ESTABLISHED
    0     0 ACCEPT     icmpv6    *      sit2    ::/0                 ::/0



ad 3) I can reach every IP on my server. 2a01:5f0:100e:2::1, 2a01:5f0:100e::2 - 7.

ad 4) Mikrotis route configuration:

Code Select Expand

/ipv6 route add comment="" disabled=no distance=1 dst-address=2000::/3 gateway=2a01:5f0:100e:2::1 scope=30 target-scope=10

tcpdump: pings from Mikrotik. pings to 2a01:5f0:100e::1 timeouts, others are OK.

Code Select Expand


tcpdump -nqi sit1 ip6
tcpdump: WARNING: sit1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on sit1, link-type RAW (Raw IP), capture size 96 bytes
22:47:01.425000 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 0, length 10
22:47:02.337929 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 256, length 10
22:47:03.360067 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 512, length 10
22:47:04.377364 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 768, length 10
22:47:05.390970 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 1024, length 10
22:47:09.297218 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::2: ICMP6, echo request, seq 1280, length 10
22:47:09.297319 IP6 2a01:5f0:100e::2 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 1280, length 10
22:47:10.333736 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::2: ICMP6, echo request, seq 1536, length 10
22:47:10.333834 IP6 2a01:5f0:100e::2 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 1536, length 10
22:47:11.346508 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::2: ICMP6, echo request, seq 1792, length 10
22:47:11.346644 IP6 2a01:5f0:100e::2 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 1792, length 10
22:47:15.264464 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::3: ICMP6, echo request, seq 2048, length 10
22:47:15.264554 IP6 2a01:5f0:100e::3 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 2048, length 10
22:47:16.277182 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::3: ICMP6, echo request, seq 2304, length 10
22:47:16.277286 IP6 2a01:5f0:100e::3 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 2304, length 10
22:47:17.291575 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::3: ICMP6, echo request, seq 2560, length 10
22:47:17.291662 IP6 2a01:5f0:100e::3 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 2560, length 10
22:47:20.990812 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::7: ICMP6, echo request, seq 2816, length 10
22:47:20.990903 IP6 2a01:5f0:100e::7 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 2816, length 10
22:47:22.002028 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::7: ICMP6, echo request, seq 3072, length 10
22:47:22.002111 IP6 2a01:5f0:100e::7 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 3072, length 10
22:47:23.015578 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::7: ICMP6, echo request, seq 3328, length 10
22:47:23.015674 IP6 2a01:5f0:100e::7 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 3328, length 10
22:47:29.773507 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e:2::1: ICMP6, echo request, seq 3584, length 10
22:47:29.773616 IP6 2a01:5f0:100e:2::1 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 3584, length 10
22:47:30.786680 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e:2::1: ICMP6, echo request, seq 3840, length 10
22:47:30.786769 IP6 2a01:5f0:100e:2::1 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 3840, length 10
22:47:31.791553 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e:2::1: ICMP6, echo request, seq 4096, length 10
22:47:31.791654 IP6 2a01:5f0:100e:2::1 > 2a01:5f0:100e:2::2: ICMP6, echo reply, seq 4096, length 10

29 packets captured
29 packets received by filter
0 packets dropped by kernel


[kristiankrohn]

Hmm, I have to admit that I'm absolutely befuddled; everything seems correct to me.
Hopefully someone else can jump in and offer advice.

But in the meantime: If I were sitting at your devices I would try two more things:

1) "tcpdump -nqi eth0 ip6" while pinging the ISP/gateway from the Mikrotik. (To see if the packets are actually sent to the gateway or somehow get lost on your machine.) You can also use "-i any" to detect if the packets are dropped or somehow get sent out via the wrong interface.

2) Since you've got a second Mikrotik which is supposed to be part of the network anyway, set it up and see if routing between the two devices is at least possible.

Again: If your linux machine sees the packets from your Mikrotik destined for the ISP gateway, I see no reason why it shouldn't relay them.

[liquidcz]

Hmm.. it confuse me alot.

So, look et this. Ping from Mikrotik to MyISP GW.

Code Select Expand


tcpdump -nqi eth0 ip6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
10:34:06.091211 IP6 fe80::21c:c0ff:febf:89bd > fe80::219:7ff:fea8:8340: ICMP6, neighbor solicitation, who has fe80::219:7ff:fea8:8340, length 32
10:34:06.091544 IP6 fe80::219:7ff:fea8:8340 > fe80::21c:c0ff:febf:89bd: ICMP6, neighbor advertisement, tgt is fe80::219:7ff:fea8:8340, length 24
10:34:11.125486 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 8704, length 10
10:34:11.126654 IP6 2a01:5f0:100e::1 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32
10:34:12.124291 IP6 fe80::219:7ff:fea8:8340 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32
10:34:12.135723 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 8960, length 10
10:34:13.123304 IP6 fe80::219:7ff:fea8:8340 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32
10:34:13.149108 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 9216, length 10
10:34:14.163547 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 9472, length 10
10:34:14.164535 IP6 2a01:5f0:100e::1 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32
10:34:15.163333 IP6 fe80::219:7ff:fea8:8340 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32
10:34:15.177148 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 9728, length 10
10:34:15.678183 IP6 fe80::a00:20ff:fec9:421c.546 > ff02::1:2.547: UDP, length 64
10:34:16.163337 IP6 fe80::219:7ff:fea8:8340 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32
10:34:16.182201 IP6 2a01:5f0:100e:2::2 > 2a01:5f0:100e::1: ICMP6, echo request, seq 9984, length 10

15 packets captured
15 packets received by filter
0 packets dropped by kernel


And this is my SYSCTL for IPV6:

Code Select Expand


net.ipv6.conf.default.autoconf = 0
net.ipv6.conf.default.accept_ra = 0
net.ipv6.conf.default.accept_ra_defrtr = 0
net.ipv6.conf.default.accept_ra_pinfo = 0
net.ipv6.conf.default.accept_source_route = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv6.conf.default.forwarding = 0
net.ipv6.conf.all.autoconf = 0
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.all.accept_ra_defrtr = 0
net.ipv6.conf.all.accept_ra_pinfo = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.all.forwarding = 1


[kristiankrohn]

Now I see something that shouldn't happen:

Code Select Expand

10:34:11.126654 IP6 2a01:5f0:100e::1 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32
10:34:12.124291 IP6 fe80::219:7ff:fea8:8340 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2a01:5f0:100e:2::2, length 32

The ISP gateway shouldn't try using neighbor solicitation to find an IP address that is in 2a01:5f0:100e::/48 but not in 2a01:5f0:100e::/64. Instead it should know that the non-zero subnets of your /48 can be reached via 2a01:5f0:100e::2.
I wouldn't be surprised if the netmask on ISP gateway was set to /48 where it should be /64. Maybe you could ask your ISP to verify the configuration.

[liquidcz]

Hmm.. MyISP gw is sets up as 2A01:5F0:100E::1/48.

Well, what is your suggestin now? Ask my ISP to set it up as 2A01:5F0:100E::1/64 and route 2A01:5F0:100E::/48 to my IP 2A01:5F0:100E::2/64 ?

[kristiankrohn]

Yes, exactly.

I actually don't quite understand how they could come up with the idea of putting a /48 on the interface. You'd have to somehow fake neighbor advertisements to enable subnetting -- and that can't be right.