problems with Linux IPv6 router (updated)

[andersman]

Hi,

I'm trying to set up a router for my network running Linux. At the moment it acts as a NAT router for IPv4 which works great, but since I now can use HE.net tunnels I wanted to set up IPv6 for my whole network with proper addresses using radvd.

After a lot of testing, it was suggested that I add the point to point IP address to my WAN interface instead of the tunnel interface. But this doesn't seem to work so well for me, the "default address" would be the point-to-point IP then. I'm not sure the wan0 ipv6 ip ever worked. If you want me to try this more I'll do it.

I have a /64 and a /48. The /48 is 2001:470:dc84::/48 (fixed, said /64) so I set my lan0 interface to use the address 2001:470:dc84:1::1/64 and in radvd I use "2001:470:dc84:1::/64"

I have two interfaces: wan0 for internet, lan0 for my local network. The tunnel interface is named he-ipv6.

My problems now are:

• I don't know how to set the default outgoing IP from the router, for example if I "wget http://ipv6.google.com". I heard "default via src blah blah" but that doesn't help me...
• I can't ping the clients on my network from the router until the client pings the router
• When I enable shorewall6, I can still access IPv6 hosts from the router, but not from any client on my network
• I have all my IPv6 addresses on the he-ipv6 interface. I don't understand if this is a good idea or not (am I meant to put it on my wan interface? lan interface? where!?)

Code Select Expand


# ping6 2001:470:dc84:1:a00:27ff:fe0d:d24c
PING 2001:470:dc84:1:a00:27ff:fe0d:d24c(2001:470:dc84:1:a00:27ff:fe0d:d24c) 56 data bytes
From 2001:470:dc84:1::1 icmp_seq=2 Destination unreachable: Address unreachable
From 2001:470:dc84:1::1 icmp_seq=3 Destination unreachable: Address unreachable
From 2001:470:dc84:1::1 icmp_seq=4 Destination unreachable: Address unreachable
From 2001:470:dc84:1::1 icmp_seq=5 Destination unreachable: Address unreachable
From 2001:470:dc84:1::1 icmp_seq=6 Destination unreachable: Address unreachable
From 2001:470:dc84:1::1 icmp_seq=7 Destination unreachable: Address unreachable

(this is where I started to ping the router from my PC)

64 bytes from 2001:470:dc84:1:a00:27ff:fe0d:d24c: icmp_seq=8 ttl=64 time=579 ms
64 bytes from 2001:470:dc84:1:a00:27ff:fe0d:d24c: icmp_seq=9 ttl=64 time=0.517 ms
64 bytes from 2001:470:dc84:1:a00:27ff:fe0d:d24c: icmp_seq=10 ttl=64 time=0.774 ms
64 bytes from 2001:470:dc84:1:a00:27ff:fe0d:d24c: icmp_seq=11 ttl=64 time=0.840 ms
^C
--- 2001:470:dc84:1:a00:27ff:fe0d:d24c ping statistics ---
11 packets transmitted, 4 received, +6 errors, 63% packet loss, time 10012ms
rtt min/avg/max/mdev = 0.517/145.498/579.864/250.781 ms


Code Select Expand


he-ipv6   Link encap:UNSPEC  HWaddr 5A-E3-F3-2E-00-00-60-AE-00-00-00-00-00-00-00-00
         inet6 addr: 2001:470:28:103::1/64 Scope:Global
         inet6 addr: 2001:470:27:103::2/64 Scope:Global
         inet6 addr: fe80::5ae3:f32e/128 Scope:Link
         UP POINTOPOINT RUNNING NOARP  MTU:1480  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

lan0      Link encap:Ethernet  HWaddr 00:1F:D0:5E:BD:BB
         inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
         inet6 addr: 2001:470:dc84:1::1/64 Scope:Global
         inet6 addr: fe80::21f:d0ff:fe5e:bdbb/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:58584932 errors:0 dropped:0 overruns:0 frame:0
         TX packets:33793017 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:84723123003 (80798.2 Mb)  TX bytes:5917983239 (5643.8 Mb)
         Interrupt:29 Base address:0xa000

wan0      Link encap:Ethernet  HWaddr 00:50:22:E1:2C:1D
         inet addr:90.227.243.46  Bcast:90.227.243.255  Mask:255.255.255.0
         inet6 addr: fe80::250:22ff:fee1:2c1d/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:5426663 errors:0 dropped:0 overruns:0 frame:0
         TX packets:4313652 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:5687191126 (5423.7 Mb)  TX bytes:1446099575 (1379.1 Mb)
         Interrupt:20


[cholzhauer]

The /48 is 2001:470:dc84::/64

Can you clarify?  I assume it should be 2001:470:dc84::/48?

What distro of Linux?

Unless radvd is working properly, you won't be able to ping6 google (unless you set a static IPv6 address on your interface.

[andersman]

I had it working, I will update with more details in a bit. Currently I'm struggling with shorewall6.

[andersman]

OK, I've updated my original post to explain the problems I have now. I also should post the relevant bits of my shorewall6 configuration:

Code Select Expand


# interfaces
#ZONE   INTERFACE       BROADCAST       OPTIONS
net     he-ipv6         detect          tcpflags,forward=1,nosmurfs
net     wan0            detect          tcpflags,forward=1,nosmurfs
loc     lan0            detect          tcpflags,forward=1


Code Select Expand


# policy
net all DROP info
loc net ACCEPT
all all REJECT info


Code Select Expand


# routestopped
#INTERFACE      HOST(S)                  OPTIONS
lan0            -


Code Select Expand


# rules
# accept PINGs from internet to fw
Ping(ACCEPT)    net     $FW

# accept all connections from the network to the firewall and internet
ACCEPT loc $FW
ACCEPT loc net

# accept all connections from the firewall to the network and internet
ACCEPT  $FW     loc
ACCEPT  $FW     net


Code Select Expand


# shorewall6.conf

...
IP_FORWARDING=On
...


Code Select Expand


# zones
fw      firewall
net     ipv6
loc     ipv6


[cholzhauer]

Well, I would remove all the extras first.  Remove the firewall, stop using RA, and just concentrate on getting traffic to flow from your router;  that makes setting your default route easy.

You never did mention what distro you're running.

[andersman]

Sorry, I run Arch Linux.

At the moment I don't run any firewall (at least on the IPv6 side), and I can use IPv6 from my computers. I do use RA though (which works with the firewall enabled, also).

When I turn off the firewall the INPUT/FORWARD chains are set to DROP, so I have to set them back to ACCEPT or I get the same issues... this might be helpful, I don't know. I would assume shorewall would handle it.

[jimb]

Hi,

I'm trying to set up a router for my network running Linux. At the moment it acts as a NAT router for IPv4 which works great, but since I now can use HE.net tunnels I wanted to set up IPv6 for my whole network with proper addresses using radvd.

After a lot of testing, it was suggested that I add the point to point IP address to my WAN interface instead of the tunnel interface. But this doesn't seem to work so well for me, the "default address" would be the point-to-point IP then. I'm not sure the wan0 ipv6 ip ever worked. If you want me to try this more I'll do it.

I have a /64 and a /48. The /48 is 2001:470:dc84::/48 (fixed, said /64) so I set my lan0 interface to use the address 2001:470:dc84:1::1/64 and in radvd I use "2001:470:dc84:1::/64"

I have two interfaces: wan0 for internet, lan0 for my local network. The tunnel interface is named he-ipv6.

My problems now are:

• I don't know how to set the default outgoing IP from the router, for example if I "wget http://ipv6.google.com". I heard "default via src blah blah" but that doesn't help me...

Not really sure how to parse this since I don't know how wget would tell you about a default route, etc.  But anyway, to set a default route for IPv6 you would use:  ip route add default via 2001:470:28:103::1

I also note that this command might fail since you appear to have set both the server and client IPv6 addresses on your tunnel interface:

he-ipv6   Link encap:UNSPEC  HWaddr 5A-E3-F3-2E-00-00-60-AE-00-00-00-00-00-00-00-00
         inet6 addr: 2001:470:28:103::1/64 Scope:Global
         inet6 addr: 2001:470:27:103::2/64 Scope:Global
         inet6 addr: fe80::5ae3:f32e/128 Scope:Link

You need to remove the ::1 address, as that is the IP address of the HE server on the other side of the tunnel.  You only want the ::2 address on the tunnel interface.

• I can't ping the clients on my network from the router until the client pings the router

This is likely to do with shorewall.  The only other explanation is that ND is failing somehow.  Please include the output of these commands so I can figure out how shorewall has your iptables/ip6tables configured.

ip6tables -n --list
iptables -n --list
iptables -t nat -n --list

• When I enable shorewall6, I can still access IPv6 hosts from the router, but not from any client on my network

Again, likely shorewall configuration issues.  It could also be lack of a default route on your client machines (presuming you're not running radvd or it's not working), or that your DNS server doesn't return or is filtering AAAA records.  You can obviously eliminate the DNS term from the equation if you test by pinging by IPv6 addres.

• I have all my IPv6 addresses on the he-ipv6 interface. I don't understand if this is a good idea or not (am I meant to put it on my wan interface? lan interface? where!?)

This is wrong.  You shouldn't have "all" of your IPv6 addresses on the he-ipv6 interface as I pointed out above.

The "Client IPv6 address" should be on your tunnel interface (he-ipv6).
The Routed /64, or a /64 subnet of your routed /48 should be on your LAN interface.
The WAN interface shouldn't have an IPv6 address at all, since your ISP doesn't support IPv6 natively.  If it did, you wouldn't need an HE tunnel.

Also, make certain when you set up your tunnel IPv6 addresses that you source the traffic from the WAN interface.  In other words, make sure "local 90.227.243.46 dev wan0" is included in the "ip tunnel add" command line.

[andersman]

Not really sure how to parse this since I don't know how wget would tell you about a default route, etc.  But anyway, to set a default route for IPv6 you would use:  ip route add default via 2001:470:28:103::1

wget was a bad example, because you would have to be able to check the access log on the server to see what address it came from. But thanks, that should do it.

ip6tables -n --list
iptables -n --list
iptables -t nat -n --list

I've run these and pasted here:

http://fgsfd.se/~anders/ip6tables_list.txt
http://fgsfd.se/~anders/iptables_list.txt
http://fgsfd.se/~anders/iptables_nat_list.txt

Again, likely shorewall configuration issues.  It could also be lack of a default route on your client machines (presuming you're not running radvd or it's not working), or that your DNS server doesn't return or is filtering AAAA records.  You can obviously eliminate the DNS term from the equation if you test by pinging by IPv6 addres.

I'm using radvd and the machines are receiving addresses properly. The DNS isn't filtering out AAAA records, verified this with nslookup. But this seems to work now anyway (???), really odd - probably related to some other change I had done yesterday.

This is wrong.  You shouldn't have "all" of your IPv6 addresses on the he-ipv6 interface as I pointed out above.

The "Client IPv6 address" should be on your tunnel interface (he-ipv6).
The Routed /64, or a /64 subnet of your routed /48 should be on your LAN interface.
The WAN interface shouldn't have an IPv6 address at all, since your ISP doesn't support IPv6 natively.  If it did, you wouldn't need an HE tunnel.

Also, make certain when you set up your tunnel IPv6 addresses that you source the traffic from the WAN interface.  In other words, make sure "local 90.227.243.46 dev wan0" is included in the "ip tunnel add" command line.

Okay, thanks. Your help is much appreciated! I'll try this and see where it leads me.

[andersman]

Here are the commands I run to set up my IPv6 access now:

Code Select Expand


+ ip tunnel add he-ipv6 mode sit remote 216.66.80.90 local 90.227.243.46 dev wan0 ttl 255
+ ip link set he-ipv6 up
+ ip addr add 2001:470:27:103::2/64 dev he-ipv6
+ ip addr add 2001:470:dc84:1::1/64 dev lan0
+ ip route add ::/0 dev he-ipv6
+ ip -6 route add default via 2001:470:28:103::1


I don't know if the last bit is correct - but it isn't working. For example, if I go on freenode I get:

01:23 -!- anders [anders@andersman-2-pt.tunnel.tserv24.sto1.ipv6.he.net]

I tried adding it like "ip route add ::/0 dev he-ipv6 via 2001:470:28:103::1" etc, in various combinations, but I just get "RTNETLINK answers: No route to host".

... and now ipv6 access from my client machines is gone again :-(

Code Select Expand


Pinging ipv6.l.google.com [2a00:1450:8001::93] with 32 bytes of data:
Destination host unreachable.
Destination host unreachable.
Destination host unreachable.
Destination host unreachable.

Ping statistics for 2a00:1450:8001::93:
   Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),


[jimb]

Not really sure how to parse this since I don't know how wget would tell you about a default route, etc.  But anyway, to set a default route for IPv6 you would use:  ip route add default via 2001:470:28:103::1

wget was a bad example, because you would have to be able to check the access log on the server to see what address it came from. But thanks, that should do it.

ip6tables -n --list
iptables -n --list
iptables -t nat -n --list

I've run these and pasted here:

http://fgsfd.se/~anders/ip6tables_list.txt
http://fgsfd.se/~anders/iptables_list.txt
http://fgsfd.se/~anders/iptables_nat.list

OOPS.  I should have included a -v flag so I could see the interface in the listings too.  Also, the nat table is missing (404).

The rules shorewall produce are pretty gnarly, tons of chains all calling each-other, etc, sometimes multiple times in the same sequence of chains!  Talk about overhead!  From glancing at it, it appears like it allows outbound ICMPv6 from the linux box.  But I can't fully follow it without the -v info, so, could you do it again with:

ip6tables -n -v --list
iptables -n -v --list
iptables -t nat -n -v --list

This will include the interface in the rule lists so I can tell which interface a rule might apply to.

Again, likely shorewall configuration issues.  It could also be lack of a default route on your client machines (presuming you're not running radvd or it's not working), or that your DNS server doesn't return or is filtering AAAA records.  You can obviously eliminate the DNS term from the equation if you test by pinging by IPv6 addres.

I'm using radvd and the machines are receiving addresses properly. The DNS isn't filtering out AAAA records, verified this with nslookup. But this seems to work now anyway (???), really odd - probably related to some other change I had done yesterday.

This is wrong.  You shouldn't have "all" of your IPv6 addresses on the he-ipv6 interface as I pointed out above.

The "Client IPv6 address" should be on your tunnel interface (he-ipv6).
The Routed /64, or a /64 subnet of your routed /48 should be on your LAN interface.
The WAN interface shouldn't have an IPv6 address at all, since your ISP doesn't support IPv6 natively.  If it did, you wouldn't need an HE tunnel.

Also, make certain when you set up your tunnel IPv6 addresses that you source the traffic from the WAN interface.  In other words, make sure "local 90.227.243.46 dev wan0" is included in the "ip tunnel add" command line.

Okay, thanks. Your help is much appreciated! I'll try this and see where it leads me.

Yeah that'd eliminate at least one possible issue.

[jimb]

Here are the commands I run to set up my IPv6 access now:

Code Select Expand


+ ip tunnel add he-ipv6 mode sit remote 216.66.80.90 local 90.227.243.46 dev wan0 ttl 255
+ ip link set he-ipv6 up
+ ip addr add 2001:470:27:103::2/64 dev he-ipv6
+ ip addr add 2001:470:dc84:1::1/64 dev lan0
+ ip route add ::/0 dev he-ipv6
+ ip -6 route add default via 2001:470:28:103::1


I don't know if the last bit is correct - but it isn't working. For example, if I go on freenode I get:

01:23 -!- anders [anders@andersman-2-pt.tunnel.tserv24.sto1.ipv6.he.net]

I tried adding it like "ip route add ::/0 dev he-ipv6 via 2001:470:28:103::1" etc, in various combinations, but I just get "RTNETLINK answers: No route to host".

... and now ipv6 access from my client machines is gone again :-(

Code Select Expand


Pinging ipv6.l.google.com [2a00:1450:8001::93] with 32 bytes of data:
Destination host unreachable.
Destination host unreachable.
Destination host unreachable.
Destination host unreachable.

Ping statistics for 2a00:1450:8001::93:
   Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),


You have a mismatch with your interface address and the default route you add.
ip addr add 2001:470:27:103::2/64 dev he-ipv6
ip -6 route add default via 2001:470:28:103::1

Note the difference in bold.  Make sure you use the correct IPv6 address.  I'm not sure if it's the 27 or the 28 one.  It'll be on the tunnelbroker page for your tunnel.  

EDIT: I believe the 27 address will be the correct one.  If your tunnel server works the same as the others, the lower # is the client IPv6, which should be the only IPv6 address on your he-inet6 interface.  The 28 address is your routed /64, which you can use anywhere (But not on the he-inet6 interface).  The way it's supposed to work is like this:  If you only have a single LAN in which you want to run IPv6, you only need that routed /64, and it should go on your LAN interface.  If you have more than one LAN you want to run IPv6 on behind your router, you need the /48.  But since you've already set up your LAN with a subnet from the /48, it doesn't really hurt anything.  Leave it.

The last two route commands are duplicates.  You don't need both, just one.  Either should work (the 2nd after the address is corrected).  If the first doesn't work for whatever reason, omit it and try the 2nd form.

FYI:  ip route add ::/0 dev he-ipv6 says "send any IPv6 traffic you don't have a specific route for out of the he-ipv6 interface."  ip -6 route add default via 2001:470:28:103::1 says "send any IPv6 traffic you don't have a specific route for to the gateway 2001:470:28:103::1", which the router table will have a connected route for pointing it to the he-ipv6 interface.  Also, ::/0 and default are synonymous, if you didn't know.

[andersman]

Here is the new ip6tables/iptables output:

http://fgsfd.se/~anders/ip6tables.txt
http://fgsfd.se/~anders/iptables.txt
http://fgsfd.se/~anders/nat.txt

And as for the route, that does make sense and was what I thought at first. I now have "ip route add default dev he-ipv6 via 2001:470:27:103::1", which works.

I understand the use of the /48 now - I should probably set it up so I use the /64 instead but as it is already set up, so as you say, I'll leave it for now...

The problem with the "outgoing address", not sure if I explained it correctly. If I log in to the router now, run "wget http://some.ipv6.server.example.net/file.txt", that server will see my point to point tunnel IP in the logs. I would like it to see an address from my assigned /64 (or /48). When I added more addresses to the tunnel interface, that worked, but it's obviously not the way to go.

[andersman]

It all seems fine now, with the exception of the "default used address" with connections originating from the router... I can ping IPv6 addresses from my computers and they get IPs from my assigned /64. Thanks for the help! :D

I've uploaded my IPv6 scripts to http://fgsfd.se/~anders/ipv6-scripts/ in case anyone wants to have a look.

[jimb]

Your iptables look fine.  I'm not sure why it was not permitting a ping6 initiated from your FW to the LAN before, but the rules I see certainly don't forbid that.

As for the 'default address' thing, this is the normal behavior.  A system will use the IP or IPv6 address of the interface closest to the destination as a source address unless you tell the program not to.  So, by default, whenever you go to the IPv6 internet from your router, it will use the outgoing interface to the internet for the source address, which is your he-inet6 interface.

If for instance you want wget to use a your LAN interface IPv6, you can specify it with the "--bind-address=ADDRESS" argument.

Daemons and so forth will typically listen on all interfaces, so they can be addressed by any IPv6 on the box.  But if you want it to initiate traffic from a particular address, you'll have to configure it to use that address.

I think it may be possible to set a default system policy using the ip addrlabel and/or gai.conf file, but I've never tried to do this.  You may want to have a look at this http://people.redhat.com/drepper/linux-rfc3484.html

But really, since the tunnel interface IPv6 is a perfectly valid address to use (many don't even use the routed IPv6s at all).  The only caveat is that HE controls the RDNS for that address, so if you want your box to connect out with an IPv6 for which you control the RDNS, you'd have to force it to use an IP out of the routed IPv6s they assign you.

[andersman]

OK, well that isn't a problem. Thank you again for all of your assistance, and I intend to play more with IPv6 and learn :)