Subnetting a /48 into /64s based on IPv6 link or MAC address

[cocodude]

Hi,

My problem

I have a /48 (native or tunneled - shouldn't matter) going to eth0.

There are many virtualised instances running, which are bridged against eth0.

I want to assign a unique /64 prefix to each of these, so based on the client's IPv6 link or MAC address.

My questions

Is this possible to do with something like radvd? I can see that it's possible to advertise different prefixes based on the interface, but I can't find a way to advertise specific prefixes to specific clients. Is this beyond the scope of radvd?

If radvd can't be used here, is some DHCP v6 server the way forward? I'd rather make this at least semi-automatic rather than static.

When router advertisements are used, IPv6 settings are configured automatically on clients when a link is active. With DHCP v6, the client needs to send out a DHCP request. For most Linux distros, am I right in thinking that a DHCP v6 request isn't sent out by default and that the user would need to add something to their network scripts? At the moment, IPv4 is configured statically, so I would be surprised if the clients sent a DHCP v6 request 'just in case'.

Many thanks for your help,

Marc

[cholzhauer]

You're right when you say that you'll have to look into DHCPv6 for this.  Just like DHCP for IPv4, you can reserve addresses for specific clients.

[cocodude]

Thanks for that. I've now got the RA providing the prefix and options, and DHCPv6 sending an IP address and DNS servers. This brings me to a new problem though.

I want to assign a specific /64 to each VPS. DHCPv6 seems to use a DUID by default, but the DHCPv6 server does not know what DUID corresponds to what VPS (i.e. the DUID is generated in the client VPS with the server having no idea what the DUID is).

Is there a way to configure something like the Wide DHCPv6 Server to hand out certain IPv6 addresses depending on MAC address or even IPv6 link local addresses? If not, and if this sounds like I'm going about it the wrong way, do you have any suggestions?

[cocodude]

Hi again,

It looks like no-one has an answer yet.

If so, does anyone know why radvd (and similar) are so simple? Is there any logical reason why a RA program couldn't be written that offers a prefix on a more complicated basis (e.g. a different /64 prefix depending on the MAC address)? I'm aware that DHCPv6 likes doing things against DUIDs, but this seems like a sensible reason to base things on the MAC address.

I'm thinking of hacking the radvd code you see to do some interesting manipulation based on MAC addresses.

Marc

[kasperd]

Is there any logical reason why a RA program couldn't be written that offers a prefix on a more complicated basis (e.g. a different /64 prefix depending on the MAC address)?

I'm not sure I completely understand your question, but I think there is going to be some technical problems with it.

You want it to be such that when two nodes on the LAN asks for a router, the same router will reply with two different prefixes to those two nodes. But I think radvd also periodically multicasts the announcement to all IPv6 nodes on the LAN. Which one should it multicast? Once it multicasts one, all the nodes will pick up that announcement even if they already have a different prefix. That will give you multiple IPv6 addresses on each interface.

If you completely stop multicasting the announcement, what are the nodes on the LAN going to do? They get online ask for a router, and gets a reply. But then the nodes will never see the multicasts from the router. I wouldn't be surprised if some nodes under such circumstances would think the router has gone away. Hopefully they will ask for routers at least a couple of times before they declare it dead.

I don't get why you would want this sort of separation in the first place. What would the benefit be?

[cocodude]

Ah yes, multicasting. You're right - it wouldn't be able to multicast different prefixes.

Let me first explain why the separation is a good thing. On one server, I will have multiple customers each with a VPS. Sometimes these customers would like a /64 (I have a /48 going to the server) so, to make it as easy as possible for them to get running with IPv6, I would like to have each server automatically set up with an address from their own /64.

Please me know if you think this would work:

    1. radvd advertises the defaulr router as normal and hints that DHCPv6 should be used to get the IP address and other information.
    2. Each VPS uses DHCPv6.
    3. I hack a DHCPv6 server so that it can assign a specific /64 based on MAC address (as opposed to DUIDs, which I will not know).

I understand why DUIDs are used for DHCPv6 now, but it seems to be a bit of a shame that this is the only method for host-specific setup and that MAC address matching doesn't seem to be in use. If you can name a DHCPv6 server that works on MAC address matching, please let me know!

[kasperd]

Sometimes these customers would like a /64

Why do they want a /64? What do they expect to happen with that /64?

If I was a customer asking for a /64 for my node (virtual or physical), I would expect that I get one IPv6 address which is not assigned out of my /64 for my external interface, and I get everything in my own /64 routed to me using that single IPv6 address as gateway. My node should be free to assign that entire /64 to an internal interface (virtual or physical) or to use in any other way as it please.

So I think you should consider going the following route:

As a starting point you use radvd to assign each node a single IPv6 address, and all of them get IPv6 addresses from the same /64. SLAAC doesn't consume all addresses in that /64. You will have plenty of addresses left over for DHCPv6 and static configurations, should you need it.

With that the customers get IPv6 on their nodes, but only a single IPv6 address. (They can actually get multiple addresses this way as well, but not an entire prefix.)

For those customers who ask for a prefix, you assign them a /64 and either delegate the prefix through DHCPv6 or by static configuration. In your routing table you can use any of the addresses assigned to that node as the next hop for the /64 prefix. There is no technical reason you couldn't use the link local address of that node has the next hop for the /64 prefix.

In theory it would even be possible to use the MAC address directly as the next hop such that the router doesn't even have to know the IP address of the interface it is forwarding the packet to. But I don't know if any routing engine actually supports such operation.

The receiving node doesn't know how the router found which MAC address to forward the packet to. The packet does not tell which of the IPv6 addresses on the interface (if any) it performed neighbour discovery on to forward the packet.

It might even be possible to create static IPv6 address to MAC mapping for neighbours, similar to how you'd create a static ARP entry. That would give you an IPv6 address that you can use in your routing table. It doesn't matter that that address isn't even assigned to the receiving interface. Those addresses would never be visible to anything outside of the routing engine itself. If you go that route, you could for example use fd92:ce71:1691:1494::/64 for those static neighbour mappings.

The last suggestion will become irrelevant if you know how the individual nodes use SLAAC.

[amcintosh]

I'm a noob exploring ipv6 on several virtual boxes.  I'm currently working with multiple adapter cards (physical, host only, named in the virtual environment) and how that interacts with radvd.  Having disclosed that ...

I almost always fight new systems the first several months, then finally settle down and decide the designers actually did know something.  At this epoch, I have tried to get ahead of that by figuring out how to do the minimal magic work and use the system as it came to me.

In this case, I would look for the DUID as it comes in.  I am assuming it stays the same, and is based loosely on a MAC address, as that is how the Cisco Linksys E1200 seems to behave (instruct me gently if I'm wrong here.)

I would set up a MySQL environment to grab the next available /64 and associate it with the DUID.  I'd have a "last used" field in the table, and always use the "longest abandoned" DUID when forced to rotate.  You could initialize the "last used" filed to 1-1-70 0:0:1, 1-1-70 0:0:2, ... if that amuses you.

Probably, just an autoindex field in the MySQL table would work for the filler between the /48 and the /64.  So I guess you're limited to 32,768 customers for the time being.

[kasperd]

It might even be possible to create static IPv6 address to MAC mapping for neighbours, similar to how you'd create a static ARP entry. That would give you an IPv6 address that you can use in your routing table. It doesn't matter that that address isn't even assigned to the receiving interface. Those addresses would never be visible to anything outside of the routing engine itself. If you go that route, you could for example use fd92:ce71:1691:1494::/64 for those static neighbour mappings.

I just tested this on Ubuntu and it works. Here are the commands I used

Code Select Expand

ip route add fd92:ce71:1691:1494::/64 dev wlan0
ip neigh add fd92:ce71:1691:1494::10 lladdr 00:xx:xx:11:d5:c2 dev wlan0
ip route add 2001:470:1f0b:1da2::/64 via fd92:ce71:1691:1494::10That way you can route any prefix to a MAC address. Obviously you have to change the above to use your own prefix and MAC address.

ip route add fd92:ce71:1691:1494::/64 dev wlan0 creates a /64 that I am not actually routing anywhere. I use seven random bytes in the network address. This command is used just once regardless of the number of routes I would set up using this method.

ip neigh add fd92:ce71:1691:1494::10 lladdr 00:xx:xx:11:d5:c2 dev wlan0 is repeated once for each destination MAC address. Of course the host portion of the IPv6 address has to be different. If I had split the IPv6 address into a /32 network prefix and 96 bits for the host address, I could have used the MAC address directly in the IPv6 address to make the entries easier to read. Unfortunately that would violate RFC 4193. Site local addresses would be suitable for the purpose, but those are deprecated.

ip route add 2001:470:1f0b:1da2::/64 via fd92:ce71:1691:1494::10 creates the actual routing table entry for the /64 that I want to route to the above MAC address. One such line is used for each prefix that I want routed to a MAC address.

Repeating the above example with site local addresses for readability:

Code Select Expand

ip route add fec0::/32 dev wlan0
ip neigh add fec0::00:xx:xx:11:d5:c2 lladdr 00:xx:xx:11:d5:c2 dev wlan0
ip route add 2001:470:1f0b:1da2::/64 via fec00::00:xx:xx:11:d5:c2

[cocodude]

Why do they want a /64? What do they expect to happen with that /64?

They'd want a /64 because of any reason someone wants more than one IP address. For instance, they might be hosting multiple sites and one one IP address per site.

As a starting point you use radvd to assign each node a single IPv6 address, and all of them get IPv6 addresses from the same /64. SLAAC doesn't consume all addresses in that /64. You will have plenty of addresses left over for DHCPv6 and static configurations, should you need it.

For those customers who ask for a prefix, you assign them a /64 and either delegate the prefix through DHCPv6 or by static configuration. In your routing table you can use any of the addresses assigned to that node as the next hop for the /64 prefix. There is no technical reason you couldn't use the link local address of that node has the next hop for the /64 prefix.

I agree that your suggest could work well indeed. I just argue that it could be easier to automatically assign a /64 to each customer without having to make configuration changes if the customer decides they want one. As said, I've got a /48 on that server so there are plenty of /64s to go around for each customer to have one.

Having the default route as the other side of the link local address (to the dom0/master domain, as radvd or similar would hand out) would work I imagine instead of configuring routing through the single IP address in the shared /64. Perhaps this would be a lot easier than setting up manual static routes?

Is your suggestion the standard in the hosting industry? I do agree that it would work, and I also agree that only a few customers would actually ask for a /64. However, I do feel that just sorting out the configuration up front and handing out /64s is easier for me. I do appreciate your comments and thank you for what you've provided so far.

I almost always fight new systems the first several months, then finally settle down and decide the designers actually did know something.

I agree with you here! Hence my asking whether this would be a good idea. I still don't understand why MAC address matching just doesn't seem to be supported in DHCPv6 servers though!

I would set up a MySQL environment to grab the next available /64 and associate it with the DUID.  I'd have a "last used" field in the table, and always use the "longest abandoned" DUID when forced to rotate.  You could initialize the "last used" filed to 1-1-70 0:0:1, 1-1-70 0:0:2, ... if that amuses you.

That would definitely work, I agree, but it involves significant coding changes to a DHCPv6 server anyway (unless you know of a hugely configurable one). I think that static matching on a MAC address would be easier if I need to hack in some DHCPv6 server code anyway.

[kasperd]

I just argue that it could be easier to automatically assign a /64 to each customer without having to make configuration changes if the customer decides they want one. As said, I've got a /48 on that server so there are plenty of /64s to go around for each customer to have one.

No problem with that. I think it makes perfect sense to give customers a /64 by default. Just keep two things in mind. If at some point you decide that the /48 needs to be split across multiple servers, it will be convenient to have the /64s that you did assign be located closely together. I don't know how many customers you expect to have on that server. If you do get to for example 8000 customers on that server, try to do a survey on how many customers are using the /64 for something.

Even as you do give each virtual node a /64 you are still going to need a way to route between them.

Having the default route as the other side of the link local address (to the dom0/master domain, as radvd or similar would hand out) would work I imagine instead of configuring routing through the single IP address in the shared /64. Perhaps this would be a lot easier than setting up manual static routes?

AFAIK radvd cannot delegate prefixes. It can just tell nodes what prefix is on a specific link.

I assume you can setup separate virtual network interfaces between dom0 and the individual virtual nodes. From a security perspective it would be better to do it that way. Once the connections to the individual nodes are on different interfaces you can of course use router advertisements to announce a different prefix on each of those interfaces.

But again that is still just a /64 for the link between dom0 and the customer node. They are not actually getting a /64 routed to their node, they are getting a /64 in which they can use SLAAC to consume as many addresses as they like.

Is your suggestion the standard in the hosting industry?

I don't know what is standard. Neither do I know if the industry has agreed on a standard.

My guess is that delegating a prefix to individual servers (physical or virtual) is not standard. They are more likely manually configured as needed.

Having each customer on logically independent links is a good idea from a security perspective, and it more or less force you to allocate a /64 to each link. As far as I remember xen does make it independent links by default, and you have to explicitly bridge them in dom0 if you want them to be one link. On a physical network you can make independent links through the use of VLANs.

Do you think it would better suit your needs to allocate a /64 for the link between dom0 and each customer, and then not route a /64 to each customer. If you don't route a /64 to each customer, then you don't need to mess with DHCPv6 and routing.

I agree with you here! Hence my asking whether this would be a good idea. I still don't understand why MAC address matching just doesn't seem to be supported in DHCPv6 servers though!

I seem to recall some ISC representative stating that there are lots of features that they'd like to add, but they'll have to prioritize, and they need input from users of the DHCPv6 server to decide how to prioritize.

[cocodude]

I just argue that it could be easier to automatically assign a /64 to each customer without having to make configuration changes if the customer decides they want one. As said, I've got a /48 on that server so there are plenty of /64s to go around for each customer to have one.

No problem with that. I think it makes perfect sense to give customers a /64 by default. Just keep two things in mind. If at some point you decide that the /48 needs to be split across multiple servers, it will be convenient to have the /64s that you did assign be located closely together. I don't know how many customers you expect to have on that server. If you do get to for example 8000 customers on that server, try to do a survey on how many customers are using the /64 for something.

Indeed, I agree that keeping them all as contiguous as possible is a good thing to do. It won't be hard with 2^16 /64s to play with!

I assume you can setup separate virtual network interfaces between dom0 and the individual virtual nodes. From a security perspective it would be better to do it that way. Once the connections to the individual nodes are on different interfaces you can of course use router advertisements to announce a different prefix on each of those interfaces.

Well, I will be bridging them all to eth0, but Xen does create a virtual interface for each domU to brige to eth0. It's a bit tricky to configure radvd to offer prefixes on dynamically changing interfaces though, but I'm hoping just a router advertisement + DHCPv6 will do the trick. I'll have a play and see what works.

But again that is still just a /64 for the link between dom0 and the customer node. They are not actually getting a /64 routed to their node, they are getting a /64 in which they can use SLAAC to consume as many addresses as they like.

I think it should work as I expect as all virtual interfaces should be bridged to the dom0 interface with separation provided by ip6tables rules. I'll have a play anyway - it can be hard to see potential network issues unless you try!

As far as I remember xen does make it independent links by default, and you have to explicitly bridge them in dom0 if you want them to be one link. On a physical network you can make independent links through the use of VLANs.

The default script that Xen users seem to use is vif-bridge, which does simply bridge all virtual interfaces to the one 'real' interface.

Do you think it would better suit your needs to allocate a /64 for the link between dom0 and each customer, and then not route a /64 to each customer. If you don't route a /64 to each customer, then you don't need to mess with DHCPv6 and routing.

Possibly, but I'm going to give this a try. If it does work out, and I can get a DHCPv6 server offering a unique /64 to each domU, I believe I won't have to do any further configuration going forwards. I've been chatting to the developers of one DHCPv6 server and will see how difficult it would be to get link-specific unique /64s handed out.

Thanks again for your valued advice.

[kasperd]

The default script that Xen users seem to use is vif-bridge, which does simply bridge all virtual interfaces to the one 'real' interface.

Yeah, I recall playing a bit around with that five years ago. However I am thinking that setup might actually not be suitable for your use case.

From a switch you have an Ethernet link to your physical server. By bridging, the rest of the world will see your physical server as another switch with a number of hosts connected to it. That way all the handling of addresses can be pushed from your server one level up to the network infrastructure. And you can move virtual hosts between physical hosts and by moving the MAC address along, it will be completely transparent to anything working at a level above the switches.

However it seems you don't want the entire management of addresses to happen outside of your physical server. If you want all that management to happen on the server, with different prefixes routed to different VMs, I don't see what you would benefit from bridging.

It's a bit tricky to configure radvd to offer prefixes on dynamically changing interfaces though

Then it is convenient that SLAAC is stateless. When interfaces change you could have a script to generate a new configuration for radvd and restart it. I do not know details of radvd, so even if I know what is possible with the protocols, I cannot tell you if it can be achieved with radvd.