netflix revisit

[cshilton]

After working well for a couple of years, my DNS workaround for Netflix failed again last weekend. After digging around, I discovered that Netflix is now rolling services on AWS instances behind AWS names. My method was to view all outgoing DNS queries from the box affected by this: my Apple TV. I used tcpdump -ni eth1 ether host my:apple:tv:xx:xx:xx and udp port 53 to view the queries that the Apple TV was doing. I caught two things of interest: firstly, Netflix is using some system called some-prefix.dradis.netflix.com. If they are battlestar galactica fans, I would assume that those boxen are the ones detecting that our devices are being an HE tunnel. Second, the DNS resolution chain for some of the boxes that they have deployed goes like this:

     some-identfier.some-network.netflix.com -> (CNAME) some-identier.aws.com -> (A, AAAA) records.

In short the issue with DNS blocking has become that the first lookup will give you an address to lookup in Amazon's DNS and it won't block the return of the AAAA record. You're Apple TV prefers it's IPv6 address so after that DNS resolution, it will hit Netflix from your tunneled address triggering their "You are behind a proxy, we can't serve you" flag.

I fixed this in the short term by finding the IPv6 addresses that my AppleTV was trying to hit and just outright blocking them:

PF:

     block out return on v6_egress to <netflix-v6-space>

Note well the block out return here is much better than drop. The AppleTV will immediately try again using IPv4 by happy eyeballs.

In the long term, the downside of this is that I have to maintain a table of netflix's IPv6 space which can change at a moments notice. I'm really tempted to change from using SLAAC to DHCPv6 and then blocking all outbound IPv6 from my Apple TV.

[cholzhauer]

Why would you have to change from SLAAC to DHCPv6 to block the oubound traffic?  I assume it's because you'd just block the IP of your device, but the SLAAC address shouldn't change, right?

[cshilton]

Why would you have to change from SLAAC to DHCPv6 to block the oubound traffic?  I assume it's because you'd just block the IP of your device, but the SLAAC address shouldn't change, right?

The issue is privacy extensions and knowing what the current IP address is on each of my Apple TVs. Under privacy extensions, the IPv6 addresses do change. So, I'm thinking about moving from SLAAC to DHCPv6 because lacking a way to convert the ethernet address of my Apple TV into the IPv6 address that it's currently using, I would instead move to an address protocol that allows me to force certain devices to have certain IPv6 addresses. Then listing those addresses in a firewall table that blocked traffic over IPv6 to netflix is simple.

Here's a question: Is ndp a cache like arp? Or is it something else? If ndp is a cache like ARP then I expect that I cannot reasonably translate Ethernet address  into a temporary IPv6 allocation in a script. That leads to this question: Given an ethernet address, can I figure out which IPv6 address a specific device is using?

The root problem here is that the IP addresses on both sides of the equation are dynamic.

[cholzhauer]

Ah yes, I forgot about privacy extensions.

[cshilton]

It looks like I could maintain a table of IPv6 addresses that the Apple TV's are using by running:

     ndp -a | egrep '(mac[0]|mac[1]...)'

on the firewall and throwing that addresses that I find into a table.

[nemesis101fc]

We ran into this a few weeks ago as well, after having worked perfectly for a year or more.  Setup is unbound, as site resolver, forwarding netflix domain requests to a bind instance that strips AAAA responses.  After some digging with tcpdump, found, as did the original poster, that some of the netflix responses were now cnames to aws, and other, domains.  These are then resolved in the usual way and AAAA responses are returned breaking netflix.  We fixed it by getting unbound to forward the specific cname destinations to the bind stripping instance.  This has been working, for us, for a couple of weeks now.  We are located in Northern Britain and obviously the cnames returned will be region specific.  Just thought I'd list the domains/hosts we are forwarding for AAAA stripping in case this helps anyone else.
netflix.com.
netflix.net.
nflxext.com.
nflximg.net.
nflximg.com.
nflxvideo.net.
nflxso.net.
e13252.dscg.akamaiedge.net.
dualstack.ichnaea-vpc0-1803858966.eu-west-1.elb.amazonaws.com.
dualstack.beaconserver-ce-vpc0-1537565064.eu-west-1.elb.amazonaws.com.
dualstack.wwwservice2--frontend-san-vpc0-138074574.eu-west-1.elb.amazonaws.com.
dualstack.wwwservice--frontend-san-vpc0-445693027.eu-west-1.elb.amazonaws.com.
dualstack.ichnaea-web-323206729.eu-west-1.elb.amazonaws.com.

Really sucks that we have to jump through hoops like this to watch content we've paid for especially as netflix lists our /48 as being from the UK!!!!

[thunderwolf]

The FQDNs would suggest that these hosts may be dynamic and/or temporary in nature.  Have you had to change the FQDNs since October?  I wonder if since these are done originally as a query to netflix.com which provides the resultant CNAME records they could simply forward the associated CNAMEs for A only resolution instead of resolving them as independent FQDNs?

Presently, I have just blocked the two North America AWS AS ranges to resolve the issue here.  But if these FQDNs are stable I may resort to doing that instead.  If they're stable though why does netflix not assign them names under netflix.com instead of under amazonaws?

US (West) Android device aws FQDNs -

dualstack.apiproxy-appboot-android-vpc0-1984084159.us-west-2.elb.amazonaws,com
dualstack.apiproxy-device-prod-nlb-1-54e80724f8c6ba2d.elb.us-west-2.amazonaws,com
dualstack.apiproxy-device-prod-nlb-2-9a34ae4608b5939d.elb.us-west-2.amazonaws,com

Media PC for Television is IPv4 only which eliminates any potential streaming issues there.

[tjeske]

Does somebody have a more definitive list? I got complains about Netflix complaining about VPN. However, I don't serve DNS - that is up to some other entity in my setup - and I can't intercept it. Being in Europe, so far blocking 2a01:578:3::/48 used to suffice. But apparently not so anymore. What other IP range would I need to block?

EDIT: I have also heard reports that Netflix stopped blocking HE tunnels altogether? Can anyone confirm this?
EDIT2: I am continuing to hearing those reports. Is the Netflix-block over?
EDIT3: Tried it myself now. Seems to depend on the series. Some work, some don't.

[eancode]

For more detailed list of Netflix prefixes, use the BGP toolkit.

https://bgp.he.net/AS2906#_prefixes6

[tjeske]

@eancode: Unfortunately, that doesn't help as Netflix is using AWS (I think it was AWS?) now many times with Amazon's IP ranges.