Hurricane Electric's IPv6 Tunnel Broker Forums

Tunnelbroker.net Specific Topics => Questions & Answers => Topic started by: bimmerdriver on October 10, 2016, 12:01:09 PM

Title: Possible Routing Problem
Post by: bimmerdriver on October 10, 2016, 12:01:09 PM
I'm using an HE tunnel for ipv6, connected through a pfsense router. ipv6-test.com and test-ipv6.com both indicate ipv4 and ipv6 are workign properly. However, over the past few days, I've noticed a few websites that aren't accessible, e.g., mail.yahoo.com and speedtest.xfinity.com. tracert -4 and -6 both work, but I can only access the sites when I disable ipv6 on the client, which is windows. I've also noticed that mobiles connected to the wifi are having the same problem. Changing to a different browser doesn't make any difference either. To eliminate the router, I upgraded to the latest version. It made no difference. To eliminate the tunnel, I tested using a different pfsense router, running a development version of pfsense on another computer, which is configured to use native ipv6. The problem doesn't occur on this configuration, so it's looking like there is a routing problem with the tunnel. Any suggestions what to do with this?
Title: Re: Possible Routing Problem
Post by: broquea on October 10, 2016, 12:07:25 PM
If traces work but HTTP(S) doesn't, that isn't a routing issue, that sounds more like PMTUD. Try tuning your MTU on both sides of the tunnel.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 10, 2016, 12:54:00 PM
Quote from: broquea on October 10, 2016, 12:07:25 PM
If traces work but HTTP(S) doesn't, that isn't a routing issue, that sounds more like PMTUD. Try tuning your MTU on both sides of the tunnel.
Thanks for the reply. I posted on the pfsense forum. I'll update this thread when I find a solution.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 10, 2016, 01:54:42 PM
I ran the ICSI / UC Berkeley Netalyzer. It reported some dns issues as well as this:

QuoteIPv6 Path MTU (?): Warning – Your system can receive fragmented traffic, but can not send fragmented traffic over IPv6. The path between our system and your network has an MTU of 1480 bytes. The bottleneck is at IP address 2001:470:X:Y::2. The path between our system and your network does not appear to handle fragmented IPv6 traffic properly.
Title: Re: Possible Routing Problem
Post by: broquea on October 10, 2016, 03:10:13 PM
Well then I guess its time to fiddle with your MTU settings on the tunnel. I'd recommend setting both sides to 1280 and raising it until i breaks again. If you are on PPPOE, then your max would be 1472 instead of 1480.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 10, 2016, 05:28:38 PM
Quote from: broquea on October 10, 2016, 03:10:13 PM
Well then I guess its time to fiddle with your MTU settings on the tunnel. I'd recommend setting both sides to 1280 and raising it until i breaks again. If you are on PPPOE, then your max would be 1472 instead of 1480.

I set the tunnel to 1280 but it didn't make any difference. Here is the result according to netalyzer:

QuoteYour system can receive fragmented traffic, but can not send fragmented traffic over IPv6. The path between our system and your network has an MTU of 1452 bytes. The bottleneck is at IP address 2001:470:0:9d::2. The path between our system and your network does not appear to handle fragmented IPv6 traffic properly.

Also, I reconfigured the DNS to be exactly the same as my other system for which I have no problem accessing mail.yahoo.com. The system that's not working was using a dns forwarder. I changed it to use the resolver and also removed any non-default settings. Now pfsense is the dns resolver for both ipv4 and ipv6. Here is what netalyzer says about the DNS WRT mail.yahoo.com:

QuoteOne popular name has a moderate anomaly: we are unable to find a reverse name associated with the IP address provided by your ISP's DNS server, although we expected to find a name. This is most likely due to a slow responding DNS server. If you rerun Netalyzr and see this condition remain, it could be due to a misconfiguration on the part of the domain owner, deliberate blocking using DNS, or your DNS server could be misconfigured or enabling a Man-in-the-Middle attack.

What's strange is that I'm seeing the same message on the other system, except I can access mail.yahoo.com. The differences between the systems are the one that's not working is using the latest release version of pfsense (2.3.2_1) and HE tunnel for ipv6. The other is using the latest development version (2.4) and native ipv6. I have no way to prove whether the problem is with pfsense because I can't use 2.3.2_1 with native ipv6. I'll see if I can set up the development version to use the HE tunnel.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 11, 2016, 08:44:41 AM
I tried various MTU settings and it made no difference. I found this mtu path utility (http://www.iea-software.com/products/mtupath.cfm (http://www.iea-software.com/products/mtupath.cfm)) and ran it a few times using ipv4 and ipv6 to various sites. The mtu to google.com is 1500 and it was 1480 to mail.yahoo.com. That's the default for tunnelbroker and even with the router set to 1480 also, it made no difference. I cannot access mail.yahoo.com through the tunnel, but I can access it natively on my other system. I'm really stumped by this. The only notable difference between the two systems is the tunnel.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 12, 2016, 03:26:17 PM
The MTU for the tunnel is currently set to the maximum, 1480. I did some testing using ping -6. I found that the largest size for mail.yahoo.com is 1432 and the largest size for google.com is 1452. Does that imply the MSS is 1432 and 1452, respectively? I ran mtupath and it reported 1452 MSS for both, suggesting 1500 "estimated MTU". Not sure why there's a discrepancy. What should I add to the MSS to get the MTU?

Also, I'm trying to set up another pfsense system using 2.4 and the tunnel so I can make a direct comparison between 2.4 native ipv6 and 2.4 tunnel. 2.4 is developmental and has an issue so it might be a while before I can post an update about this.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 18, 2016, 08:51:29 PM
I did some additional testing and I'd appreciate some feedback. I can access mail.yahoo.com on my test pfsense system, running 2.4 with native ipv6. I can also access it using a host through a mullvad vpn (dual-stack) terminating in the USA. I cannot access it using my "production" pfsense system, running 2.3.2_1 and with an HE tunnel.

I noticed when I looked at the route table that the route to the server end of the tunnel had mtu of 1280 so I set the mtu of opt1 and the tunnel to 1280. It did not fix the problem. I still can't load mail.yahoo.com. I'm not clear how this route got set to 1280 in the first place, because the MTU of opt1 was originally left at the default.

I also did a ping test to mail.yahoo.com and google.com using the 1280 mtu setting. I can ping -6 both of them up to 1232 bytes.

I've attached the ipv6 routes, opt1 configuration, tracert -6 mail.yahoo.com and the results of the dns and ipv6 sections of the netalyzer test. What's interesting is that now netalyzr is now saying the bottleneck is within the HE network, at 2001:470:0:9d::2. Does that indicate a problem in the HE network? I don't even know if MTU is the cause of this problem or whether it's dns (or something else).

Considering I can access mail.yahoo.com natively and through a vpn, but not through the tunnel, it really looks like the tunnel. What else is there to try? I've been using an HE tunnel to access mail.yahoo.com for several years with no recent changes to my network.

(https://dl.dropboxusercontent.com/u/61356231/Routes.PNG)
(https://dl.dropboxusercontent.com/u/61356231/OPT1.PNG)
(https://dl.dropboxusercontent.com/u/61356231/dns%20ipv6.PNG)
(https://dl.dropboxusercontent.com/u/61356231/tracert.PNG)
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 20, 2016, 01:23:12 PM
Any comments on this? I've verified MTU settings and I'm getting consistent results using ping and mtu path. It's really looking like something is messed up in the tunnel.
Title: Re: Possible Routing Problem
Post by: cholzhauer on October 20, 2016, 06:15:59 PM
If you think it's a tunnel issue, email ipv6@he.net
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 21, 2016, 10:16:16 AM
Quote from: cholzhauer on October 20, 2016, 06:15:59 PM
If you think it's a tunnel issue, email ipv6@he.net
Thanks, I'm in contact with support.

I have a question about the normal / expected behaviour of a tunnel. Should it be possible to ping the client and server ends of the tunnel, as well as the allocated gateway address from a remote system? I can ping all three addresses using the wan interface of the router that is connected to the tunnel, but the only address I can ping from a system on another network is the server address of the tunnel. If I try to tracert the addresses, it completes to the server address, but times out part way into the HE network, stopping at tserv1.lax1.he.net. Is that normal?

If I go to ipv6-test.com, icmp to the host doesn't work and the ping test fails. My firewall has a rule to allow ipv6 icmp, but the rule is not firing, suggesting the pings aren't making it to my router. If I use exactly the same firewall rules on my system with native ipv6, I can see the rule firing when the ping test runs.
Title: Re: Possible Routing Problem
Post by: cholzhauer on October 21, 2016, 10:52:45 AM
What number do you get on ipv6-test.com ?

I get 19/20 because my desktop doesn't have RDNS
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 21, 2016, 01:57:21 PM
I completely reconfigured my router with a new tunnel. I'm using a closer tunnel server with much lower latency and fewer hops. I get 19/20 on ipv6-test.com. The only deficiency is the hostname of for the ipv6 address. The speed and ping tests also work well. I also carefully verified the mtu settings on all of the routes. They are all at 1480 and when I run mtupath to mail.yahoo.com, it's showing 1480. It's working much better than it was before, with the exception that I still cannot access mail.yahoo.com.

I'm really stumped by this. Until a few weeks ago, this had never happened. There were no changes to the router whatsoever. After the problem started happening, I upgraded the router to the latest version. That made no difference. I also changed the dns from the forwarder to the resolver. Also no difference. Now I completely deleted all of the ipv6 configuration and the tunnel, reconfigured it and it's still not working.
Title: Re: Possible Routing Problem
Post by: cholzhauer on October 22, 2016, 08:49:50 AM
Works OK for me, where is your tunnel from?
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 22, 2016, 10:31:03 AM
Quote from: cholzhauer on October 22, 2016, 08:49:50 AM
Works OK for me, where is your tunnel from?
Originally, LA. Now Seattle with the new tunnel.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 22, 2016, 12:17:57 PM
The default MTU in my system is 1500. The MTU for the tunnel is 1480. I tried pinging the system with the tunnel using my other system which has native ipv6. I can ping the server end with -l 1452. I can ping the client end with -l 1432. I can ping the gateway with -l 1432. I also ran mtupath and it gave the same results. It is reasonable to conclude that these results imply this isn't an MTU issue?

I'm completely at a loss as to why I can't access mail.yahoo.com from the system with the tunnel unless I disable ipv6 either on the pc or on the router.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 25, 2016, 07:27:07 PM
I was going to set up a tunnel from SixXS to prove or disprove that the problem is being caused by the HE tunnel or their interconnect with yahoo. SixXS is still providing tunnel service, but they are not accepting new registrations. I emailed their support and asked if they would make a temporary exception. The person who replied back lectured me that I should switch to another ISP if mine won't provide native ipv6. He was rude and obnoxious. I haven't experienced such unprofessional service in a long time. I have to say in all the years I've used HE, I've found the company to provide excellent support.
Title: Re: Possible Routing Problem
Post by: cholzhauer on October 26, 2016, 05:01:21 AM
Heh,  Jeroen is unique.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on October 26, 2016, 10:08:05 AM
Quote from: cholzhauer on October 26, 2016, 05:01:21 AM
Heh,  Jeroen is unique.
That's far more politeness than he deserves. I was thinking more like <self-censored>.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on November 03, 2016, 06:16:55 PM
I posted about this on the pfsense forum. Two other people using he.net tunnels are also unable to access mail.yahoo.com, so it's definitely tunnel related. Whether that means there is a problem in he.net or whether yahoo is handling tunnel traffic differently from other traffic, it's not clear.
Title: Re: Possible Routing Problem
Post by: cholzhauer on November 03, 2016, 06:17:53 PM
Or it's a software issue.

Like I mentioned, I have no issues hitting it over an HE tunnel.
Title: Re: Possible Routing Problem
Post by: bimmerdriver on November 03, 2016, 07:23:36 PM
Quote from: cholzhauer on November 03, 2016, 06:17:53 PM
Or it's a software issue.

Like I mentioned, I have no issues hitting it over an HE tunnel.
If I wasn't using exactly the same software for months with no problem and I upgraded, then encountered the problem, I would agree. However, I didn't upgrade. The problem spontaneously occurred. Upgrading to the next version didn't cause the problem to go away. It's the same with the others who have reported this problem.
Title: Re: Possible Routing Problem
Post by: snarked on April 08, 2017, 05:56:28 PM
However, I DO have problems with yahoo's webmail and IPv6.  It works when I disable IPv6 and use IPv4, but that's not a viable solution.  I note that they do have an IPv6 address after one goes through a handful (currently 3) of CNAME aliases.

Seeing what is posted at other forums, it appears that their login API does not respond over IPv6.  Changing the MTU did not work as a solution.  I conclude that yahoo may not have an IPv6 listener but only IPv4 for that particular API and that's why it fails with IPv6.  This is not an HE problem.

This posting (https://forums.yahoo.net/t5/Password-and-sign-in/api-login-yahoo-com-not-working-from-IPv6-addresses-FOR-OVER-2/td-p/139061) has a "hosts" file solution that may or may not work for some configurations.  However, I agree that we shouldn't have to insert a hack for THEIR misconfiguration.