Google forcing ReCAPTCHA on all searches from my HE assigned IPv6 address

[cshilton]

I'm not sure what everyone is using for a firewall. I'm using OpenBSD. For the purposes of this discussion it might as well be a variant of pfSense / opnSense.. When I had to deal with the netflix problem, I went with defense-in-depth and stopped quad-A searches for netflix's domains. As a second layer, I also added all of their IP space relative to me to a table. I block new connections at my edge to addresses in the table with a TCP RST / UDP port unreachable. The pros and cons are: pro - this plays well with happy eyeballs; the target never sees traffic from HE's ASN coming from your tunnelbroker space; cons - the addresses that you are blocking are dynamic so you need to have a mechanism to keep them up to date; you also may be blocking content temporarily that you don't need to block. I added manually added www.google.com to this space when this first started happening since I hoped that the problem would get resolved. Apparently, it hasn't so I'll have to automate this process.

I'd bet dollars to donuts that some a**hat has found a way to abuse the tunnels again and is causing problems for google and this is just a response. This was the case with Netflix and it's also the problem with Wikipedia. As such, I think that we are going to be dealing with problems like this until IPv6 is much more relevant. Right now, kick/banning an entire ISP is an easy way to deal with the abuse if you look at it from the application provider's side.

[doktornotor]

Right now, kick/banning an entire ISP is an easy way to deal with the abuse if you look at it from the application provider's side.

Right now, Google has got pretty much a monopoly in the search engines market. Similar behavior is completely unacceptable. Apparently, they are asking for more trouble in the EU.

[Jenick]

I'm not sure what everyone is using for a firewall. I'm using OpenBSD. For the purposes of this discussion it might as well be a variant of pfSense / opnSense.. When I had to deal with the netflix problem, I went with defense-in-depth and stopped quad-A searches for netflix's domains. As a second layer, I also added all of their IP space relative to me to a table. I block new connections at my edge to addresses in the table with a TCP RST / UDP port unreachable. The pros and cons are: pro - this plays well with happy eyeballs; the target never sees traffic from HE's ASN coming from your tunnelbroker space; cons - the addresses that you are blocking are dynamic so you need to have a mechanism to keep them up to date; you also may be blocking content temporarily that you don't need to block. I added manually added www.google.com to this space when this first started happening since I hoped that the problem would get resolved. Apparently, it hasn't so I'll have to automate this process.

I'd bet dollars to donuts that some a**hat has found a way to abuse the tunnels again and is causing problems for google and this is just a response. This was the case with Netflix and it's also the problem with Wikipedia. As such, I think that we are going to be dealing with problems like this until IPv6 is much more relevant. Right now, kick/banning an entire ISP is an easy way to deal with the abuse if you look at it from the application provider's side.

I'm on pfSense Plus 23.05 latest release with the latest patch set.  There's an option in pfBlockerNG under DNSBL Python mode to disable AAAA lookups against a list right there in the UI.  I just spotted it, enabled it, and added www.google.com and google.com to the list.  Got google access back now, not that we use it much anyhow but we do sometimes go there when duckduckgo isn't giving enough relevant results.  Hope this helps some pfSense users.  I'm not sure if the pfSense CE 2.6.0 version has this option since that version used to require the pfBlockerNG-devel branch package to get the extra features.  It's easy enough to upgrade to 23.05 Plus from CE 2.6.0 anyhow for personal use.  Even with a work-around place, HE.net has to continue the pushback on Google to STOP doing this crap!

[cshilton]

...snip...

I'd bet dollars to donuts that some a**hat has found a way to abuse the tunnels again and is causing problems for google and this is just a response. This was the case with Netflix and it's also the problem with Wikipedia. As such, I think that we are going to be dealing with problems like this until IPv6 is much more relevant. Right now, kick/banning an entire ISP is an easy way to deal with the abuse if you look at it from the application provider's side.

I'm on pfSense Plus 23.05 latest release with the latest patch set.  There's an option in pfBlockerNG under DNSBL Python mode to disable AAAA lookups against a list right there in the UI.  I just spotted it, enabled it, and added www.google.com and google.com to the list.  Got google access back now, not that we use it much anyhow but we do sometimes go there when duckduckgo isn't giving enough relevant results.  Hope this helps some pfSense users.  I'm not sure if the pfSense CE 2.6.0 version has this option since that version used to require the pfBlockerNG-devel branch package to get the extra features.  It's easy enough to upgrade to 23.05 Plus from CE 2.6.0 anyhow for personal use.  Even with a work-around place, HE.net has to continue the pushback on Google to STOP doing this crap!

OpenBSD is distantly related to, and more primitive than pfSense and opnSense. It's like the OS from those products without the web based UI. Your solutions is much simpler than mine, would you mind posting it to the general solutions thread - https://forums.he.net/index.php?topic=4263.0 so that other people only have to look in one place to find it?

[Jenick]

...snip...

I'd bet dollars to donuts that some a**hat has found a way to abuse the tunnels again and is causing problems for google and this is just a response. This was the case with Netflix and it's also the problem with Wikipedia. As such, I think that we are going to be dealing with problems like this until IPv6 is much more relevant. Right now, kick/banning an entire ISP is an easy way to deal with the abuse if you look at it from the application provider's side.

I'm on pfSense Plus 23.05 latest release with the latest patch set.  There's an option in pfBlockerNG under DNSBL Python mode to disable AAAA lookups against a list right there in the UI.  I just spotted it, enabled it, and added www.google.com and google.com to the list.  Got google access back now, not that we use it much anyhow but we do sometimes go there when duckduckgo isn't giving enough relevant results.  Hope this helps some pfSense users.  I'm not sure if the pfSense CE 2.6.0 version has this option since that version used to require the pfBlockerNG-devel branch package to get the extra features.  It's easy enough to upgrade to 23.05 Plus from CE 2.6.0 anyhow for personal use.  Even with a work-around place, HE.net has to continue the pushback on Google to STOP doing this crap!

OpenBSD is distantly related to, and more primitive than pfSense and opnSense. It's like the OS from those products without the web based UI. Your solutions is much simpler than mine, would you mind posting it to the general solutions thread - https://forums.he.net/index.php?topic=4263.0 so that other people only have to look in one place to find it?

I'll do that but have to test a few more computers and devices with the change in place.  It seems to work only if you implement the chromium/firefox tweaks to disable the browser's built in DNS client DOH/DOT.  Our iPhones/iPads are still getting the block error even with the pfSense pfBlockerNG changes in place because they're likely resolving via their own built-in DNS client implementations.  This is so frustrating!

Yeah, I recall attempting OpenBSD about 15 years ago, had some luck getting it up and running but ended up going with pfSense when it first came out and just stuck with that.  I'm mainly a Linux/Windows/VMware/Cisco guy.  I can manage my way around the FreeBSD cli side of pfSense but only go there as needed.  So, more frustration, I just rebooted my iPhone, then hit google.com no issues inside the duckduckgo browser app.  As soon as I tried hitting it in the Safari browser I got kicked again.  So iOS is caching the IPv6 from the Safari attempt and then I get kicked from duckduckgo app on the next attempt.  This is ridiculous! Here we are downgrading security, kludging work-arounds, etc. all to visit a search engine that's become way too involved in our private lives.  I'll still post the work-arounds I put in place likely over the weekend.

[cshilton]

...snip...

I'll do that but have to test a few more computers and devices with the change in place.  It seems to work only if you implement the chromium/firefox tweaks to disable the browser's built in DNS client DOH/DOT.  Our iPhones/iPads are still getting the block error even with the pfSense pfBlockerNG changes in place because they're likely resolving via their own built-in DNS client implementations.  This is so frustrating!

A combination of your fix and my fix might do it for the clients where DNS blocking isn't working. I don't have enough experience with pfSense to figure out how to to modify the ruleset by hand though. I do wish that I knew how to turn off DNS over https for my network from a central place. It appears that blocking or redirecting ports 53 and 853 will do that for DoT. As far as I'm concerned DoH is a bandaid that's an unfortunate requirement for those people not running a proper DNS resolver on their network. But, running a DNS resolver is hard so most people don't do that in their home networks and I get this.

[Jenick]

...snip...

I'll do that but have to test a few more computers and devices with the change in place.  It seems to work only if you implement the chromium/firefox tweaks to disable the browser's built in DNS client DOH/DOT.  Our iPhones/iPads are still getting the block error even with the pfSense pfBlockerNG changes in place because they're likely resolving via their own built-in DNS client implementations.  This is so frustrating!

A combination of your fix and my fix might do it for the clients where DNS blocking isn't working. I don't have enough experience with pfSense to figure out how to to modify the ruleset by hand though. I do wish that I knew how to turn off DNS over https for my network from a central place. It appears that blocking or redirecting ports 53 and 853 will do that for DoT. As far as I'm concerned DoH is a bandaid that's an unfortunate requirement for those people not running a proper DNS resolver on their network. But, running a DNS resolver is hard so most people don't do that in their home networks and I get this.

Yep, exactly! The Google GPO policy settings I mentioned previously will disable DoH globally in the chromium-based browsers.  I'm not sure if Microsoft's Edge version has the same config settings.  I know Chrome and Brave both work fine with the policies in place as both of those hit google.com no problem now for me.  For enterprises it's fairly simple, if they have AD in place and healthy, with DCs running 2016 or 2019 server and the domain and forest functional levels set at least to 2012R2 they can simply download the Chrome GPO ADMX templates into their central policy directory and make the changes inside a GPO.  Brave might be more difficult, I'm not sure they have ADMX templates.  Smaller shops could do it with a powershell script by creating the proper registry keys and values then push that out as a startup or logon script.

This is the basic script I use at home on our PCs:

Code Select Expand

# This assumes you have winrm enabled on all the computers and can remotely execute powershell commands.
#
# This will enable winrm but might require a visit to each PC, or use of Sysinternals Suite psexec:
# winrm quickconfig
# Set-Service -Name WinRM -StartupType Automatic
# Start-Service -Name WinRM

# Disable Google Chrome DoH and the built-in DNS client to force OS lookups.
# Get an Administrator level windows credential for the domain joined PCs, or,
# if in WORKGROUP mode an admin account that has the same name and password on all computers in the workgroup.
$cred = Get-Credential
Invoke-Command -ScriptBlock {
New-Item -Path 'HKLM:\Software\Policies\Google'
New-Item -Path 'HKLM:\Software\Policies\Google\Chrome'
New-ItemProperty -Path 'HKLM:\Software\Policies\Google\Chrome' -Name 'DnsOverHttpMode' -Value 'off' -PropertyType String
New-ItemProperty -Path 'HKLM:\Software\Policies\Google\Chrome' -Name 'BuiltInDnsClientEnabled' -Value 0x0 -PropertyType Dword
} -Credential $cred -ComputerName PC01,PC02,PC03

# For Brave browser it would be:
Invoke-Command -ScriptBlock {
New-Item -Path 'HKLM:\Software\Policies\BraveSoftware'
New-Item -Path 'HKLM:\Software\Policies\BraveSoftware\Brave'
New-ItemProperty -Path 'HKLM:\Software\Policies\BraveSoftware\Brave' -Name 'DnsOverHttpMode' -Value 'off' -PropertyType String
New-ItemProperty -Path 'HKLM:\Software\Policies\BraveSoftware\Brave' -Name 'BuiltInDnsClientEnabled' -Value 0x0 -PropertyType Dword
} -Credential $cred -ComputerName PC01,PC02,PC03


[troz]

This type of stunt...

This isn't a "stunt". It's how their AUTOMATIC processes work. Bad stuff happens, the "AI" looks up the address (WHOIS), and arrives at a /48. Bingo. The pool from which thousands of tunnel routed /64's goes poof. Even if they block the /64, the baddie will just get another one.

Good luck getting either HE or Google (ESP. GOOGLE!) to do anything about it. Not that HE can... anyone can get an account, and setup tunnels.

(If enough /48's get tagged, the entire /32 allocation will be blocked. But I don't know what that threshold is.)

[alekssmacuks]

Hi there,

Just to confirm the same behaviour with Google in Latvia. Just tested again, and got 403 error on ipv6.google.com. Sad but true.

[ChrisDos]

Has there been any official comments or ticket from HE regarding the status of this?  For now I've just disabled IPv6 for myself and two other customers.

[papamidnight]

Has there been any official comments or ticket from HE regarding the status of this?  For now I've just disabled IPv6 for myself and two other customers.

Not a word. I likewise have an entry in DNS to do the following:

Code Select Expand

private-address: 2607:f8b0::/32


[kpanchev]

I have the pfSense successfully filtering Google by adding a rule to reject any traffic going to its IPv6 ranges:
1. Create a firewall ip alias with the ipv6 addresses from here: https://md5calc.com/google/ip
2. Create a reject rule on your LAN interface for IPv6, source any, destination single host or alias and point to your Google alias.
3. Save and apply.

Job done.
By the way, I have a similar rule for Yahoo mail, which did not want to play ball with IPv6 either.

[ezramus]

I have the pfSense successfully filtering Google by adding a rule to reject any traffic going to its IPv6 ranges:

Until HE fix things with Google, this seems to be the best solution by far. I was able to do the same, setting up a "Networks" alias on my OPNsense firewall. It avoids having to mess with any specific app to disable DNS over TLS/HTTPS etc. Properly coded apps that find they can't connect over IPv6 should transparently fall back to IPv4 and that's what I've seen with all clients on my network so far.

[Volui]

It's seems to Google working again through v6 tunnel... can anyone confirm?

[supergvozd]

It's seems to Google working again through v6 tunnel... can anyone confirm?

Yes, I confirm