Unable to use Google Search

[nelgin]

I've had to switch my systems to prefer ipv4 because I'm getting an error accessing Google with ipv6. On my cellphone I get

403. That's an error.

Your client does not have permissions to get URL /search?..... from this server. That's all we know.

HE need to sort this out in a hurry since it seems there's no way to prefer ipv4 over ipv6 on Android (tho I'm willing to be educated).

[cshilton]

This has been an ongoing problem for a few weeks. At the start, www.google.com forced you through a CAPTCHA to use the site. Now they are 403 / blocking things. It's a good bet that there's some a**hat on the internet who is using HE.net tunnelbroker IP space to abuse www.google.com in some way. Once you pass the tests, you get a tunnel and you can get a new tunnel with new IP space through the api. Someone could run many kinds of attack through that and they could change their IP address nearly at will through the API while doing it.

I started a thread on this three or four weeks ago: "Google forcing ReCAPTCHA..." This issue responds well to the solutions for the Netflix problem, basically change your edge router so you don't go to www.google.com over your HE.net IPv6 tunnel. I'm not sure what you are using for a router but your solution probably lies there.

[snarked]

Google doesn't play well with others.

As noted, Google sees a lot of activity from HE tunnels; so much that it thinks there are web spiders from HE (and there may be).  When it thinks there's a spider (or other automated program due to "high volume"), it throws the captcha at it.  Continued high volume then yields the 403 response.  Google doesn't care whether this blocks similar IP addresses or not.  This is not a 2023 problem.  It's happened before.

Other google issues they won't change (examples):

1) Googlebot will make requests with bad HTTP header data, including invalid mailbox syntax in the optional "From:" header and for nonexistent Google-site-verification keys.  The latter is OK, intentionally seeking a 404 response.  The "From:" header problem started in 2018.  I have even complained to their representatives at CES in person in 2019, yet they won't fix it.

2) Gmail sometimes generates invalid "Received:" email headers for messages claiming SMTP compliance by omitting the required "from" clause.  This is usually for messages originating via Gmail.

Deviations from Internet standards is bad, but Google thinks it's large enough that it may do what it wants, and that's the real problem.  HE may not be a large-enough player to be able to fight back.

[Pentium4User]

At least for the Frankfurt tunnels, Google search works fine again, no captchas, no error 403.

[kurahaupo]

@snarked I feel your pain over malformed Received: headers and the intransigent companies who refuse to fix them, but Google is hardly the worst; omitting a clause at least leaves it syntactically valid, whereas for over 20 years Microsoft products intentionally injected invalid syntax ("with Microsoft SMTP").

However the "from" clause is not actually mandated by RFC 2822 (or its successors), and for some protocols it doesn't even make sense.

If you're trying to track back to the source of a message, any Received header without a "from" clause should simply be ignored; the implication is that the receiving host trusted the sending host, and therefore you can trust the previous Received: header (the one after it in the byte stream).

[snarked]

To kurahaupo - Aside - email:  The "From" clause of a "Received:" header may be optional under RFCs 822, 2822, and 5322, but is mandatory under RFC 821 (section 4.1.2; BNF on page 32), 2821 (section 4.4), and 5321 (also section 4.4) for SMTP (when the "with *SMTP*" wildcarded protocol clause is used).  Omission leaves it invalid - see the defined ABNF syntax in 5321.

If Google can't respect that, my point was do you all really expect them to listen to your complaints?  (Microsoft Exchange and Qmail are problems as well).