DNS maintenance protocol problem

[sdgathman]

While the ns?.he.net servers are down for maintenance, they should be unresponsive - or return SERVFAIL.  Instead, they are returning NXDOMAIN for all queries.  This is playing havoc with clients that query an he.net secondary DNS server first.

The simplest solution might be to block port 53 in the firewall during maintenance.  A permission denied from the firewall would also avoid TIMEOUTs, and would avoid the NXDOMAIN.

[primordial]

Precisely. This failure mode of providing a negative response is the worst possible thing to do during an outage, and is wreaking havoc on our services. A timeout or SERVFAIL would allow requests to gracefully retry other server until our primary/master server (in my case on-premesis) is queried and provides a valid response.

On top of that, having 5 public targets (leading to many anycast servers on the back-end) is supposed to provide resiliency to this sort of thing. To have all 5 targets fail in the same way is pretty unforgivable in the ultra-critical field of serving DNS. We'll definitely need a root-cause analysis report of what transpired to cause this blunder. Then we'll decide whether to jump ship. (Continued lack of support for DNSSEC has us looking for an exit strategy already... sigh.)

[Aeular]

So, infact the message for the "maintenance" showed up now about 133 min ago, AFTER the dns started having issues.  As part of trying to fix it, I signed into site normally (no lag) then went to tell it to validate and fetch a fresh copy (as it had nothing).  Site then got slow, after probably 20 min, they put up that message.  Sounds not like "maintenance" but more like "it broke, lets call it maintenance while we fix it".

I agree returning NXDOMAIN is the worst possible response they could be doing, even just simply shutting them off till they have it fixed, so its time outs would be better.

Could we get an updated ETA please, as you are more than twice what was given?

[keencs]

Where do you see an ETA?  I have been looking but can't find a status page.

Ryan

[Aeular]

They had a huge thing on http://dns.he.net about it, its gone now that its back and running.