Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade dnsjava to cope with Azure CNAME lists #344

Closed
anjackson opened this issue Jul 30, 2020 · 1 comment
Closed

Upgrade dnsjava to cope with Azure CNAME lists #344

anjackson opened this issue Jul 30, 2020 · 1 comment
Labels

Comments

@anjackson
Copy link
Collaborator

We have a problem with the latest H3 release - it is unable to resolve the domain coronavirus.data.gov.uk.

Poking via the scripting console, using code cribbed from FetchDNS, we run:

//Groovy
import org.xbill.DNS.DClass;
import org.xbill.DNS.Lookup;
import org.xbill.DNS.Record;
import org.xbill.DNS.Type;
import org.xbill.DNS.Name;

lookupName = "coronavirus.data.gov.uk.";
look = new Lookup(lookupName, Type.A, DClass.IN);
look.getDefaultCache(DClass.IN).clearCache();
look.setCache(null);
rrecordSet = look.run();
rawOut.println(rrecordSet);
rawOut.println(look.getErrorString());
rawOut.println(look.getAliases());

gives

null
CNAME loop
[coronavirus.data.gov.uk., covid19stat.azureedge.net., covid19stat.afd.azureedge.net., afd.t-0001.t-msedge.net., t-0001.t-msedge.net.]

i.e. a null result due to a CNAME loop. Inspecting dnsjava Lookup @ v.2.0.3, we can see that after 6 CNAME steps it assumes it's a CNAME loop.

A command-line lookup shows:

# dig @1.1.1.1 coronavirus.data.gov.uk

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-16.P2.el7_8.3 <<>> @1.1.1.1 coronavirus.data.gov.uk
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63111
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;coronavirus.data.gov.uk.       IN      A

;; ANSWER SECTION:
coronavirus.data.gov.uk. 244    IN      CNAME   covid19stat.azureedge.net.
covid19stat.azureedge.net. 1744 IN      CNAME   covid19stat.afd.azureedge.net.
covid19stat.afd.azureedge.net. 244 IN   CNAME   afd.t-0001.t-msedge.net.
afd.t-0001.t-msedge.net. 184    IN      CNAME   t-0001.t-msedge.net.
t-0001.t-msedge.net.    4       IN      CNAME   Edge-Prod-MAN30r3.ctrl.t-0001.t-msedge.net.
Edge-Prod-MAN30r3.ctrl.t-0001.t-msedge.net. 184 IN CNAME standard.t-0001.t-msedge.net.
standard.t-0001.t-msedge.net. 184 IN    A       13.107.246.10

;; Query time: 4 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Thu Jul 30 20:38:43 BST 2020
;; MSG SIZE  rcvd: 268

i.e. the CNAME path is seven steps long.

As of dnsjava v.2.1.9 the maximum number of iterations has been upped to 10. Therefore upgrading javadns to a more recent version should solve this problem. The most recent version is dnsjava v.3.2.2.

@anjackson
Copy link
Collaborator Author

As a workaround, I think we can set FetchDNS.setAcceptNonDnsResolves(true); which should fall back on host DNS resolution when dnsjava fails.

@ato ato added the bug label Feb 15, 2021
anjackson added a commit to ukwa/heritrix3 that referenced this issue May 20, 2021
This commit updates to version 3.3.1 of the dnsjava library.

Incidentally, the restlet.org certificate is currently invalid, so this
patch switched to maven.restlet.com.
anjackson added a commit that referenced this issue May 20, 2021
Update to latest version of dnsjava, for #344
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants