Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Name resolution failure at final restart #58

Closed
byRoadrunner opened this issue Sep 28, 2022 · 3 comments
Closed

Name resolution failure at final restart #58

byRoadrunner opened this issue Sep 28, 2022 · 3 comments

Comments

@byRoadrunner
Copy link

byRoadrunner commented Sep 28, 2022 •
edited
Loading

Hello,

I just installed LabCA on a fresh Ubuntu 22.04 server.
After installing and finishing the setup on the web, it is stuck at the final restart ("Now we will request a certificate for this website and restart one more time...").

In the boulder-labca-1 container I always get this error spammed:

boulder-labca-1    | 2022/09/28 16:33:19 errorHandler: err=dial tcp: lookup control: Temporary failure in name resolution
boulder-labca-1    | 2022/09/28 16:33:19 http: superfluous response.WriteHeader call from main.errorHandler (main.go:289)
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000407d28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000713770?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000408028?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0007134a0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000408328?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0007131d0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000408628?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000712f00?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000408928?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000712c30?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000408c28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000712960?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000408f28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000712690?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000409228?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0007123c0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000409528?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0007120f0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000409828?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000917e00?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000409b28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000917b30?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000409e28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000917860?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc00040a128?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0009174a0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc00040a428?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0009171d0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc00040a728?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000916f00?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc00040aa28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000916be0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000522d28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0009168c0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000523028?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000916410?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000523328?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000079f90?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000523628?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000079c70?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000523928?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000079810?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000523c28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000079540?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000523f28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000078dc0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000524228?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000078190?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000524528?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000078f50?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000524828?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000078e10?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000524b28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000078a50?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000524e28?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000917e50?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000525128?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc000917b80?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main.getLog({0x1351a40, 0xc0008ce460}, 0xc000525428?, {0xe0de0c, 0x4})
boulder-labca-1    |    /go/src/labca/main.go:1404 +0x3c5
boulder-labca-1    | main.errorHandler({0x1351a40?, 0xc0008ce460}, 0xe16c5c?, {0x134b9e0?, 0xc0009178b0?}, 0x1f4)
boulder-labca-1    |    /go/src/labca/main.go:316 +0x505
boulder-labca-1    | main._hostCommand({0x1351a40, 0xc0008ce460}, 0xc0005257e0?, {0xe16ac4, 0xc}, {0x0, 0x0, 0x160?})
boulder-labca-1    |    /go/src/labca/main.go:1681 +0x1d5
boulder-labca-1    | main.finalHandler({0x1351a40?, 0xc0008ce460}, 0xc000d1b000)
boulder-labca-1    |    /go/src/labca/main.go:2294 +0x61e
boulder-labca-1    | net/http.HandlerFunc.ServeHTTP(0xd08da0?, {0x1351a40?, 0xc0008ce460?}, 0xc0008ce460?)
boulder-labca-1    |    /usr/local/go/src/net/http/server.go:2084 +0x2f
boulder-labca-1    | main.authorized.func1({0x1351a40, 0xc0008ce460}, 0xc000d1b000)
boulder-labca-1    |    /go/src/labca/main.go:2770 +0x26d
boulder-labca-1    | net/http.HandlerFunc.ServeHTTP(0xc000d1af00?, {0x1351a40?, 0xc0008ce460?}, 0x735db4?)
boulder-labca-1    |    /usr/local/go/src/net/http/server.go:2084 +0x2f
boulder-labca-1    | github.com/gorilla/mux.(*Router).ServeHTTP(0xc0000fe840, {0x1351a40, 0xc0008ce460}, 0xc000d1ae00)
boulder-labca-1    |    /root/go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210 +0x1cf
boulder-labca-1    | net/http.serverHandler.ServeHTTP({0xc0008e9ec0?}, {0x1351a40, 0xc0008ce460}, 0xc000d1ae00)
boulder-labca-1    |    /usr/local/go/src/net/http/server.go:2916 +0x43b
boulder-labca-1    | net/http.(*conn).serve(0xc000970aa0, {0x13524c8, 0xc0008c9b00})
boulder-labca-1    |    /usr/local/go/src/net/http/server.go:1966 +0x5d7
boulder-labca-1    | created by net/http.(*Server).Serve
boulder-labca-1    |    /usr/local/go/src/net/http/server.go:3071 +0x4db

Name resolution should be working. Is there any special record I could have forgotten? I have a CAA record and an A record.
If you need any more logs I am happy to provide them!

Thanks!
@hakwerk
Copy link
Owner

hakwerk commented Sep 29, 2022

This seems to be a DNS issue between the docker containers: the boulder-labca-1 container cannot find the boulder-control-1 container: lookup control: Temporary failure in name resolution.

It usually has to do with the /etc/resolv.conf file (on the host, which gets copied into the container). Maybe this thread can help you troubleshoot: https://stackoverflow.com/questions/44761246/temporary-failure-in-name-resolution-errno-3-with-docker

Ping is not present in the image, but wget can be used to see if the name can be resolved:

# docker exec -it boulder-labca-1 bash
root@397c62ddb9ba:/go/src/labca# wget control
--2022-09-29 18:02:08--  http://control/
Resolving control (control)... 10.77.77.3
Connecting to control (control)|10.77.77.3|:80... failed: Connection refused.
root@397c62ddb9ba:/go/src/labca# wget google.com
...

@byRoadrunner
Copy link
Author

byRoadrunner commented Oct 2, 2022 •
edited
Loading

Hmm, okay.
The interesting thing is the resolv.conf in boulder-labca looks like this:

nameserver 127.0.0.11
options edns0 trust-ad ndots:0

But on the host:

nameserver 127.0.0.53
options edns0 trust-ad
search [my internal domain]

But I also noticed the control container is stuck in a restart loop and is failing its script.

./control.sh: line 94: docker: command not found

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists...
Building dependency tree...
Reading state information...
4 packages can be upgraded. Run 'apt list --upgradable' to see them.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem.```

hakwerk added a commit that referenced this issue Oct 5, 2022
@hakwerk
Fix docker installation (#60)
f087a9b 
Possibly also is the solution for #58 ?
@hakwerk
Copy link
Owner

hakwerk commented Oct 5, 2022

That last bit with ./control.sh: line 94: docker: command not found should now be resolved in the latest v22.10 release.

I don't think that the docker internal DNS issue on Ubuntu 22.04 will be solved by that, but it looks like I have now reproduced the Temporary failure in name resolution error myself so I will investigate further when I get a chance

@hakwerk hakwerk closed this as completed Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@byRoadrunner @hakwerk