Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DNS lookup failure on Arch Linux #664

Closed
0xj0hn opened this issue Mar 8, 2022 · 30 comments · Fixed by #760
Closed

DNS lookup failure on Arch Linux #664

0xj0hn opened this issue Mar 8, 2022 · 30 comments · Fixed by #760
Assignees
Labels
bug Something isn't working

Comments

@0xj0hn
Copy link

0xj0hn commented Mar 8, 2022

Expected behavior (what should have happened):

  • it should update the status of my coding... but as you can see in below pictures, it cannot do that.
    Actual behavior (what went wrong):
  • Api error / cannot update anything to the server:

image

Environment:

  • OS: Arch Linux
  • Platform: amd64 / x86_64

Logs:
image

Any suggestions?

@github-actions github-actions bot added the triage label Mar 8, 2022
@gandarez
Copy link
Member

gandarez commented Mar 8, 2022

It could happen when wakatime-cli can't reach the API. Are you behind a proxy or corporate firewall? Did it work before?

@gandarez gandarez added need investigation Issue that needs to be investigated before any conclusion and removed triage labels Mar 8, 2022
@gandarez gandarez self-assigned this Mar 8, 2022
@alanhamlett
Copy link
Member

The last line says dial tcp: lookup api.wakatime.com timeout which I think means it timed out when connecting to your DNS server to lookup the ip associated with the api.wakatime.com domain.

@0xj0hn
Copy link
Author

0xj0hn commented Mar 9, 2022

It could happen when wakatime-cli can't reach the API. Are you behind a proxy or corporate firewall? Did it work before?

Ah, I'm using a OpenVPN Connection. May it cause of the problem?
by the way, it doesn't work with my original IP address too...

@0xj0hn
Copy link
Author

0xj0hn commented Mar 9, 2022

i turned vpn off, but still doesn't work.

@TurnrDev
Copy link

I am having this issue but I'm able to ping api.wakatime.com
Also on Arch Linux, visual-studio-code-bin package, 1.65.2
wakatime- vscode plugin 18.0.7
wakatime-cli v1.38.0

@alanhamlett
Copy link
Member

@0xj0hn has it worked previously or was always broken?

@FutureInventor
Copy link

FutureInventor commented Jun 9, 2022

I'm having simillar problem with vim package. I've installed it properly, but do not see it on my 'plugins status page'. I've tried to use "WakaTimeToday" commend, but nothing happend and in the logs I can see this info:
{"caller":"cmd/run.go:276","func":"cmd.runCmd","level":"error","message":"failed to run command: today fetch failed: api error: failed fetching today from api: failed to make request to \"https://api.wakatime.com/api/v1/users/curren t/statusbar/today\": Get \"https://api.wakatime.com/api/v1/users/current/statusbar/today\": dial tcp: lookup api.wakatime.com on [::1]:53: read udp [::1]:41052->[::1]:53: read: connection refused","now":"2022-06-09T22:42:22Z","versi on":"v1.48.1"} {"caller":"pkg/api/diagnostic.go:28","func":"api.(*Client)","level":"debug","message":"sending diagnostic data to api at https://api.wakatime.com/api/v1/plugins/errors","now":"2022-06-09T22:42:22Z","version":"v1.48.1"} {"caller":"cmd/run.go:282","func":"cmd.runCmd","level":"warning","message":"failed to send diagnostics: failed to send diagnostics to the API: failed making request to \"https://api.wakatime.com/api/v1/plugins/errors\": Post \"https ://api.wakatime.com/api/v1/plugins/errors\": dial tcp: lookup api.wakatime.com on [::1]:53: read udp [::1]:41102->[::1]:53: read: connection refused","now":"2022-06-09T22:42:22Z","version":"v1.48.1"}
I'm also able to ping api.wakatime.com.

@alanhamlett
Copy link
Member

@FutureInventor those logs say the plugin couldn't connect to the API. Are you able to try it from a different internet connection?

@FutureInventor
Copy link

I've connected to different internet connection and still could not connect to API (I am still able to ping api.wakatime.com though).
Logs after use of "WakaTimeToday" command:
{"caller":"cmd/run.go:81","func":"cmd.Run","level":"debug","message":"command: version","now":"2022-06-10T18:35:14Z","version":"v1.48.1"} {"caller":"cmd/run.go:81","func":"cmd.Run","level":"debug","message":"command: version","now":"2022-06-10T18:35:21Z","version":"v1.48.1"} {"caller":"cmd/run.go:99","func":"cmd.Run","level":"debug","message":"command: today","now":"2022-06-10T18:35:30Z","version":"v1.48.1"} {"caller":"cmd/run.go:276","func":"cmd.runCmd","level":"error","message":"failed to run command: today fetch failed: api error: failed fetching today from api: failed to make request to \"https://api.wakatime.com/api/v1/users/curren t/statusbar/today\": Get \"https://api.wakatime.com/api/v1/users/current/statusbar/today\": dial tcp: lookup api.wakatime.com on [::1]:53: read udp [::1]:43076->[::1]:53: read: connection refused","now":"2022-06-10T18:35:30Z","versi on":"v1.48.1"} {"caller":"pkg/api/diagnostic.go:28","func":"api.(*Client)","level":"debug","message":"sending diagnostic data to api at https://api.wakatime.com/api/v1/plugins/errors","now":"2022-06-10T18:35:30Z","version":"v1.48.1"} {"caller":"cmd/run.go:282","func":"cmd.runCmd","level":"warning","message":"failed to send diagnostics: failed to send diagnostics to the API: failed making request to \"https://api.wakatime.com/api/v1/plugins/errors\": Post \"https ://api.wakatime.com/api/v1/plugins/errors\": dial tcp: lookup api.wakatime.com on [::1]:53: read udp [::1]:33393->[::1]:53: read: connection refused","now":"2022-06-10T18:35:30Z","version":"v1.48.1"}

@alanhamlett
Copy link
Member

alanhamlett commented Jun 10, 2022

Can you run:

curl https://api.wakatime.com/api/v1/meta

@FutureInventor
Copy link

Message I've got after run 'curl https://api.wakatime.com/api/v1/meta':
{"data":{"ip_descriptions":{"api":"Public IPs used by api.wakatime.com api servers. These IPs are used for receiving coding activity from developer plugins, and should be whitelisted for outgoing HTTPS (port 443) connections. These IPs do not make requests to external servers, and therefore do not need to be whitelisted in any incoming firewall rules.","website":"Public IPs used by wakatime.com website servers. These IPs will only make requests to external servers from user generated actions, such as when setting up a self-hosted GitLab integration, and therefore should be whitelisted in your firewall rules for incoming connections anytime you configure your GitLab integration. These IPs also receive webhooks from your self-hosted GitLab servers if the GitLab integration is enabled, and should be whitelisted for outgoing HTTPS (port 443) connections on an ongoing basis.","worker":"Public IPs used by background processing tasks. These IPs make external requests, such as when syncing commits for a self-hosted GitLab integration, and should be whitelisted in your firewall rules for incoming connections on an ongoing basis. These IPs never receive webhook requests, and therefore do not need to be whitelisted for outgoing connections."},"ips":{"api":["147.182.229.252","143.244.180.65","137.184.80.72","143.244.210.202"],"website":["143.198.244.187","137.184.90.248"],"worker":["137.184.41.156","128.199.1.211","137.184.39.171","147.182.229.82","137.184.45.227","137.184.39.212","137.184.43.151","137.184.43.75","137.184.43.111"]},"last_modified_at":"2022-05-19T14:31:17Z"}}

@alanhamlett
Copy link
Member

That means you're not behind any proxy and wakatime-cli should also be able to connect. Maybe it's being blocked by some firewall or antivirus on your machine?

@alanhamlett
Copy link
Member

Did it work before and stop working recently? If so, could you try downloading an old version and see if running that with --today works when the current version doesn't?

@FutureInventor
Copy link

From what I know, I do not have any firewall or antivirus on my device. If that helps, I try to use this vim plugin, on Android 12 in Termux terminal and neo-vim text editor. I've installed that plugin two days ago, so I do not know how older versions works, but I might give it a shot and try to download it.

@alanhamlett
Copy link
Member

I've installed that plugin two days ago

In that case, it probably also won't work with older versions. You're also using Linux same as the others in this thread? What flavor of linux?

@alanhamlett
Copy link
Member

Looking closer, that error message is a DNS error meaning it's not able to resolve the IP address for api.wakatime.com:
dial tcp: lookup api.wakatime.com on [::1]:53: read udp [::1]:41052->[::1]:53: read: connection refused

Does it work if you add these 2 lines to your ~/.wakatime.cfg file:

api_url = https://143.244.210.202/api/v1
no_ssl_verify = true

@FutureInventor
Copy link

Changing config file brought some progress I guess. Now I've managed to fetch "WakaTimeToday" request, although it's empty for some reason. I still can not see vim-plug in my Plugin Status page as well. Logs:
{"caller":"pkg/offline/offline.go:64","func":"offline.WithQueue","level":"debug","message":"pushing 1 heartbeat(s) to queue due to error","now":"2022-06-11T19:36:55Z","version":"v1.48.1"} {"caller":"cmd/run.go:276","func":"cmd.runCmd","level":"error","message":"failed to run command: sending heartbeat(s) later due to api error: won't send heartbeat due to backoff","now":"2022-06-11T19:36:55Z","version":"v1.48.1"} {"caller":"pkg/api/diagnostic.go:28","func":"api.(*Client)","level":"debug","message":"sending diagnostic data to api at https://143.244.210.202/api/v1/plugins/errors","now":"2022-06-11T19:36:55Z","version":"v1.48.1"} {"caller":"cmd/run.go:319","func":"cmd.sendDiagnostics","level":"debug","message":"successfully sent diagnostics","now":"2022-06-11T19:36:56Z","version":"v1.48.1"} {"caller":"cmd/run.go:81","func":"cmd.Run","level":"debug","message":"command: version","now":"2022-06-11T19:37:00Z","version":"v1.48.1"} {"caller":"cmd/run.go:81","func":"cmd.Run","level":"debug","message":"command: version","now":"2022-06-11T19:37:04Z","version":"v1.48.1"} {"caller":"cmd/run.go:81","func":"cmd.Run","level":"debug","message":"command: version","now":"2022-06-11T19:37:25Z","version":"v1.48.1"} {"caller":"cmd/run.go:81","func":"cmd.Run","level":"debug","message":"command: version","now":"2022-06-11T19:37:28Z","version":"v1.48.1"} {"caller":"cmd/run.go:99","func":"cmd.Run","level":"debug","message":"command: today","now":"2022-06-11T19:37:36Z","version":"v1.48.1"} {"caller":"cmd/today/today.go:51","func":"today.Run","level":"debug","message":"successfully fetched today for status bar","now":"2022-06-11T19:37:37Z","version":"v1.48.1"}

@alanhamlett
Copy link
Member

The reason you don't see Vim yet is won't send heartbeat due to backoff. You can delete this file to reset that backoff rate limiting:
~/.wakatime-internal.cfg

This proves it's a DNS problem, since using the IP address of api.wakatime.com directly works. Still need to figure out if it's a bug in wakatime-cli or something misconfigured on your ISP. Can you check the output of these 3 commands:

  1. dig api.wakatime.com
  2. grep "nameserver" /etc/resolv.conf
  3. ip route show

@FutureInventor
Copy link

FutureInventor commented Jun 12, 2022

Output of following commands:
dig api.wakatime.com:

; <<>> DiG 9.16.27 <<>> api.wakatime.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33837
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;api.wakatime.com.              IN      A

;; ANSWER SECTION:
api.wakatime.com.       3360    IN      A       143.244.210.202

;; Query time: 56 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Jun 12 16:45:27 CEST 2022
;; MSG SIZE  rcvd: 61

grep "nameserver" /etc/resolv.conf:
grep: /etc/resolv.conf: No such file or directory

ip route show:
192.168.0.0/24 dev wlan0 proto kernel scope link src 192.168.0.185

@FutureInventor
Copy link

I've also deleted ~/.wakatime-internal.cfg and somehow the plugin start to run properly (I do believe so at least).

@alanhamlett
Copy link
Member

Since dig api.wakatime.com resolves the correct IP that means it's a bug in Go's DNS client or our usage of it.

@FutureInventor
Copy link

Let me know if I can help you further with my logs in any way.

@alanhamlett alanhamlett changed the title Api Error(102); Check your .wakatime.log file for more details. DNS lookup failure on Arch Linux Jun 16, 2022
@alanhamlett alanhamlett added bug Something isn't working need investigation Issue that needs to be investigated before any conclusion and removed need investigation Issue that needs to be investigated before any conclusion labels Jun 16, 2022
@TurnrDev
Copy link

Was there a short-term solution to this? Should I delete ~/.wakatime-internal.cfg from my machine?

@alanhamlett
Copy link
Member

@TurnrDev the short-term solution is adding these lines to your ~/.wakatime.cfg file:

api_url = https://143.244.210.202/api/v1
no_ssl_verify = true

Deleting your ~/.wakatime-internal.cfg file isn't necessary, it will just reset your client-side rate limit sooner if the rate limit backoff is currently enabled due to past DNS failures.

@TurnrDev
Copy link

Perfect! I had to manually run the cli to sync offline activity but I think I got it all in, thanks! Been not working for months so I ran it with 10k heartbeats a few times

@alanhamlett
Copy link
Member

alanhamlett commented Jun 21, 2022

@FutureInventor and @TurnrDev if you remove the api_url setting, then add 8.8.8.8 to your /etc/resolv.conf file does it start working?

found from docker/cli#2618 (comment)

@0xj0hn
Copy link
Author

0xj0hn commented Jul 26, 2022

@0xj0hn has it worked previously or was always broken?

It's working now. fixed. I've changed my DNS and it started working

@alanhamlett
Copy link
Member

Changing your dns is the same as modifying your /etc/resolv.conf file. We should detect DNS failures and retry the https request using a custom DNS resolver with 1.1.1.1, 8.8.8.8, and their ipv6 equivalents:

https://stackoverflow.com/a/59889883/1290627

@alanhamlett alanhamlett removed the need investigation Issue that needs to be investigated before any conclusion label Jul 26, 2022
@0xj0hn 0xj0hn closed this as completed Jul 28, 2022
@alanhamlett
Copy link
Member

Let's leave this open until it works without manual modifications.

@alanhamlett
Copy link
Member

@TurnrDev after removing the api_url line and upgrading wakatime-cli (should happen automatically when you restart your editor) does it now work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants