Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a health check for the tracker container #512

Merged

Conversation

josecelano
Copy link
Member

@josecelano josecelano commented Nov 21, 2023

We need to check the three services provided by the container:

  • API
  • HTTP Tracker (1 or more)
  • UDP Tracker (1 or more)

And we also need to:

  • Check them only when they are enabled in the configuration.

Implementation

  • High-level health-checker API (in the future management API). It only checks enabled services. It makes a request to the service healthcheck endpoints.
  • Healthcheck API endpoint
  • Healthcheck HTTP Tracker endpoint
  • Healthcheck UDP Tracker request (using the connect request endpoint).

With the default configuration, you can use the endpoint: http://localhost:1313/health_check

It will return an OK Json response:

{
  "status": "Ok", 
  "message": ""
}

or an Error response:

{ 
  "status": "Error",
  "message": "API is not healthy. Health check endpoint: http://127.0.0.1:1212/health_check"
}

NOTICE: health checks are not executed when services use port 0 in the configuration. Service launchers must be changed to return the bound port so the health checker handler can connect.

@josecelano josecelano linked an issue Nov 21, 2023 that may be closed by this pull request
Copy link

codecov bot commented Nov 21, 2023

Codecov Report

Attention: 69 lines in your changes are missing coverage. Please review.

Comparison is base (50372d8) 84.28% compared to head (5e0a686) 84.20%.

Files Patch % Lines
src/servers/health_check_api/handlers.rs 38.33% 37 Missing ⚠️
src/bootstrap/jobs/health_check_api.rs 0.00% 17 Missing ⚠️
src/servers/health_check_api/resources.rs 58.33% 5 Missing ⚠️
src/servers/health_check_api/responses.rs 50.00% 3 Missing ⚠️
src/lib.rs 85.71% 2 Missing ⚠️
src/servers/health_check_api/server.rs 86.66% 2 Missing ⚠️
src/app.rs 0.00% 1 Missing ⚠️
.../servers/apis/v1/context/health_check/resources.rs 66.66% 1 Missing ⚠️
src/servers/http/v1/handlers/health_check.rs 83.33% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #512      +/-   ##
===========================================
- Coverage    84.28%   84.20%   -0.08%     
===========================================
  Files          100      110      +10     
  Lines         7141     7289     +148     
===========================================
+ Hits          6019     6138     +119     
- Misses        1122     1151      +29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@josecelano josecelano requested a review from da2ce7 November 22, 2023 17:31
@josecelano josecelano force-pushed the issue-508-add-container-healthcheck branch from 2065ff8 to 42d5c89 Compare November 23, 2023 17:39
@josecelano josecelano force-pushed the issue-508-add-container-healthcheck branch from 42d5c89 to 6355e26 Compare November 23, 2023 17:41
@josecelano josecelano force-pushed the issue-508-add-container-healthcheck branch from 6355e26 to 394623c Compare November 24, 2023 06:38
It'll be used by a litle script (health check) that needs to make HTTP request.
It makes a request to an HTTP endpoint to check that the service is
healthy.
New Health CHeck API, but it is not checking anything yet.

You can call it at:

http://localhost:1313/health_check
The app health check endpoint checks is the API is running healthy when
is enabled.
http://localhost:7070/health_check

And call the endpoint from the general application health check
endpoint:

http://localhost:1313/health_check

Do not call the endpoint if:

- The tracker is disabled.
- The tracker configuration uses port 0 only knwon after starting the
  socket.

todo: call the enpoint also when the port is 0 in the configuration. THe
service can return back to the main app the port assiged by the OS. And
the app can pass that port to the global app health check handler.
@josecelano josecelano force-pushed the issue-508-add-container-healthcheck branch from 80044c2 to 7421306 Compare November 24, 2023 17:08
We will use the UDP tracker connection request to check if the UDP
tracker service is healthy.
Using the `connect` UDP request. If the UDP trackers replies to a
connection request we assume is healthy.
@josecelano josecelano requested a review from mario-nt November 24, 2023 18:02
@josecelano josecelano marked this pull request as ready for review November 24, 2023 18:08
@josecelano
Copy link
Member Author

Hi @da2ce7 @mario-nt @WarmBeer this is ready for review.

For the time being, it only actually checks the internal services (API, UDP and HTTP tracker) if the port defined in the configuration is not 0. It's a big effort to get the real port back after launching those services. We have some pending issues for that. We can improve this PR after refactoring those services. I think this solution is enough at least to check that containers are "ready".

If you think it's a good solution I can add the missing tests at least for the part that it's easily testable.

@josecelano
Copy link
Member Author

ACK 5e0a686

@josecelano josecelano merged commit ae8ef29 into torrust:develop Nov 27, 2023
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Implement health check for the tracker application
1 participant