Implement real liveness and readiness endpoints

[gkumbhat]

Description

Currently we a health check endpoint which is stubbed out version. We need to implement real health check endpoint that is able to measure health of the server.

Tasks

• Figure out how to measure liveness of orchestr8 server
• Implement liveness endpoint
• Figure out how to measure readiness of orchestr8 server
• implement liveness endpoint

Acceptance Criteria

• Liveness and readiness probes implemented in orchestr8 server and merged to main branch

[gkumbhat]

Brainstorming a bit on this topic:

• Liveness
  • we want this probe to help us / kubernetes check if the server stay up and running
  • Potential solutions:
    • Make a call to server and see if it is able to accept request (healthcheck endpoint would suffice)
• Readiness
  • we want this probe to help us / k8 check if the server is ready to take connections and serve requests.
  • To start with, we can check if the detector is able to connect / call out to chunker or not.
  • To not put pressure on chunker with all these probe calls, we can instead call out to healthcheck endpoints for chunkers from our readiness probe, which would be much lighter weight.

[evaline-ju]

• To start with, we can check if the detector is able to connect / call out to chunker or not.

Q - is the idea here to make sure we test with an expected available chunker that we parse the list of configured detectors and check a call to the particular chunker?

[gkumbhat]

is the idea here to make sure we test with an expected available chunker that we parse the list of configured detectors and check a call to the particular chunker?

@evaline-ju yes, precisely