Add configuration value for the default readiness statuses (UP/DOWN) for empty check responses

[xstefank]

Maybe it would be useful to take into consideration the startup or bootstrap phase of the underlying container which may start before the actual checks are called. In this sense, the container may choose to respond with some predefined health check response before the user checks can be invoked.

Would it make sense to add a default status values properties for empty check responses for liveness and readiness checks? The default should be UP for liveness (so the user can then say that the app is down with a custom check) and DOWN for readiness (so the user can say when the app is ready). This would be useful to unify across containers if there is more than one container that uses this mechanism which would help with portability.

[pgunapal]

+1 for the idea.

To my understanding, for the line The default should be DOWN for readiness (so the user can say when the app is ready), if the user has empty checks (no user checks), won't the default status for Readiness check be always DOWN, even if the app is ready? Or do you mean that users can have the ability to configure the default statuses properties, expecting that they have defined user checks?

[xstefank]

@pgunapal yes, this issue is about configuring the values by the user. So if the user knows that there are no readiness checks defined, he/she will change the value for empty readiness to true.

Why did I suggest the default of readiness to be DOWN? Take this example:

1. -- health/ready --> connection refused
2. start container
3. -- health/ready --> connection refused (for some time)
4. container initialized but checks are not yet processed
5. -- health/ready --> returns empty readiness checks response
6. container bootstrapped and the user checks are processed
7. -- health/ready --> returns the user checks with the user checks response

Now take the step 5. -- health/ready --> returns empty readiness checks response:

If the empty check response is DOWN we have the following sequence from repeated /health/ready calls:

• connection refused
• connection refused
  ...
• connection refused
• empty check response -- DOWN
• empty check response -- DOWN
  ...
• empty check response -- DOWN
• user check response -- DOWN/UP

But if the empty check response is UP we have the following sequence from repeated /health/ready calls:

• connection refused
• connection refused
  ...
• connection refused
• empty check response -- UP --- This is wrong behavior because the container is saying that the traffic can be already delivered but the user readiness checks have not been processed
• remainder is not important

[donbourne]

@xstefank , I get why we'd want the default readiness check to return DOWN until all readiness checks have initialized, but I'm not clear on why this should be configurable. It seems like it would be easier if the server vendor tracked when the app(s) had initialized. If you had that ability then it seems the readiness check should return DOWN until app(s) had initialized, after which it would return whatever any readiness check returned (as normal) or UP (if no readiness checks exist).

[xstefank]

@donbourne correct, the configuration would be for empty responses which could be requested even after the user application (with no user-defined checks) is started. It doesn't make sense for the user to configure it to return down in such cases as app would be continuously restarted or the traffic would be prohibited.

The configuration would be helpful to vendor as for instance in readiness case you can respond UP sooner. But of course, it's questionable if this is worth the configuration support.

[xstefank]

we agreed on gitter that configuration for empty liveness doesn't make sense. So we will add only the configuration for empty readiness checks with the default value of Down (for now).

We will discuss the elaboration of the spec on the multip app deployments in the next hangout. It is possible that we may distinguish default value for readiness probe depending on whether we have a single or multip app runtime.