-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support partial/incomplete batch job results #430
Comments
FYI: in the "large area" feature, we aim to split a very large job in smaller jobs at the level of the aggregator, and distribute this work as separate "sub" batch jobs on one or more back-ends. It would be frustrating if partial results would not be accessible because one "sub" batch jobs fails or is stuck. |
Adding this to the existing endpoint sounds reasonable although it could be an issue that existing clients could start downloading and propagating partial results as "complete" as the semantic for providing partial results was not present before. My first idea was using status code 206 instead of 200, but that doesn't fully fit. How would you envision this to work with things like |
My current thought is to have explicit opt-in to view partial results to stay backward compatible (because it would behaviorally be a breaking change):
opt-in could be with a request parameter I'm not sure about the 206 status for partial results, as that seems to be more about returning a targeted subrange of the result, chosen by the user request. But I'm not that familiar with usage in the wild of 206, so I don't have strong opinion against it either. |
Yeah, I think a parameter works for me... |
A first draft is now available in PR #433 |
openeo-api/openapi.yaml
Lines 3044 to 3045 in f303d65
The API currently requires that a batch job is fully finished before results can be requested with
/jobs/{job_id}/results
.In the context of very large batch jobs I think it could be useful to relax this and already allow listing of incomplete results (properly indicating that result is incomplete of course).
I'm not sure what is the best option: changing the behavior of the existing endpoint
/jobs/{job_id}/results
, or adding a parameter to enable this partial listing, or adding a new endpoint, or ...re: https://github.com/openEOPlatform/architecture-docs/issues/12
The text was updated successfully, but these errors were encountered: