Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[batch] warns the user if client or server is using deprecated functionality #14621

Merged
merged 5 commits into from
Jul 31, 2024

Conversation

iris-garden
Copy link
Contributor

@iris-garden iris-garden commented Jul 17, 2024

Closes #14485.

Tested that this works by deploying the branch to my dev namespace and pointing my dev config at it:

hailctl dev deploy -b iris-garden/hail:batch/deprecated-apis -s deploy_batch,add_developers
hailctl dev config set default_namespace irademac

And then running the following:

from hailtop.batch import ServiceBackend
batch_client = await ServiceBackend(billing_project='test', remote_tmpdir='gs://irademac/test/')._batch_client()
# one of the deprecated endpoints
await batch_client._get("/api/v1alpha/batches/402/jobs/1/log")

@iris-garden iris-garden force-pushed the batch/deprecated-apis branch 2 times, most recently from 568fe78 to 51d3e57 Compare July 17, 2024 20:02
Copy link
Collaborator

@patrick-schultz patrick-schultz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Iris, this looks great! Just one concern.

Comment on lines 228 to 236
async def rest_get_version(request: web.Request) -> web.Response:
headers = None
server_version = version()
client_version = request.headers.get("X-Hail-Version")
if client_version is not None and client_version != server_version:
headers = {
"X-Hail-Deprecated": f"The version of the Batch Client ({client_version}) does not match the version of Batch running on the server ({server_version}). Please upgrade your Hail version to the latest release."
}
return web.Response(text=server_version, headers=headers)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think warning whenever there is a newer release is probably too noisy. For now lets just warn when they are specifically depending on deprecated functionality.

If we did do this, we'd also need to be sure we're only comparing the release version, not the commit hash. Otherwise, since we deploy batch on every commit, every user would be out of date as soon as any PR merges after a release.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! I had honestly not realized we deploy Batch on every commit. If at some point down the line we come up with a policy for e.g. how many patch versions the difference between client and server can be, we can always reintroduce the version checking, but for now I agree it doesn't need to be there. Removed!

Comment on lines 671 to 673
headers={
"X-Hail-Deprecated": 'The endpoint "/api/v1alpha/batches/<batch_id>/jobs/<job_id>/log" is deprecated. Please upgrade your Hail version to the latest release.'
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How many routes are we likely to deprecate? While this solution is fine I'm not too fond of the repeated route names.
Would a modified routes decorator that automatically adds this deprecation message be too complicated/overly fancy:

@get(route, deprecated=True)
async def get_foo(request: Request) -> Response: 
   ...


def get(path, deprecated: bool): 
    return route(routes.get, path, deprecated)

def post(path, deprecated):
    return route(routes.post, path, deprecated)

...

def route(verb, path, deprecated):
    def wrapper(fn):
        @verb(path)
        async def handler(request, *args, **kwargs):
            response = await fn(request, args, kwards)
            if deprecated:
                response.headers['X-Hail-Deprecated'] = (
                    f'The endpoint "{path}" is deprecated. '
                    'Please upgrade your Hail version to the latest release.'
                )
           return response
        return handler
    return wapper

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's fine if Iris wants to do it, but I expect deprecated endpoints to be pretty rare.

@@ -219,6 +219,18 @@ def cast_query_param_to_bool(param: Optional[str]) -> bool:
return True


def deprecated(fun):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a much nicer solution than I proposed. Thank you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Sorry, meant to reply to your comment after the tests ran but forgot

@hail-ci-robot hail-ci-robot merged commit 9cd2526 into hail-is:main Jul 31, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[hailtop] The Batch client should warn users when they are using deprecated APIs
4 participants