Deprecate and remove hail API tokens in favor of OAuth access tokens

[daniel-goldstein]

What happened?

#13131 Adds the ability for users to authenticate with the hail service using access tokens from their hail identity provider (GCP IAM or Azure AAD) instead of using tokens minted by the hail auth service. It does not, however, remove the old form of authentication. There are two actions that must be taken to fully remove the old authentication method:

1. In [auth] Allow use of cloud access tokens for hail batch authorization #13131, the hail python client attempts to use the new identity.json cloud credentials for authentication, but falls back to the old tokens.json credentials if present.
2. The auth server still supports the old /api/v1alpha/login endpoint. This is unused in the new authentication flow and should ultimately be removed.

Removing these old code paths can be done in a two-step process, first with deprecation/warnings (the user need only run hailctl auth login to start using the new code paths) and then with removal of the old code paths. This issue is considered complete when the new code paths are removed.

Version

0.2.120

Relevant log output

No response

[danking]

@daniel-goldstein Did we complete this?

[daniel-goldstein]

No, only step 1 up there has been completed. We currently accept both the old and new authentication tokens. We need to decide what kind of deprecation approach is appropriate here and then do it. Also as it currently stands copy-paste tokens require the old-style of authentication (I think because of an implementation detail that internally these tokens reference session_ids instead of users), so those either need to be migrated to be compatible with a user not having any session_ids in the database or removed entirely.

A useful bit of context though, the only thing users need to do to get off the old tokens is hailctl auth login on a hail version with the changes in #13131.

[danking]

I'm fine removing copy-paste-tokens. They were for a prototype with Terra. We obviously are pursuing a different approach now.

Hmm. I suppose old versions of hailctl have no way to know that the fix is to upgrade to a newer version of hailctl? Like, the server can't send a message in the auth failure? We can just ask our local users to upgrade. As long as there's a stable & robust version of query that they can rely on, I think they're happy to upgrade. Which version of hailctl is compatible with new auth?

[daniel-goldstein]

Based on when the above PR merged, it looks like >=0.2.122. Ya, I've long wanted a deprecation response header that clients blanket warn about when they receive it as a way to communicate the need for upgrading / moving off of old endpoints. Obviously doesn't help for this instance but seems good to have.

[danking]

That's a great idea and a great use of a response header.

OK, I think we might need to give them a couple months to settle into newer Hail versions. Does delaying step 2 delay any other work?

[daniel-goldstein]

AFAIK it is only a matter of deleting old code paths and secrets. I don't believe there is anything blocked on the old authentication tokens existing.