Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,14 @@ You can also use pip:
pip install polaris-lib
```

## Authentication

Set an API key in your environment variables file for programmatic access to the Polaris Hub.

```bash
POLARIS_API_KEY="<your_api_key>"
```

## Development lifecycle

### Setup dev environment
Expand Down
5 changes: 0 additions & 5 deletions docs/api/hub.external_client.md

This file was deleted.

26 changes: 15 additions & 11 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,21 +25,25 @@ Polaris explicitly distinguished **datasets** and **benchmarks**.
One dataset can therefore be associated with multiple benchmarks.

## Login
To submit or upload artifacts to the [Polaris Hub](https://polarishub.io/) from the client, you must first authenticate yourself. If you don't have an account yet, you can create one [here](https://polarishub.io/sign-up).
To submit or upload artifacts to the [Polaris Hub](https://polarishub.io/) from the client, you must first authenticate yourself. If you don't have an account yet, you can create one [here](https://polarishub.io/auth/sign-up).

You can do this via the following command in your terminal:
Use an API key for programmatic access. Create one from your Hub settings page under the "Security" tab and set it as an environment variable in your .env:

```bash
polaris login
```

or in Python:
```py
from polaris.hub.client import PolarisHubClient

with PolarisHubClient() as client:
client.login()
POLARIS_API_KEY="<your_api_key>"
```
- and run the following command in your terminal:
```bash
polaris login
```

- or in Python:
```py
from polaris.hub.client import PolarisHubClient

with PolarisHubClient() as client:
client.login()
```

## Benchmark API
To get started, we will submit a result to the [`polaris/hello-world-benchmark`](https://polarishub.io/benchmarks/polaris/hello-world-benchmark).
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/submit_to_benchmark.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
"metadata": {},
"source": [
"## Login\n",
"We first need to authenticate ourselves using our Polaris account. If you don't have an account yet, you can create one [here](https://polarishub.io/sign-up)."
"We first need to authenticate ourselves using our Polaris account. If you don't have an account yet, you can create one [here](https://polarishub.io/auth/sign-up)."
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/submit_to_competition.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@
"metadata": {},
"source": [
"## Login\n",
"As before, we first need to authenticate ourselves using our Polaris account. If you don't have an account yet, you can create one [here](https://polarishub.io/sign-up)."
"As before, we first need to authenticate ourselves using our Polaris account. If you don't have an account yet, you can create one [here](https://polarishub.io/auth/sign-up)."
]
},
{
Expand Down
1 change: 0 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ nav:
- Evaluation: api/evaluation.md
- Hub:
- Client: api/hub.client.md
- External Auth Client: api/hub.external_client.md
- Additional:
- Base classes: api/base.md
- Types: api/utils.types.md
Expand Down
7 changes: 2 additions & 5 deletions polaris/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,16 @@ def login(
client_env_file: Annotated[
str, typer.Option(help="Environment file to overwrite the default environment variables")
] = ".env",
auto_open_browser: Annotated[
bool, typer.Option(help="Whether to automatically open the link in a browser to retrieve the token")
] = True,
overwrite: Annotated[
bool, typer.Option(help="Whether to overwrite the access token if you are already logged in")
] = False,
):
"""Authenticate to the Polaris Hub.

This CLI will use the OAuth2 protocol to gain token-based access to the Polaris Hub API.
Set POLARIS_API_KEY in your environment and run this command to cache a Hub token.
"""
client = PolarisHubClient(settings=PolarisHubSettings(_env_file=client_env_file))
client.login(auto_open_browser=auto_open_browser, overwrite=overwrite)
client.login(overwrite=overwrite)


@app.command(hidden=True)
Expand Down
81 changes: 35 additions & 46 deletions polaris/hub/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
from polaris.dataset import DatasetV1, DatasetV2
from polaris.evaluate import BenchmarkResultsV1, BenchmarkResultsV2, CompetitionPredictions
from polaris.prediction._predictions_v2 import BenchmarkPredictionsV2
from polaris.hub.external_client import ExternalAuthClient
from polaris.hub.oauth import CachedTokenAuth
from polaris.hub.settings import PolarisHubSettings
from polaris.hub.storage import StorageSession
Expand Down Expand Up @@ -114,66 +113,49 @@ def __init__(
**kwargs,
)

# We use an external client to get an auth token that can be exchanged for a Polaris Hub token
self.external_client = ExternalAuthClient(
settings=self.settings, cache_auth_token=cache_auth_token, **kwargs
)

def __enter__(self: Self) -> Self:
super().__enter__()
return self

@property
def has_user_password(self) -> bool:
return bool(self.settings.username and self.settings.password)

def _prepare_token_endpoint_body(self, body, grant_type, **kwargs):
"""
Override to support required fields for the token exchange grant type.
See https://datatracker.ietf.org/doc/html/rfc8693#name-request
"""
if grant_type == "urn:ietf:params:oauth:grant-type:token-exchange":
kwargs.update(
{
"subject_token": self.external_client.token["access_token"],
"subject_token_type": "urn:ietf:params:oauth:token-type:access_token",
"requested_token_type": "urn:ietf:params:oauth:token-type:jwt",
}
)
return super()._prepare_token_endpoint_body(body, grant_type, **kwargs)

def ensure_active_token(self, token: OAuth2Token | None = None) -> bool:
"""
Override the active check to trigger a refetch of the token if it is not active.
"""
# This won't be needed with if we set a lower bound for authlib: >=1.3.2
# See https://github.com/lepture/authlib/pull/625
# As of now, this latest version is not available on Conda though.
token = token or self.token
is_active = super().ensure_active_token(token) if token else False
if is_active:
return True

# Check if external token is still valid, or we're using password auth
if not (self.has_user_password or self.external_client.ensure_active_token()):
return False
# If we have an API key, use it to get a new Hub JWT
if self.settings.api_key:
self.token = self.fetch_token()
return True

# If so, use it to get a new Hub token
self.token = self.fetch_token()
return True
return False

def fetch_token(self, **kwargs):
"""
Handles the optional support for password grant type, and provide better error messages.
Fetch a Hub JWT using the API key grant.
"""
try:
return super().fetch_token(
username=self.settings.username,
password=self.settings.password,
grant_type="password"
if self.has_user_password
else "urn:ietf:params:oauth:grant-type:token-exchange",
**kwargs,
if self.settings.api_key:
return super().fetch_token(
grant_type="api_key",
api_key=self.settings.api_key,
**kwargs,
)
# No API key set: raise a clear error
raise PolarisHubError(
message=(
"No API key configured. Please set POLARIS_API_KEY or pass settings.api_key to PolarisHubClient."
)
)
except (OAuthError, OAuth2Error) as error:
raise PolarisHubError(
Expand All @@ -190,6 +172,13 @@ def _base_request_to_hub(self, url: str, method: str, withhold_token: bool, **kw
response.raise_for_status()
return response
except HTTPStatusError as error:
# 401: try one transparent retry if we can refresh via API key
if error.response.status_code == 401 and not withhold_token and bool(self.settings.api_key):
self.token = self.fetch_token()
response = self.request(url=url, method=method, withhold_token=withhold_token, **kwargs)
response.raise_for_status()
return response

# If JSON is included in the response body, we retrieve it and format it for output. If not, we fall back to
# retrieving plain text from the body. 500 errors will not have a JSON response.
try:
Expand Down Expand Up @@ -229,20 +218,20 @@ def request(self, method, url, withhold_token=False, auth=httpx.USE_CLIENT_DEFAU
except (MissingTokenError, InvalidTokenError, OAuthError) as error:
raise PolarisUnauthorizedError() from error

def login(self, overwrite: bool = False, auto_open_browser: bool = True):
"""Login to the Polaris Hub using the OAuth2 protocol.
def login(self, overwrite: bool = False):
"""Login to the Polaris Hub.

Warning: Headless authentication
It is currently not possible to login to the Polaris Hub without a browser.
See [this Github issue](https://github.com/polaris-hub/polaris/issues/30) for more info.

Args:
overwrite: Whether to overwrite the current token if the user is already logged in.
auto_open_browser: Whether to automatically open the browser to visit the authorization URL.
If an API key is configured, a Hub JWT will be fetched and cached; otherwise an error is raised.
"""
if overwrite or self.token is None or not self.ensure_active_token():
self.external_client.interactive_login(overwrite=overwrite, auto_open_browser=auto_open_browser)
if self.settings.api_key:
# Force-fetch and cache a Hub JWT
self.token = self.fetch_token()
elif overwrite or self.token is None or not self.ensure_active_token():
raise PolarisHubError(
message=(
"No API key configured. Please set POLARIS_API_KEY in your environment variables file."
)
)

logger.info("You are successfully logged in to the Polaris Hub.")

Expand Down
Loading