Switch Python client backend to use REST (#152)

* py: add generated REST backend

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* py: initial generated code

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* py: remove gRPC backend

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* py: rename exceptions

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* py: wrap generated types

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* py: core: wrap async client with ctx manager

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* py: client: wrap async core with sync client

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* GHA: update python constraints

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

* py: setup mypy

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

---------

Signed-off-by: Isabella do Amaral <idoamara@redhat.com>

.github/workflows/build-image-pr.yml

@@ -80,6 +80,6 @@ jobs:
           pip install dist/*.whl
       - name: Connect with Python client
         run: |
-          kubectl port-forward -n kubeflow service/model-registry-service 9090:9090 &
+          kubectl port-forward -n kubeflow service/model-registry-service 8080:8080 &
           sleep 5
-          python test/python/test_mr_conn.py localhost 9090
+          python test/python/test_mr_conn.py http://localhost 8080

.github/workflows/constraints.txt

@@ -1,5 +1,6 @@
-pip==23.3.1
-nox==2023.04.22
-nox-poetry==1.0.2
-poetry==1.6.1
+pip==23.3.2
+nox==2023.4.22
+nox-poetry==1.0.3
+poetry==1.8.3
+poetry-plugin-export==1.8.0
 virtualenv==20.24.6

.github/workflows/python-release.yml

@@ -20,7 +20,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.10"
+          python-version: "3.12"
       - name: Upgrade pip
         run: |
           pip install --constraint=.github/workflows/constraints.txt pip

.github/workflows/python-tests.yml

@@ -11,11 +11,15 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python: ["3.10"]
+        python: ["3.12"]
         session: [lint, tests, mypy, docs-build]
         include:
           - python: "3.9"
             session: tests
+          - python: "3.10"
+            session: tests
+          - python: "3.11"
+            session: tests
     env:
       NOXSESSION: ${{ matrix.session }}
       FORCE_COLOR: "1"

clients/python/Makefile

@@ -0,0 +1,36 @@
+.PHONY: install
+install:
+	../../bin/openapi-generator-cli generate -i ../../api/openapi/model-registry.yaml -g python -o src/ --package-name mr_openapi --additional-properties=library=asyncio,generateSourceCodeOnly=true,useOneOfDiscriminatorLookup=true
+	mv src/mr_openapi{_,/}README.md
+	git apply patches/*
+	poetry install
+
+.PHONY: clean
+clean:
+	rm -rf src/mr_openapi
+
+.PHONY: test
+test:
+	poetry run pytest -s
+
+.PHONY: lint-check
+lint-check:
+	poetry run ruff check
+	poetry run black src/mr_openapi --check
+
+.PHONY: lint-apply
+lint-apply:
+	poetry run ruff check --fix --unsafe-fixes || true
+	poetry run black src/mr_openapi
+
+.PHONY: build
+build: install lint-apply
+	poetry build
+
+.PHONY: publish
+publish: install
+	poetry publish --build -u ${PYPI_USERNAME} -p ${PYPI_PASSWORD}
+
+.PHONY: update
+update:
+	poetry lock

clients/python/README.md

@@ -10,9 +10,9 @@ This library provides a high level interface for interacting with a model regist
 ```py
 from model_registry import ModelRegistry
 
-registry = ModelRegistry("server-address", author="Ada Lovelace")  # Defaults to a secure connection via port 443
+registry = ModelRegistry("https://server-address", author="Ada Lovelace")  # Defaults to a secure connection via port 443
 
-# registry = ModelRegistry("server-address", 1234, author="Ada Lovelace", is_secure=False)  # To use MR without TLS
+# registry = ModelRegistry("http://server-address", 1234, author="Ada Lovelace", is_secure=False)  # To use MR without TLS
 
 model = registry.register_model(
     "my-model",  # model name
@@ -34,9 +34,9 @@ model = registry.register_model(
 
 model = registry.get_registered_model("my-model")
 
-version = registry.get_model_version("my-model", "v2.0")
+version = registry.get_model_version("my-model", "2.0.0")
 
-experiment = registry.get_model_artifact("my-model", "v2.0")
+experiment = registry.get_model_artifact("my-model", "2.0.0")
 ```
 
 ### Importing from S3
@@ -102,51 +102,21 @@ There are caveats to be noted when using this method:
     )
     ```
 
-## Advanced use-cases
-
-### Using Model Registry Python Client with newer Python versions (>=3.11)
-
-> [!CAUTION]
-> The mechanism described in this section is a temporary workaround and likely will never be supported.
-> This workaround is ONLY applicable if your Python/Notebook project does NOT make use of MLMD directly or indirectly.
-
-<!-- a longer-term plan to address this ties to the investigations to rebase this client on top of MR REST api directly,
-so to avoid having to wrap the MLMD Wheel. See more: https://github.com/kubeflow/model-registry/pull/59 -->
-
-This project _currently_ depends for internal implementations on the Google's [MLMD Python library](https://pypi.org/project/ml-metadata/).
-Due to this dependency, this project supports [only the Python versions](https://github.com/kubeflow/model-registry/blob/8d77c13100c6cc5a9465d4293403114a3576fdd7/clients/python/pyproject.toml#L14) which are also available for the MLMD library (see more [here](https://pypi.org/project/ml-metadata/#files)).
-
-As a workaround, **only IF your Python/Notebook project does NOT make use of MLMD directly or indirectly**,
-you could opt-in to make use of a non-constrained variant of the MLMD dependency supporting _only_ remote gRPC calls (and not constrained by specific Python versions or architectures):
-
-```
-!pip install "https://github.com/opendatahub-io/ml-metadata/releases/download/v1.14.0%2Bremote.1/ml_metadata-1.14.0+remote.1-py3-none-any.whl" # need a Python 3.11 compatible version
-!pip install --no-deps --ignore-requires-python --pre "model-registry" # ignore dependencies because of the above override
-```
-
-You can read more about this use-case, in the [Remote-only packaging of MLMD Python lib](https://github.com/kubeflow/model-registry/blob/main/docs/remote_only_packaging_of_MLMD_Python_lib.md) document.
-
 ## Development
 
 Common tasks, such as building documentation and running tests, can be executed using [`nox`](https://github.com/wntrblm/nox) sessions.
 
 Use `nox -l` to list sessions and execute them using `nox -s [session]`.
 
-### Running Locally on Mac M1 or M2 (arm64 architecture)
-
-If you want run tests locally you will need to set up a development environment, including docker engine; we recommend following the instructions [here](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#docker-engine).
+Alternatively, use `make install` to setup a local Python virtual environment with `poetry`.
 
-You will also have to change the package source to one compatible with ARM64 architecture. This can be actioned by uncommenting lines 14 or 15 in the pyproject.toml file. Run the following command after you have uncommented the line.
+To run the tests you will need `docker` (or equivalent) and the `compose` extension command.
+This is necessary as the test suite will manage a Model Registry server and an MLMD instance to ensure a clean state on
+each run.
+You can use `make test` to execute `pytest`.
 
-```sh
-poetry lock
-```
-
-Use the following commands to directly run the tests with individual test output. Alternatively you can use the nox session commands above.
+### Running Locally on Mac M1 or M2 (arm64 architecture)
 
-```sh
-poetry install
-poetry run pytest -v
-```
+Check out our [recommendations on setting up your docker engine](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#docker-engine) on an ARM processor.
 
 <!-- github-only -->