-
Notifications
You must be signed in to change notification settings - Fork 835
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
6 changed files
with
267 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # Circuit Breakers with Seldon and Ambassador | ||
|
|
||
| Follow the [notebook](ambassador_circuit_breakers.ipynb) to show how you can deploy Seldon Deployments with circuit breakers. | ||
|
|
193 changes: 193 additions & 0 deletions
193
examples/ambassador/circuit_breakers/ambassador_circuit_breakers.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,193 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Circuit Breakers with Seldon and Ambassador\n", | ||
| "\n", | ||
| "This notebook shows how you can deploy Seldon Deployments which can have circuit breakers via Ambassador's circuit breakers configuration.\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Setup Seldon Core\n", | ||
| "\n", | ||
| "Use the setup notebook to [Setup Cluster](../../seldon_core_setup.ipynb#Setup-Cluster) with [Ambassador Ingress](../../seldon_core_setup.ipynb#Ambassador) and [Install Seldon Core](../../seldon_core_setup.ipynb#Install-Seldon-Core). Instructions [also online](./seldon_core_setup.html)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!kubectl create namespace seldon" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!kubectl config set-context $(kubectl config current-context) --namespace=seldon" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Launch main model\n", | ||
| "\n", | ||
| "We will create a very simple Seldon Deployment with a dummy model image `seldonio/mock_classifier:1.0`. This deployment is named `example`. We will add following circuit breakers configurations.\n", | ||
| "\n", | ||
| "```\n", | ||
| " \"seldon.io/ambassador-circuit-breakers-max-connections\":\"200\",\n", | ||
| " \"seldon.io/ambassador-circuit-breakers-max-pending-requests\":\"100\",\n", | ||
| " \"seldon.io/ambassador-circuit-breakers-max-requests\":\"200\",\n", | ||
| " \"seldon.io/ambassador-circuit-breakers-max-retries\":\"3\"\n", | ||
| "```\n", | ||
| "\n", | ||
| "Where\n", | ||
| "\n", | ||
| " * `\"seldon.io/ambassador-circuit-breakers-max-connections\":\"200\"` is the maximum number of connections will make to the Seldon Deployment\n", | ||
| " * `\"seldon.io/ambassador-circuit-breakers-max-pending-requests\":\"100\"` is the maximum number of requests that will be queued while waiting for a connection\n", | ||
| " * `\"seldon.io/ambassador-circuit-breakers-max-requests\":\"200\"` is the maximum number of parallel outstanding requests to the Seldon Deployment\n", | ||
| " * `\"seldon.io/ambassador-circuit-breakers-max-retries\":\"3\"` the maximum number of parallel retries allowed to the Seldon Deployment\n", | ||
| " " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!pygmentize model_circuit_breakers_ambassador.json" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!kubectl create -f model_circuit_breakers_ambassador.json" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=production-model-example -o jsonpath='{.items[0].metadata.name}')" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Get predictions" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from seldon_core.seldon_client import SeldonClient\n", | ||
| "sc = SeldonClient(deployment_name=\"example\",namespace=\"seldon\")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "#### REST Request" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "r = sc.predict(gateway=\"ambassador\",transport=\"rest\")\n", | ||
| "assert(r.success==True)\n", | ||
| "print(r)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!kubectl delete -f model_circuit_breakers_ambassador.json" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "anaconda-cloud": {}, | ||
| "kernelspec": { | ||
| "display_name": "Python 3", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.7.4" | ||
| }, | ||
| "varInspector": { | ||
| "cols": { | ||
| "lenName": 16, | ||
| "lenType": 16, | ||
| "lenVar": 40 | ||
| }, | ||
| "kernels_config": { | ||
| "python": { | ||
| "delete_cmd_postfix": "", | ||
| "delete_cmd_prefix": "del ", | ||
| "library": "var_list.py", | ||
| "varRefreshCmd": "print(var_dic_list())" | ||
| }, | ||
| "r": { | ||
| "delete_cmd_postfix": ") ", | ||
| "delete_cmd_prefix": "rm(", | ||
| "library": "var_list.r", | ||
| "varRefreshCmd": "cat(var_dic_list()) " | ||
| } | ||
| }, | ||
| "types_to_exclude": [ | ||
| "module", | ||
| "function", | ||
| "builtin_function_or_method", | ||
| "instance", | ||
| "_Feature" | ||
| ], | ||
| "window_display": false | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 1 | ||
| } |
46 changes: 46 additions & 0 deletions
46
examples/ambassador/circuit_breakers/model_circuit_breakers_ambassador.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| { | ||
| "apiVersion": "machinelearning.seldon.io/v1alpha2", | ||
| "kind": "SeldonDeployment", | ||
| "metadata": { | ||
| "labels": { | ||
| "app": "seldon" | ||
| }, | ||
| "name": "example" | ||
| }, | ||
| "spec": { | ||
| "name": "production-model", | ||
| "annotations": { | ||
| "seldon.io/ambassador-circuit-breakers-max-connections":"200", | ||
| "seldon.io/ambassador-circuit-breakers-max-pending-requests":"100", | ||
| "seldon.io/ambassador-circuit-breakers-max-requests":"200", | ||
| "seldon.io/ambassador-circuit-breakers-max-retries":"3" | ||
| }, | ||
| "predictors": [ | ||
| { | ||
| "componentSpecs": [{ | ||
| "spec": { | ||
| "containers": [ | ||
| { | ||
| "image": "seldonio/mock_classifier_rest:1.4", | ||
| "imagePullPolicy": "IfNotPresent", | ||
| "name": "classifier" | ||
| } | ||
| ], | ||
| "terminationGracePeriodSeconds": 1 | ||
| }} | ||
| ], | ||
| "graph": | ||
| { | ||
| "children": [], | ||
| "name": "classifier", | ||
| "type": "MODEL", | ||
| "endpoint": { | ||
| "type": "REST" | ||
| }}, | ||
| "name": "single", | ||
| "replicas": 1 | ||
| } | ||
| ] | ||
| } | ||
| } | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters