Seldon Components Usage Examples (#1036)

* adding seldon inference graph that uses input transformer for spam classification example

* adding seldon inference graph that uses input transformer for spam classification example

* commiting contents of classifier

* updated readme

* deleted

* Delete .DS_Store

* updated readme

* updated readme

* example for Seldon combiner component

* Delete .DS_Store

* more descriptive names for classifiers

* Delete deploy.yaml

* adding Translator

* adding readme files

* updating readme files

* upating readme

* adding inference graph image

* updated README

* adding model pipeline image and updating readme"

* adding model pipeline image and updating readme"

* README updated"

* graph explanation pic

* updated readme

* updated readme

* graph explanation pic

* updated readme

* added dockerhub path for docker images

* updated deploy.yaml

* updated README

* updated Combiner.py

* updated Combiner.py

* updated Combiner.py

* adding graph explanation

* updated README

* updated README

* updated README

* updated README

* updated README

* updated README files

* updated README files

* updated README files

* working example with correct model file and vectorizer

* Delete .DS_Store

* Delete Untitled.ipynb

* Delete .DS_Store

* added example of how to test the image

* added example of how to test the image

* updated code and README for Translator image

* updated skl-spam-classifier

* added keras model architecture files along with working python code

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

* updated code for Translator plus added requirements

* updated code for Translator plus added requirements

* updated readme

* updated readme

* added .gitignore

* added .gitignore

* Delete .DS_Store

* tested combiner image

* updated requirements

* updated src code

* deployment files to work as in on k8s cluster assuming seldon-core is installed

* updated Readme

* updated Readme

* Delete .DS_Store

* notebook file to generate the model binaries

* notebook file to generate the model binaries

* notebook files to train the model locally and build docker images using s2i

* renamed notebook files"

* renamed notebook file

* renamed notebook file

* nblink for combiner example

* nblink for transformer example

* Delete input_transformer_spam.nblink

* added path to new nblink files

* deleted model pkl files

* deleted model pkl files

doc/source/examples/combiner_spam_model.nblink

@@ -0,0 +1,3 @@
+{
+  "path": "../../../examples/combiners/spam_clf_combiner/spam-classification.ipynb"
+}

doc/source/examples/notebooks.rst

@@ -47,4 +47,6 @@ Notebooks
    Sklearn, Xgboost, Tensorflow Server Examples <server_examples>
    Tensorflow MNIST <deep_mnist>
    TFserving MNIST <tfserving_mnist>
+   Spam Classification TRANSFORMER component <transformer_spam_model>
+   Spam Classification TRANSFORMER + COMBINER Component <combiner_spam_model>
 

doc/source/examples/transformer_spam_model.nblink

@@ -0,0 +1,3 @@
+{
+  "path": "../../../examples/input_tranformer/spam-classification.ipynb"
+}

examples/combiners/spam_clf_combiner/.gitignore

@@ -0,0 +1 @@
+**/.DS_Store

examples/combiners/spam_clf_combiner/Combiner/.s2i/environment

@@ -0,0 +1,4 @@
+MODEL_NAME=Combiner
+API_TYPE=REST
+SERVICE_TYPE=COMBINER
+PERSISTENCE=0

examples/combiners/spam_clf_combiner/Combiner/Combiner.py

@@ -0,0 +1,22 @@
+import numpy as np
+
+class Combiner(object):
+
+    def aggregate(self, Xs, features_names=None):
+    	"""average out the probabilities from multiple classifier and return that as a result"""
+    	return np.mean([float(x[0]) for x in Xs])
+
+
+
+# if __name__== "__main__":
+#     clf1_res = np.array(['0.80', 'Spam'])
+#     clf2_res = np.array(['0.9959868467126312e-04', 'Spam'])
+
+#     example = np.array([clf1_res, clf2_res])
+#     combine = Combiner()
+#     res = combine.aggregate(example, features_names=None)
+#     print(res)
+
+
+
+

examples/combiners/spam_clf_combiner/Combiner/README.md

@@ -0,0 +1,33 @@
+# **Combiner**
+
+
+
+#### Activate Virtual Env
+
+Create a Python virtual environment:
+
+```
+    $ python3 -m venv venv
+    $ source venv/bin/activate
+```
+
+#### Install s2i
+
+
+    linux: https://computingforgeeks.com/install-source-to-image-toolkit-on-linux/
+    mac:  brew install source-to-image
+
+
+#### Build
+
+```
+$ s2i build Combiner/ seldonio/seldon-core-s2i-python3:0.7 combiner:1.0.0.1
+$ docker push <yourdockerhubusername>/combiner:1.0.0.1
+```
+
+#### Test
+
+```
+$ docker run --name "Combiner" --rm <yourdockerhubusername>/combiner:1.0.0.1
+
+```

examples/combiners/spam_clf_combiner/README.md

@@ -0,0 +1,23 @@
+
+#### Model Deployment using Seldon Transformer + Combiner Component
+
+Consider, we want to predict whether a text message is spam or not. The text data may contain multiple languages as well as each individual text may also have words from different languages. As per your needs, you are only interested in building a model on English language. So, you want to first translate the text message in English and then pass it to model to classify whether it is a spam or not. Moreover, we want to train two models: one that uses Neural network architecture using keras library and other that uses traditional machine learning classifier from scikit-learn. At runtime, we want to run both mdoels on a given input and output the average of the both predictions:
+
+Example: <br/>
+If classifier 1 outputs the probability of 0.7 for the message to be spam <br/>
+Classifier 2 outputs the probability of 0.8 for the message to be spam <br/>
+The final model using seldon-core graph would output the average probability of both classifiers: 0.75 
+
+
+
+![Model Pipeline](https://github.com/SandhyaaGopchandani/seldon-core/blob/seldon_component_example/examples/combiners/spam_clf_combiner/seldon_inference_graph.png)
+
+
+Each component in Seldon Inference graph is a microservice – specifically a docker container. The Seldon graph links those docker containers (in deploy.yaml file) to result in an end output. 
+
+
+![Graph Explanation](https://github.com/SandhyaaGopchandani/seldon-core/blob/seldon_component_example/examples/combiners/spam_clf_combiner/graph_explained.png)
+
+The repository contains the folders for for each docker image needed for seldon graph to work. The images used for this example are already published in public dockerhub. So, deploy the model on k8s cluster using:
+
+    kubectl apply -f deploy.yaml

examples/combiners/spam_clf_combiner/Translator/.s2i/environment

@@ -0,0 +1,4 @@
+MODEL_NAME=Translator
+API_TYPE=REST
+SERVICE_TYPE=TRANSFORMER
+PERSISTENCE=0

examples/combiners/spam_clf_combiner/Translator/README.md

@@ -0,0 +1,36 @@
+# **Translator**
+
+
+
+#### Activate Virtual Env
+
+Create a Python virtual environment:
+
+```
+$ python3 -m venv venv
+$ source venv/bin/activate
+```
+
+#### Install s2i
+
+
+    linux: https://computingforgeeks.com/install-source-to-image-toolkit-on-linux/
+    mac:  brew install source-to-image
+
+
+#### Build
+
+```
+$ s2i build . seldonio/seldon-core-s2i-python3:0.7 translator:1.0.0.1
+$ docker push <yourdockerhubusername>/translator:1.0.0.1
+```
+
+#### Test
+
+```
+$ docker run --name "translator" --rm <yourdockerhubusername>/translator:1.0.0.1
+
+curl -g http://localhost:5000/transform_input --data-urlencode 'json={"data": {"names": ["message"], "ndarray": ["Wie läuft dein Tag"]}}'
+
+
+```

examples/combiners/spam_clf_combiner/Translator/Translator.py

@@ -0,0 +1,26 @@
+import goslate
+import numpy as np
+
+class Translator():
+    def __init__(self):
+    	self.gs = goslate.Goslate()
+
+
+
+    def transform_input(self, text_msg, feature_names=None):
+
+        """the translator logic will go here. This shows a use of simple library. But translator service can be a Machine Learning model itself"""
+        translated = self.gs.translate(text_msg[0], 'en')
+        return np.array([translated])
+
+
+
+# if __name__== "__main__":
+#     t = Translator()
+#     example = np.array(['Wie läuft dein Tag'])
+#     translated = t.transform_input(example)
+#     print(translated)
+
+
+
+

examples/combiners/spam_clf_combiner/Translator/requirements.txt

@@ -0,0 +1,2 @@
+goslate
+numpy

examples/combiners/spam_clf_combiner/deploy.yaml

@@ -0,0 +1,58 @@
+---
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  labels:
+    app: seldon
+  name: seldon-spam-classifier
+  namespace: default
+spec:
+  annotations:
+    project_name: seldon-spam-classifier
+    deployment_version: v1
+    seldon.io/rest-read-timeout: '100000'
+    seldon.io/rest-connection-timeout: '100000'
+    seldon.io/grpc-read-timeout: '100000'
+  name: seldon-spam-classifier
+  predictors:
+  - componentSpecs:
+    - spec:
+        containers:
+        - name: combiner
+          image: sandhya1594/combiner:1.0.0.1
+          imagePullPolicy: IfNotPresent
+        - name: spam-classifier
+          image: sandhya1594/spam-classifier:1.0.0.1
+          imagePullPolicy: IfNotPresent
+        - name: keras-spam-classifier
+          image: sandhya1594/keras-spam-classifier:1.0.0.1
+          imagePullPolicy: IfNotPresent
+        - name: translator
+          image: sandhya1594/translator:1.0.0.1
+          imagePullPolicy: IfNotPresent
+        terminationGracePeriodSeconds: 20
+    graph:
+      children:
+      - name: combiner
+        endpoint:
+          type: REST
+        type: COMBINER
+        children:
+        - name: keras-spam-classifier
+          endpoint:
+            type: REST
+          type: MODEL
+          children: []
+        - name: spam-classifier
+          endpoint: 
+            type: REST
+          type: MODEL
+          children: []
+      name: translator
+      endpoint:
+        type: REST
+      type: TRANSFORMER
+    name: seldon-combiner
+    replicas: 1
+    annotations:
+      predictor_version: v1

examples/combiners/spam_clf_combiner/keras-spam-classifier/.s2i/environment

@@ -0,0 +1,4 @@
+MODEL_NAME=KerasSpamClassifier
+API_TYPE=REST
+SERVICE_TYPE=MODEL
+PERSISTENCE=0

examples/combiners/spam_clf_combiner/keras-spam-classifier/KerasSpamClassifier.py

@@ -0,0 +1,44 @@
+from pathlib import Path
+import numpy as np
+import pickle
+from sklearn.externals import joblib 
+from keras.preprocessing import sequence
+from keras.engine.saving import model_from_json
+
+class KerasSpamClassifier():
+
+    def __init__(self, model_path = Path('./model')):
+        self._architecture_path = model_path.joinpath('architecture.json')
+        self._weights_path = model_path.joinpath('weights.h5')
+        tokenizer_path = model_path.joinpath('tokenizer.pkl')
+
+        self.max_len = 150
+        # loading of trained tokenizer
+        with tokenizer_path.open('rb') as handle:
+            self.tokenizer = joblib.load(handle)
+
+        with self._architecture_path.open() as f:
+            self.model = model_from_json(f.read())
+            self.model.load_weights(self._weights_path.as_posix())
+            self.model._make_predict_function()
+
+
+
+    def predict(self, text, feature_names=None): #List[Tuple[float, float]]:
+        """
+        Predict on a english text you got from translator service. The output returns the probability of text being spam
+        """
+        seq= self.tokenizer.texts_to_sequences([text])
+        padded = sequence.pad_sequences(seq, maxlen=self.max_len)
+        prob = self.model.predict(padded)
+        prob = prob[0][0]
+        return np.array([prob, "spam"])
+
+
+# if __name__== "__main__":
+
+#     text = 'please click this link to win the price'
+#     #vect_path=Path('/Users/sandhya.sandhya/Desktop/data/doc/data'),
+#     sc = KerasSpamClassifier(model_path=Path('./model'))
+#     res = sc.predict(text, feature_names=None)
+#     print(res)

examples/combiners/spam_clf_combiner/keras-spam-classifier/README.md

@@ -0,0 +1,45 @@
+# **Spam Classifier**
+
+
+
+#### Activate Virtual Env
+
+Create a Python virtual environment:
+
+```
+    $ python3 -m venv venv
+    $ source venv/bin/activate
+```
+
+#### Install s2i
+
+
+    linux: https://computingforgeeks.com/install-source-to-image-toolkit-on-linux/
+    mac:  brew install source-to-image
+
+
+#### Build
+
+```
+$ s2i build . seldonio/seldon-core-s2i-python3:0.7 keras-spam-classifier:1.0.0.1
+$ docker push <yourdockerhubusername>/keras-spam-classifier:1.0.0.1
+```
+
+#### Test
+
+```
+$ docker run --name "spam-classifier" --rm -d -p 5000:5000  keras-spam-classifier:1.0.0.1
+
+curl -g http://localhost:5000/predict --data-urlencode 'json={"data": {"names": ["message"], "ndarray": ["please click this link to win the price"]}}'
+
+Result:
+
+{"data":{"ndarray":["0.91942614","spam"]},"meta":{}}
+
+
+```
+
+
+
+
+

examples/combiners/spam_clf_combiner/keras-spam-classifier/requirements.txt

@@ -0,0 +1,3 @@
+scikit-learn==0.21.2
+numpy>=1.9.2
+keras

examples/combiners/spam_clf_combiner/skl-spam-classifier/.s2i/environment

@@ -0,0 +1,4 @@
+MODEL_NAME=SpamClassifier
+API_TYPE=REST
+SERVICE_TYPE=MODEL
+PERSISTENCE=0

examples/combiners/spam_clf_combiner/skl-spam-classifier/README.md

@@ -0,0 +1,43 @@
+# **Spam Classifier**
+
+
+
+#### Activate Virtual Env
+
+Create a Python virtual environment:
+
+```
+    $ python3 -m venv venv
+    $ source venv/bin/activate
+```
+
+#### Install s2i
+
+
+    linux: https://computingforgeeks.com/install-source-to-image-toolkit-on-linux/
+    mac:  brew install source-to-image
+
+
+#### Build
+
+```
+$ s2i build . seldonio/seldon-core-s2i-python3:0.7 spam-classifier:1.0.0.1
+$ docker push <yourdockerhubusername>/spam-classifier:1.0.0.1
+```
+
+#### Test
+
+```
+$ docker run --name "spam-classifier" --rm -d -p 5000:5000  spam-classifier:1.0.0.1
+
+curl -g http://localhost:5000/predict --data-urlencode 'json={"data": {"names": ["message"], "ndarray": ["click here to win the price"]}}'
+
+Result:
+
+{"data":{"ndarray":["0.9785519250237192","spam"]},"meta":{}}
+
+```
+
+
+
+