Move from a custom tornado server to a seldon-core server for serving the model

.gitignore

@@ -39,3 +39,7 @@ examples/.ipynb_checkpoints/
 
 # pyenv
 .python-version
+
+# Data files
+*.h5
+*.dpkl

github_issue_summarization/README.md

@@ -21,7 +21,7 @@ By the end of this tutorial, you should learn how to:
     datasets
 *   Train a Sequence-to-Sequence model using TensorFlow on the cluster using
     GPUs
-*   Serve the model using a Tornado Server
+*   Serve the model using [Seldon Core](https://github.com/SeldonIO/seldon-core/)
 
 ## Steps:
 

github_issue_summarization/notebooks/IssueSummarization.py

@@ -0,0 +1,22 @@
+from __future__ import print_function
+
+import dill as dpickle
+import numpy as np
+from keras.models import load_model
+
+from seq2seq_utils import Seq2Seq_Inference
+
+
+class IssueSummarization(object):
+
+    def __init__(self):
+        with open('body_pp.dpkl', 'rb') as f:
+            body_pp = dpickle.load(f)
+        with open('title_pp.dpkl', 'rb') as f:
+            title_pp = dpickle.load(f)
+        self.model = Seq2Seq_Inference(encoder_preprocessor=body_pp,
+                                       decoder_preprocessor=title_pp,
+                                       seq2seq_model=load_model('seq2seq_model_tutorial.h5'))
+
+    def predict(self, X, feature_names):
+        return np.asarray([[self.model.generate_issue_title(body[0])[1]] for body in X])

github_issue_summarization/notebooks/requirements.txt

@@ -0,0 +1,11 @@
+numpy
+keras
+dill
+matplotlib
+tensorflow
+annoy
+tqdm
+nltk
+IPython
+ktext
+h5py

github_issue_summarization/serving_the_model.md

@@ -1,21 +1,62 @@
 # Serving the model
 
-We are going to use a simple tornado server to serve the model. The [server.py](notebooks/server.py) contains the server code.
+We are going to use [seldon-core](https://github.com/SeldonIO/seldon-core) to serve the model. [IssueSummatization.py](notebooks/IssueSummatization.py) contains the code for this model. We will wrap this class into a seldon-core microservice which we can then deploy as a REST or GRPC API server.
 
-Start the server using `python server.py --port=8888`.
+> The model is written in Keras and when exported as a TensorFlow model seems to be incompatible with TensorFlow Serving. So we're using seldon-core to serve this model since seldon-core allows you to serve any arbitrary model. More details [here](https://github.com/kubeflow/examples/issues/11#issuecomment-371005885).
 
-> The model is written in Keras and when exported as a TensorFlow model seems to be incompatible with TensorFlow Serving. So we're using our own webserver to serve this model. More details [here](https://github.com/kubeflow/examples/issues/11#issuecomment-371005885).
+# Prerequisites
 
-## Sample request
+Ensure that you have the following files from the [training](training_the_model.md) step in your `notebooks` directory:
+
+* `seq2seq_model_tutorial.h5` - the keras model
+* `body_pp.dpkl` - the serialized body preprocessor
+* `title_pp.dpkl` - the serialized title preprocessor
+
+# Wrap the model into a seldon-core microservice
+
+cd into the notebooks directory and run the following docker command. This will create a build/ directory.
+
+```
+cd notebooks/
+docker run -v $(pwd):/my_model seldonio/core-python-wrapper:0.7 /my_model IssueSummarization 0.1 seldonio --base-image=python:3.6
+```
+
+The build/ directory contains all the necessary files to build the seldon-core microservice image
+
+```
+cd build/
+./build_image.sh
+```
+
+Now you should see an image named `seldonio/issuesummarization:0.1` in your docker images. To test the model, you can run it locally using
+
+`docker run -p 5000:5000 seldonio/issuesummarization:0.1`
+
+> You can find more details about wrapping a model with seldon-core [here](https://github.com/SeldonIO/seldon-core/blob/master/docs/wrappers/python.md)
+
+## Sample request and response
+
+Request
 
 ```
-curl -X POST -H 'Content-Type: application/json' -d '{"instances": ["issue overview add a new property to disable detection of image stream files those ended with -is.yml from target directory. expected behaviour by default cube should not process image stream files if user does not set it. current behaviour cube always try to execute -is.yml files which can cause some problems in most of cases, for example if you are using kuberentes instead of openshift or if you use together fabric8 maven plugin with cube"]}' http://localhost:8888/predict
+curl -X POST -d 'json={"data":{"ndarray":[["issue overview add a new property to disable detection of image stream files those ended with -is.yml from target directory. expected behaviour by default cube should not process image stream files if user does not set it. current behaviour cube always try to execute -is.yml files which can cause some problems in most of cases, for example if you are using kuberentes instead of openshift or if you use together fabric8 maven plugin with cube"]]}}' http://localhost:5000/predict
 ```
 
-## Sample response
+Response
 
 ```
-{"predictions": ["add a new property to disable detection"]}
+{
+  "data": {
+    "names": [
+      "t:0"
+    ],
+    "ndarray": [
+      [
+        "add a new property to disable detection"
+      ]
+    ]
+  }
+}
 ```
 
 Next: [Teardown](teardown.md)