Java wrappers

docs/wrappers/java.md

@@ -0,0 +1,216 @@
+# Packaging a Java model for Seldon Core using s2i
+
+
+In this guide, we illustrate the steps needed to wrap your own Java model in a docker image ready for deployment with Seldon Core using [source-to-image app s2i](https://github.com/openshift/source-to-image).
+
+If you are not familar with s2i you can read [general instructions on using s2i](./s2i.md) and then follow the steps below.
+
+
+# Step 1 - Install s2i
+
+ [Download and install s2i](https://github.com/openshift/source-to-image#installation)
+
+ * Prequisites for using s2i are:
+   * Docker
+   * Git (if building from a remote git repo)
+
+To check everything is working you can run
+
+```bash
+s2i usage seldonio/seldon-core-s2i-java-build
+```
+
+# Step 2 - Create your source code
+
+To use our s2i builder image to package your Java model you will need:
+
+ * A Maven project that depends on ```io.seldon.wrapper``` library
+ * A Spring Boot configuration class
+ * A class that implements ```io.seldon.wrapper.SeldonPredictionService``` for the type of component you are creating
+ * .s2i/environment - model definitions used by the s2i builder to correctly wrap your model
+
+We will go into detail for each of these steps:
+
+## Maven Project
+Create a Spring Boot Maven project and include the dependency:
+
+```XML
+<dependency>
+	<groupId>io.seldon.wrapper</groupId>
+	<artifactId>seldon-core-wrapper</artifactId>
+	<version>0.1.0</version>
+</dependency>
+```
+
+A full example can be found at ```wrappers/s2i/java/test/model-template-app/pom.xml```.
+
+## Spring Boot Intialization
+
+Create a main App class:
+  * Add @EnableAsync annotation (to allow the embedded gRPC server to start at Spring Boot startup)
+  * include the ```io.seldon.wrapper``` in the scan base packages list along with your App's package, in the example below the Apps's package is ```io.seldon.example```.
+  * Import the config class at ```io.seldon.wrapper.config.AppConfig.class```
+
+For example:
+
+```java
+@EnableAsync
+@SpringBootApplication(scanBasePackages = {"io.seldon.wrapper","io.seldon.example"})
+@Import({ io.seldon.wrapper.config.AppConfig.class })
+public class App {
+    public static void main(String[] args) throws Exception {
+            SpringApplication.run(App.class, args);
+	    }
+}
+```
+
+## Prediction Class
+To handle requests to your model or other component you need to implement one or more of the methods in ```io.seldon.wrapper.SeldonPredictionService```, in particular:
+
+```java
+default public SeldonMessage predict(SeldonMessage request);
+default public SeldonMessage route(SeldonMessage request);
+default public SeldonMessage sendFeedback(Feedback request);
+default public SeldonMessage transformInput(SeldonMessage request);
+default public SeldonMessage transformOutput(SeldonMessage request);
+default public SeldonMessage aggregate(SeldonMessageList request);
+```
+
+Your implementing class should be created as a Spring Component so it will be managed by Spring. There is a full H2O example in ```examples/models/h2o_mojo/src/main/java/io/seldon/example/h2o/model```, whose implmentation is show below:
+
+```java
+@Component
+public class H2OModelHandler implements SeldonPredictionService {
+	private static Logger logger = LoggerFactory.getLogger(H2OModelHandler.class.getName());
+	EasyPredictModelWrapper model;
+
+	public H2OModelHandler() throws IOException {
+		MojoReaderBackend reader =
+                MojoReaderBackendFactory.createReaderBackend(
+                  getClass().getClassLoader().getResourceAsStream(
+                     "model.zip"), 
+                      MojoReaderBackendFactory.CachingStrategy.MEMORY);
+		MojoModel modelMojo = ModelMojoReader.readFrom(reader);
+		model = new EasyPredictModelWrapper(modelMojo);
+		logger.info("Loaded model");
+	}
+
+	@Override
+	public SeldonMessage predict(SeldonMessage payload) {
+		List<RowData> rows = H2OUtils.convertSeldonMessage(payload.getData());
+		List<AbstractPrediction> predictions = new ArrayList<>();
+		for(RowData row : rows)
+		{
+			try
+			{
+				BinomialModelPrediction p = model.predictBinomial(row);
+				predictions.add(p);
+			} catch (PredictException e) {
+				logger.info("Error in prediction ",e);
+			}
+		}
+        DefaultData res = H2OUtils.convertH2OPrediction(predictions, payload.getData());
+
+		return SeldonMessage.newBuilder().setData(res).build();
+	}
+
+}
+
+```
+
+The above code:
+
+  * loads a model from the local resources folder on startup
+  * Converts the proto buffer message into H2O RowData using provided utility classes.
+  * Runs a BionomialModel prediction and converts the result back into a ```SeldonMessage``` for return
+
+### H2O Helper Classes
+
+We provide H2O utility class ```io.seldon.wrapper.utils.H2OUtils``` in seldon-core-wrapper to convert to and from the seldon-core proto buffer message types.
+
+### DL4J Helper Classes
+
+We provide a DL4J utility class ```io.seldon.wrapper.utils.DL4JUtils``` in seldon-core-wrapper to convert to and from the seldon-core proto buffer message types.
+
+## .s2i/environment
+
+Define the core parameters needed by our R builder image to wrap your model. An example is:
+
+```bash
+API_TYPE=REST
+SERVICE_TYPE=MODEL
+```
+
+These values can also be provided or overriden on the command line when building the image.
+
+# Step 3 - Build your image
+Use ```s2i build``` to create your Docker image from source code. You will need Docker installed on the machine and optionally git if your source code is in a public git repo. 
+
+Using s2i you can build directly from a git repo or from a local source folder. See the [s2i docs](https://github.com/openshift/source-to-image/blob/master/docs/cli.md#s2i-build) for further details. The general format is:
+
+```bash
+s2i build <git-repo> seldonio/seldon-core-s2i-java-build <my-image-name> --runtime-image seldonio/seldon-core-s2i-java-runtime
+s2i build <src-folder> seldonio/seldon-core-s2i-java-build <my-image-name> --runtime-image seldonio/seldon-core-s2i-java-runtime 
+```
+
+An example invocation using the test template model inside seldon-core:
+
+```bash
+s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i/python/test/model-template-app seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime
+```
+
+The above s2i build invocation:
+
+ * uses the GitHub repo: https://github.com/seldonio/seldon-core.git and the directory ```wrappers/s2i/R/test/model-template-app``` inside that repo.
+ * uses the builder image ```seldonio/seldon-core-s2i-java-build```
+ * uses the runtime image ```seldonio/seldon-core-s2i-java-runtime```
+ * creates a docker image ```seldon-core-template-model```
+
+
+For building from a local source folder, an example where we clone the seldon-core repo:
+
+```bash
+git clone https://github.com/seldonio/seldon-core.git
+cd seldon-core
+s2i build wrappers/s2i/R/test/model-template-app seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime
+```
+
+For more help see:
+
+```
+s2i usage seldonio/seldon-core-s2i-java-build
+s2i build --help
+```
+
+# Reference
+
+## Environment Variables
+The required environment variables understood by the builder image are explained below. You can provide them in the ```.s2i/enviroment``` file or on the ```s2i build``` command line.
+
+
+### API_TYPE
+
+API type to create. Can be REST or GRPC.
+
+### SERVICE_TYPE
+
+The service type being created. Available options are:
+
+ * MODEL
+ * ROUTER
+ * TRANSFORMER
+ * COMBINER
+
+
+## Creating different service types
+
+### MODEL
+
+ * [A minimal skeleton for model source code](https://github.com/cliveseldon/seldon-core/tree/s2i/wrappers/s2i/java/test/model-template-app)
+ * [Example H2O MOJO](https://github.com/SeldonIO/seldon-core/tree/master/examples/models/h2o-mojo/README.md)
+
+
+
+
+
+

docs/wrappers/readme.md

@@ -21,7 +21,16 @@ You can use either:
 
    * [R models can be wrapped using source-to-image](r.md)
 
+## Java
+
+Java based models including, [H2O](https://www.h2o.ai/), [Deep Learning 4J](https://deeplearning4j.org/), Spark (standalone exported models).
+
+  * [Java models wrapped using source-to-image](java.md)
+
+
 ## H2O
+H2O models can be wrapped either from Java or Python.
 
-   * [H2O models](./h2o.md)
+  * [Java models wrapped using source-to-image](java.md)
+  * [H2O models saved and called from python](./h2o.md)
 

examples/models/h2o_mojo/.gitignore

@@ -0,0 +1 @@
+/target/

examples/models/h2o_mojo/.s2i/environment

@@ -0,0 +1,2 @@
+API_TYPE=REST
+SERVICE_TYPE=MODEL

examples/models/h2o_mojo/Makefile

@@ -0,0 +1,13 @@
+
+
+build_jar: 
+	mvn clean package 
+
+clean:
+	mvn clean
+	rm -rf experiment
+	rm -f src/main/resources/model.zip
+train:
+	mkdir -p experiment
+	python train.py
+	mv experiment/*.zip src/main/resources/model.zip

examples/models/h2o_mojo/README.md

@@ -0,0 +1,83 @@
+# H2O MOJO Model
+An example H2O model
+
+## Dependencies
+
+ * [H2O python library](http://docs.h2o.ai/h2o/latest-stable/h2o-docs/downloading.html#install-in-python)
+
+## Train locally
+
+```bash
+make train
+```
+
+## Wrap using [s2i](https://github.com/openshift/source-to-image#installation).
+
+```bash
+s2i build . seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime
+```
+
+## Local Docker Smoke Test
+
+Run under docker.
+
+```bash
+docker run --rm -p 5000:5000 h2o-test:0.1
+```
+
+Ensure test grpc modules compiled.
+
+```bash
+pushd ../../../wrappers/testing ; make build_protos ; popd
+```
+
+Send a data request using the wrapper tester.
+
+```bash
+python ../../../wrappers/testing/tester.py contract.json 0.0.0.0 5000 -p
+```
+
+## Minikube test
+
+```bash
+minikube start --memory 4096
+```
+
+[Install seldon core](/readme.md#install)
+
+Connect to Minikube Docker daemon
+
+```bash
+eval $(minikube docker-env)
+```
+
+Build image using minikube docker daemon.
+
+```bash
+s2i build . seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime
+```
+
+Launch deployment
+
+```bash
+kubectl create -f h2o_deployment.json
+```
+
+Port forward API server
+
+```bash
+kubectl port-forward $(kubectl get pods -n seldon -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n seldon 8080:8080
+```
+
+Ensure tester gRPC modules compiled. 
+
+```bash
+pushd ../../../util/api_tester ; make build_protos ; popd
+```
+
+Send test request
+```bash
+python ../../../util/api_tester/api-tester.py contract.json 0.0.0.0 8080 --oauth-key oauth-key --oauth-secret oauth-secret -p
+```
+
+

examples/models/h2o_mojo/contract.json

@@ -0,0 +1,42 @@
+{
+    "features":[
+	{
+	    "name":"AGE",
+	    "dtype":"FLOAT",
+	    "ftype":"continuous",
+	    "range":[20,80]
+	},
+	{
+	    "name":"RACE",
+	    "ftype":"categorical",
+	    "values":[0,1,2]
+	},
+	{
+	    "name":"DCAPS",
+	    "ftype":"categorical",
+	    "values":[0,1,2]
+	},
+	{
+	    "name":"VOL",
+	    "ftype":"categorical",
+	    "values":[0,1,2]
+	},
+	{
+	    "name":"GLEASON",
+	    "dtype":"FLOAT",
+	    "ftype":"continuous",
+	    "range":[0,10]
+	}
+    ],
+    "targets":[
+	{
+	    "name":"class",
+	    "dtype":"FLOAT",
+	    "ftype":"continuous",
+	    "range":[0,1],
+	    "repeat":2
+	}
+    ]
+}
+
+