Should Kubeflow publish and maintain TF Serving Docker Images?

[jlewi]

Opening this issue to consider whether Kubeflow should publish and curate TF Serving Docker images.

I think there is a reasonable expectation in the community that users shouldn't have to build Docker images just to serve models: see

• Docker image with prebuilt tensorflow_model_server, ready to run. tensorflow/serving#513
• Tensorflow (TF) Serving on Multi-GPU box  tensorflow/serving#311 (this comment

A number of folks have already publicly shared Docker images (see tensorflow/serving#513), Kubeflow has also made a Docker image publicly available based on this Dockerfile

Kubeflow's ksonnet component for serving depends on having a publicly available Docker image to prevent customers from having to build their own image.

I think what's missing is a committement to maintaining and supporting these images. I think kubeflow should try to figure out what that would mean and whether we want to take that on.

Here are some questions to consider

• Which versions of TensorFlow serving would we support?
• What validation/verification of images we perform?
• Would we provide GPU images?
• Are there other artifacts we should release? e.g. the python client libraries for TF serving?

Initial proposal

• We provide a Docker image for each TensorFlow Serving minor release
  • Starting with the current 1.4.0 release
• We only provide an image for CPU

/cc @bhack @chrisolston @rhaertel80 @lluunn@nfiedel @bhupchan@ddysher

[bhack]

Finally! /cc @nfiedel @bhupchan @ddysher

[sub-mod]

I hope we support centos images as well.

As a sidenote wrt to tf-serving in openshift there exists

• tensorflow-serving-s2i
• tensorflow-serving-gpu-s2i
  The way we get the latest bits of model_server is via
  tensorflow-build-s2i
  I don't know how but if we can reuse some ideas here that would nice.

[bhack]

If we want to start with CPU image what flags want we activate? Cause generally AVX etc.. are disabled.

[flx42]

What's the issue with providing a GPU image?
(at the risk of sounding like a broken record, it would be nice to have the same conversation for an "official" notebook image).

[jlewi]

@flx42 Reason for trying to limit to CPU is just trying to scope work given staffing is very scarce and CPU is probably the more common case. If you or someone else wants to take the lead on providing a GPU image for TFServing; you'd have my full support.

I think there's some basic infrastructure we need to get into place for CI/CD of images. Once we have that in place hopefully we can add images to cover more use cases.

I'll open an issue to discuss official notebook images.

[jlewi]

So we've decided that we need to publish and maintain Docker images to have a good experience. At least until TFServing takes this on.

Lets use the uber tracking bug #64. To track actual work.