[Sample] Add new TFX::OSS sample

[numerology]

Add a sample showing how to run TFX pipeline on Kubeflow Runner.

TODO:

• Add proper dependency so it can be built into preload samples
• Add docs
• Add sample test coverage

---

This change is 

[numerology]

Note that the compilation of new sample here requires a HEAD version of TFX (as of 10.07.2019) and a customized value in tfx.version pointing to TFX nightly build (because there is actually no image tagged as tfx:0.15.0.dev). I suggest that we move this sample to contrib sample dir to bypass integration test and make the compiled pipeline package part of our repo so that backend server can pre-load it, otherwise we might need rather complex dependency setup, which is also error-prone because it relies on a dev version of TFX.

I will add a doc so that a user knows how to compile such hacky sample successfully.

WDYT @neuromage @gaoning777

[neuromage]

WDYT @neuromage @gaoning777

SGTM, thanks!

[gaoning777]

I think it is a little risky to load the sample in the API server without testing. The preloaded samples are supposed to be flagship samples that are guaranteed to work. Say, a new user tries out the KFP and runs the preloaded sample but failed. It could easily scare them away. Besides, we got lots of issues in the early stages when the preloaded samples break. @SinaChavoshi used to use the old TFX samples for demos and IIRC, sample breaking during the demo is bad.

My suggestion is a little change on top of your current work: write some custom sample test infra codes to cover the test for the compiled pipeline. It shouldn't be too much custom codes but is rewarding. WDYT? @numerology @neuromage

[numerology]

I think it is a little risky to load the sample in the API server without testing. The preloaded samples are supposed to be flagship samples that are guaranteed to work. Say, a new user tries out the KFP and runs the preloaded sample but failed. It could easily scare them away. Besides, we got lots of issues in the early stages when the preloaded samples break. @SinaChavoshi used to use the old TFX samples for demos and IIRC, sample breaking during the demo is bad.

My suggestion is a little change on top of your current work: write some custom sample test infra codes to cover the test for the compiled pipeline. It shouldn't be too much custom codes but is rewarding. WDYT? @numerology @neuromage

One of the current problem is that we need a dev version and a nightly build of TFX to make this compile and run. If we put those dependencies in our sample test infra it might not be very stable. Talked to TFX side and I was told 0.15.0 release should be either this week or next week. I think it will be much easier and reliable to add a sample test coverage by then.

[gaoning777]

what I meant is not adding dependency on the dev version of the TFX, but adding custom sample test infra codes to run the chicago_taxi_pipeline_simple.tar.gz directly without compilation. We add the python codes and the compiled one as you have done here.

[numerology]

what I meant is not adding dependency on the dev version of the TFX, but adding custom sample test infra codes to run the chicago_taxi_pipeline_simple.tar.gz directly without compilation. We add the python codes and the compiled one as you have done here.

I see. That's doable. Let me ping you when it's done.

[numerology]

PTAL @neuromage @gaoning777 Thanks!

[neuromage]

This is pretty awesome, thanks @numerology :-)
/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neuromage

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• backend/OWNERS [neuromage]
• samples/OWNERS [neuromage]
• test/OWNERS [neuromage]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[gaoning777]

This is great.
Thanks

[animeshsingh]

@gaoning777 @numerology @neuromage we have a talk in KF summit and TF World on this topic - are there any additional materials (slides/design docs etc.) you can point to vis a vis TFX which I can use?

[numerology]

@gaoning777 @numerology @neuromage we have a talk in KF summit and TF World on this topic - are there any additional materials (slides/design docs etc.) you can point to vis a vis TFX which I can use?

The recommended entrypoint of a variety of docs besides the code repo is the user guide where some basic concepts and orchestration mechanism of TFX are introduced. Also there is some recorded talks and tutorials here.