Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add type checking sample to sample tests #1129

Merged
merged 3 commits into from
Apr 12, 2019
Merged

add type checking sample to sample tests #1129

merged 3 commits into from
Apr 12, 2019

Conversation

gaoning777
Copy link
Contributor

@gaoning777 gaoning777 commented Apr 10, 2019

This change is Reviewable

…check_notebook_result script to not validate the pipeline runs when experiment arg is not provided
@gaoning777
Copy link
Contributor Author

gaoning777 commented Apr 11, 2019

resolving #974

@Ark-kun
Copy link
Contributor

Ark-kun commented Apr 11, 2019

AFAIK all current type checking code can be tested by unit-tests that do not require cluster.
If you want to test a sample notebook, it's easy to programmatically convert it to python and execute.

@gaoning777
Copy link
Contributor Author

Correct. It does not require the cluster. By having the sample test results shown in the same place, it is easier to check the result.

@gaoning777
Copy link
Contributor Author

And, the sample notebook is converted to python and get executed.

@Ark-kun
Copy link
Contributor

Ark-kun commented Apr 11, 2019

Don't you get the Travis test results much sooner? They're also less flaky.

I just want us to test as much as possible using local unit tests so that the contributors can easily run them after local changes.

@gaoning777
Copy link
Contributor Author

We can always add unit tests for the static type checking but it does not mean we can drop the sample tests here. The sample test here guarantees all samples work correctly.

@Ark-kun
Copy link
Contributor

Ark-kun commented Apr 12, 2019

it does not mean we can drop the sample tests here. The sample test here guarantees all samples work correctly.

Of course. I did not mean to drop them. We should test the samples. I'm just asking to run the test locally, not on cluster (using Travis, for example).

@Ark-kun
Copy link
Contributor

Ark-kun commented Apr 12, 2019

OK, let's LGTM this one.
But please think about introducing Travis-based sample tests for samples that can be executed locally.

/lgtm

@gaoning777
Copy link
Contributor Author

I do not see the benefits of introducing the Travis-based sample tests into the sample test infra now.

  1. Adding the travis based sample test infra takes some efforts while we only have this one sample that can be run locally.
  2. If adding the travis based sample test infra reduces the testing time, running this test in the cluster does not cost more time.
  3. If adding the travis based sample test infra is to make it easy to run the sample test, we can always add unit tests for the developers to quickly validate locally, which I think we do need to do.

@gaoning777
Copy link
Contributor Author

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gaoning777

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

1 similar comment
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gaoning777

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Ark-kun
Copy link
Contributor

Ark-kun commented Apr 12, 2019

3. If adding the travis based sample test infra is to make it easy to run the sample test, we can always add unit tests for the developers to quickly validate locally, which I think we do need to do.

That's what I meant.

@k8s-ci-robot k8s-ci-robot merged commit 06e544b into kubeflow:master Apr 12, 2019
@gaoning777
Copy link
Contributor Author

Got it. Cannot agree more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants