Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

component build support for both python2 and python3 #730

Merged
merged 11 commits into from
Feb 25, 2019
Merged

component build support for both python2 and python3 #730

merged 11 commits into from
Feb 25, 2019

Conversation

gaoning777
Copy link
Contributor

@gaoning777 gaoning777 commented Jan 23, 2019 •
edited by jlewi
Loading

This change is Reviewable

@gaoning777 gaoning777 mentioned this pull request Jan 23, 2019
Closed
@gaoning777
Copy link
Contributor Author

/test kubeflow-pipeline-sample-test

hongye-sun
hongye-sun reviewed Jan 24, 2019
View reviewed changes
Copy link
Contributor

@hongye-sun hongye-sun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to have a sample for using the python version here and have sample test to cover both py2 and py3. Otherwise, lgtm.

@gaoning777
Copy link
Contributor Author

/test kubeflow-pipeline-sample-test

@gaoning777
Copy link
Contributor Author

gaoning777 commented Jan 24, 2019 •
edited
Loading

I was thinking to add a sample test in another PR. How about I manually testing the functionality and then later send another PR to add a sample test? because the tfx notebook sample that demonstrates component build is already long enough.

@Ark-kun
Copy link
Contributor

Ark-kun commented Jan 24, 2019

Let's have a test that ensures generated code runs on both Python3 and Python2.

@Ark-kun
Copy link
Contributor

Ark-kun commented Jan 24, 2019

JFYI, Python supports enums:

import enum

class PythonVersion(enum.Enum):
  py2 = 'python2'
  py3 = 'python3'

def build(python_version: PythonVersion):
  if python_version is PythonVersion.py2:
     pass
  elif python_version is PythonVersion.py3:
     pass
  else:
    raise TypeError()

@gaoning777
Copy link
Contributor Author

Added the sample test

@Ark-kun
Copy link
Contributor

Ark-kun commented Jan 25, 2019 •
edited
Loading

So, did you check that the code generated by the python2 branch works? You can test it in a notebook.
I have suspicion that is does not work since I tested a similar change before.

@gaoning777
Copy link
Contributor Author

/retest

@gaoning777
Copy link
Contributor Author

/test kubeflow-pipeline-e2e-test

@gaoning777
Copy link
Contributor Author

/retest

@gaoning777
Copy link
Contributor Author

/test kubeflow-pipeline-sample-test

@gaoning777
Copy link
Contributor Author

/test kubeflow-pipeline-e2e-test

hongye-sun
hongye-sun reviewed Feb 23, 2019
View reviewed changes
Copy link
Contributor

@hongye-sun hongye-sun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Not major issues. Feel free to commit.

sdk/python/kfp/compiler/_component_builder.py Show resolved Hide resolved
sdk/python/kfp/compiler/_component_builder.py Outdated Show resolved Hide resolved
samples/notebooks/KubeFlow Pipeline Using TFX OSS Components.ipynb Show resolved Hide resolved
@gaoning777
remove the pathlib codes in python3 component build, but use python2 …
8aef378 
…code instead; add a todo to create a new sample
@hongye-sun
Copy link
Contributor

/lgtm

@gaoning777
Copy link
Contributor Author

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gaoning777

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

1 similar comment
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gaoning777

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit a6763b9 into kubeflow:master Feb 25, 2019
@gaoning777 gaoning777 deleted the configurable-python-version-for-componentbuild branch February 25, 2019 21:22
@amygdala amygdala mentioned this pull request Mar 5, 2019
Closed
cheyang pushed a commit to alibaba/pipelines that referenced this pull request Mar 28, 2019
@gaoning777 @k8s-ci-robot
component build support for both python2 and python3 (kubeflow#730)
c4e7271 
* component build support for both python2 and python3

* add sample test

* remove the annotations for python2 component build

* add pathlib for python2 component build

* fix component build unit test

* fix bug in the dockerfile generator

* remove exist_ok in path.mkdir to make python2 compatible

* adjust unit test

* remove pathlib dependency for python2 component build

* remove the pathlib codes in python3 component build, but use python2 code instead; add a todo to create a new sample
Linchin pushed a commit to Linchin/pipelines that referenced this pull request Apr 11, 2023
@jlewi
notebook tests need to use a unique subdirectory (kubeflow#730)
69bab53 
* notebook tests need to use a unique subdirectory

* When the notebook runs on a kubeflow-ci-deployment cluster it will
  upload the rendered notebook to a gcs bucket in kubeflow-ci-deployment.

* We need the GCS path for the notebook to be unique so that results
  from different runs don't overwrite each other.

Related to: kubeflow#715

* Fix subdir.

* Update notebook image.
HumairAK pushed a commit to red-hat-data-services/data-science-pipelines that referenced this pull request Mar 11, 2024
@yhwang
[test] update sample for katib (kubeflow#730)
1e526f9 
Update pip packages and remove the usage of
`dsl.ContainerOp`.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Ark-kun Ark-kun Ark-kun left review comments

@hongye-sun hongye-sun hongye-sun left review comments

@qimingj qimingj Awaiting requested review from qimingj

Assignees

@hongye-sun hongye-sun

Labels
approved lgtm size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@gaoning777 @Ark-kun @hongye-sun @k8s-ci-robot