SDK - Reduce python component limitations - no import errors for cust… #3106
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…om type annotations
By default, create_component_from_func copies the source code of the function and creates a component using that source code. No global imports are captured. This is problematic for the function definition, since any annotation, that uses a type that needs to be imported, will cause error. There were some special provisions for
NamedTuple, InputPath and OutputPath, but even they were brittle (for example, "typing.NamedTuple" or "components.InputPath" annotations still caused failures at runtime).
This commit fixes the issue by stripping the type annotations from function declarations.
Fixes cases that were failing before:
```python
import typing
import collections
MyFuncOutputs = typing.NamedTuple('Outputs', [('sum', int), ('product', int)])
@create_component_from_func
def my_func(
param1: CustomType, # This caused failure previously
param2: collections.OrderedDict, # This caused failure previously
) -> MyFuncOutputs: # This caused failure previously
pass
```
Code `print(line, end="")` was causing error: "lib2to3.pgen2.parse.ParseError: bad input: type=22, value='=', context=('', (2, 15))"
Ark-kun
force-pushed
the
SDK---Reduce-python-component-limitations---no-import-errors-for-custom-type-annotations
branch
from
54402d8
to
5b3e8a9
Compare
|
/retest |
Ark-kun
force-pushed
the
SDK---Reduce-python-component-limitations---no-import-errors-for-custom-type-annotations
branch
from
e01679d
to
c0879df
Compare
|
@numerology Can you please take a look? |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Ark-kun The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
numerology
reviewed
Feb 24, 2020
|
/retest |
Co-Authored-By: Jiaxiao Zheng <jxzheng@google.com>
|
/lgtm |
Jeffwan
pushed a commit
to Jeffwan/pipelines
that referenced
this pull request
Dec 9, 2020
kubeflow#3106) * SDK - Reduce python component limitations - no import errors for custom type annotations By default, create_component_from_func copies the source code of the function and creates a component using that source code. No global imports are captured. This is problematic for the function definition, since any annotation, that uses a type that needs to be imported, will cause error. There were some special provisions for NamedTuple, InputPath and OutputPath, but even they were brittle (for example, "typing.NamedTuple" or "components.InputPath" annotations still caused failures at runtime). This commit fixes the issue by stripping the type annotations from function declarations. Fixes cases that were failing before: ```python import typing import collections MyFuncOutputs = typing.NamedTuple('Outputs', [('sum', int), ('product', int)]) @create_component_from_func def my_func( param1: CustomType, # This caused failure previously param2: collections.OrderedDict, # This caused failure previously ) -> MyFuncOutputs: # This caused failure previously pass ``` * Fixed the compiler tests * Fixed crashes on print function Code `print(line, end="")` was causing error: "lib2to3.pgen2.parse.ParseError: bad input: type=22, value='=', context=('', (2, 15))" * Using the strip_hints library to strip the annotations * Updating test workflow yamls * Workaround for bug in untokenize * Switched to the new strip_string_to_string method * Fixed typo. Co-Authored-By: Jiaxiao Zheng <jxzheng@google.com> Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>
magdalenakuhn17
pushed a commit
to magdalenakuhn17/pipelines
that referenced
this pull request
Oct 22, 2023
* Fixed torchserve e2e test. Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com> * Upgraded torchserve-kfs image to 0.8.2 Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com> * Fixed torchserve transformer e2e. Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com> * Set PROTOCOL_VERSION env based on protocol version spec. Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com> --------- Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…om type annotations
By default, create_component_from_func copies the source code of the function and creates a component using that source code. No global imports are captured. This is problematic for the function definition, since any annotation, that uses a type that needs to be imported, will cause error. There were some special provisions for
NamedTuple, InputPath and OutputPath, but even they were brittle (for example, "typing.NamedTuple" or "components.InputPath" annotations still caused failures at runtime).
This commit fixes the issue by stripping the type annotations from function declarations.
Fixes cases that were failing before:
This change is