Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python Support #238

Merged
merged 15 commits into from
Aug 2, 2019
Merged

Python Support #238

merged 15 commits into from
Aug 2, 2019

Conversation

kothariaditya
Copy link
Contributor

@kothariaditya kothariaditya commented Jul 11, 2019

Approach discussed here: #237

Rules supported for now:

  • String
  • Bool
  • Numbers
  • Messages
  • Wrapper
  • Duration
  • Timestamp
  • Enums
  • Any

Rules to be supported in next iteration:

  • Maps
  • Repeated
  • Oneofs
  • Bytes

@kothariaditya kothariaditya force-pushed the python-new-interface branch 4 times, most recently from 041ae5f to 55e012d Compare July 11, 2019 23:33
Makefile Show resolved Hide resolved
tests/harness/python/BUILD Outdated Show resolved Hide resolved
@dkunitsk
Copy link
Contributor

High level feedback: let's transition the string building to a templating engine, maybe Jinja

kothariaditya and others added 5 commits July 23, 2019 11:37
* add idea to gitignore

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* add required as a field rule to ensure nullability

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* unimplemented flag for c++

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* add import to build bazel

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* ignore java

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* fix syntax

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* fix error

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* fix java exception

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* fix fail statement

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* Pull message rules out

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* fix tests

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* undo style change

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* removing messagerules

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* use messagerules inside rulecontext

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* remove generated files

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* prevent use of non-scalar wkts with message rules

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* Unnecessary change

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* add float rules

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* C++ support

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* java support

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* Update docs

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* Changes updated

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* fixing javabuild

Signed-off-by: Aditya Kothari <akothari@lyft.com>

* Fix file permission

Signed-off-by: Aditya Kothari <akothari@lyft.com>
Signed-off-by: Aditya Kothari <akothari@lyft.com>
Signed-off-by: Aditya Kothari <akothari@lyft.com>
Signed-off-by: Aditya Kothari <akothari@lyft.com>
Signed-off-by: Aditya Kothari <akothari@lyft.com>
Signed-off-by: Aditya Kothari <akothari@lyft.com>
@kothariaditya kothariaditya force-pushed the python-new-interface branch 3 times, most recently from 9abea0b to f933c7f Compare July 23, 2019 19:24
Signed-off-by: Aditya Kothari <akothari@lyft.com>
Signed-off-by: Aditya Kothari <akothari@lyft.com>
Copy link
Contributor

@dkunitsk dkunitsk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High level question: is it possible to somehow group the various templates into classes?? So it's clear how they fit together.

validate/validator.py Show resolved Hide resolved
validate/validator.py Outdated Show resolved Hide resolved
@dkunitsk
Copy link
Contributor

Also, one thing that could be helpful. For each tests, could you output a string of the validate function and then post the output here for us.

Signed-off-by: Aditya Kothari <akothari@lyft.com>
@kothariaditya kothariaditya force-pushed the python-new-interface branch 6 times, most recently from 22f78e8 to 15228bf Compare July 24, 2019 21:14
@kothariaditya kothariaditya changed the title [WIP] [DO NOT REVIEW] Python New Approach Python Support Jul 24, 2019
@rodaine
Copy link
Member

rodaine commented Jul 26, 2019

Can you confirm for me that this works for 2.x as well as 3.x? Do we have a test covering both versions?

rule_comparison.md Show resolved Hide resolved
tests/harness/cases/messages.proto Outdated Show resolved Hide resolved
tests/harness/cases/messages.proto Outdated Show resolved Hide resolved
tests/harness/python/BUILD Outdated Show resolved Hide resolved
tests/harness/python/harness.py Outdated Show resolved Hide resolved
class UnimplementedException(Exception):
pass

class ValidationFailed(Exception):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look at the exceptions from the other languages and include the other ancillary information that will help developers track down the source of their errors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have included the ancillary info while raising the exception. For example here:
https://github.com/envoyproxy/protoc-gen-validate/pull/238/files#diff-2c27ae30e4a405617a8988599f7e85f9R88

raise ValidationFailed(\"{{ name }} not equal to {{ o.string['const'] }}\")

This can then be caught and read as follows:

except ValidationFailed as e:
    reason = str(e)

Please let me know if you think I missed anything here.

@kothariaditya kothariaditya force-pushed the python-new-interface branch 8 times, most recently from f0bf8cc to b4fba4a Compare July 30, 2019 01:26
Signed-off-by: Aditya Kothari <akothari@lyft.com>
@kothariaditya kothariaditya force-pushed the python-new-interface branch 3 times, most recently from 0456d77 to 060370f Compare July 31, 2019 18:22
Signed-off-by: Aditya Kothari <akothari@lyft.com>
ComplexTestEnum enum_const = 11 [(validate.rules).enum.const = 2];
}

message KitchenSinkMessage { ComplexTestMsg val = 1; }
Copy link
Contributor Author

@kothariaditya kothariaditya Jul 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated code for KitchenSinkMessage would be:

# Validates KitchenSinkMessage
def validate(p):

    if _has_field(p, "val"):
        embedded = generate_validate(p.val)(p.val)
        if embedded is not None:
            return embedded
    return None

# Validates ComplexTestMsg
def validate(p):

    if p.const != "abcd":
        raise ValidationFailed("const not equal to abcd")

    if _has_field(p, "nested"):
        embedded = generate_validate(p.nested)(p.nested)
        if embedded is not None:
            return embedded
    if p.int_const != 5:
        raise ValidationFailed("int_const not equal to 5")

    if p.bool_const != False:
        raise ValidationFailed("bool_const not equal to False")

    if p.HasField("float_val"):
            if p.float_val.value <= 0.0:
                raise ValidationFailed("float_val.value is not greater than 0.0")

    if not _has_field(p, "dur_val"):
        raise ValidationFailed("dur_val is required.")
    if _has_field(p, "dur_val"):
        dur = p.dur_val.seconds + round((10**-9 * p.dur_val.nanos), 9)
        lt = 17.0
        if dur >= lt:
            raise ValidationFailed("dur_val is not lesser than 17.0")

    if _has_field(p, "ts_val"):
        ts = p.ts_val.seconds + round((10**-9 * p.ts_val.nanos), 9)
        gt = 7.0

        if ts <= gt:
            raise ValidationFailed("ts_val is not greater than 7.0")

    if _has_field(p, "another"):
        embedded = generate_validate(p.another)(p.another)
        if embedded is not None:
            return embedded

    if p.float_const >= 8.0:
        raise ValidationFailed("float_const is not lesser than 8.0")

    if p.double_in not in [456.789, 123.0]:
        raise ValidationFailed("double_in not in [456.789, 123.0]")

    if p.enum_const != 2:
        raise ValidationFailed("enum_const not equal to 2")


    if _has_field(p, "any_val"):
        if p.any_val.type_url not in ['type.googleapis.com/google.protobuf.Duration']:
            raise ValidationFailed("any_val not in ['type.googleapis.com/google.protobuf.Duration']")

    return None

requirement("MarkupSafe"),],
main = "harness.py",
visibility = ["//tests/harness:__subpackages__"],
# Python Version is set by default to PY2AND3.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rodaine confirming this works for Python 2.x and 3.x. The version is set by default to PY2AND3 as per https://docs.bazel.build/versions/master/be/python.html.

Signed-off-by: Aditya Kothari <akothari@lyft.com>
Copy link
Member

@rodaine rodaine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! And I think we can continue to iterate on it to make it better.

@rodaine rodaine merged commit e736f5d into bufbuild:master Aug 2, 2019
@kothariaditya
Copy link
Contributor Author

Awesome, thanks a bunch!

@rodaine rodaine mentioned this pull request Aug 16, 2019
@buzzdan buzzdan mentioned this pull request Oct 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants