Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict Starlark to only interact with Starlark types #7281

Closed
c-parsons opened this issue Jan 29, 2019 · 5 comments
Closed

Restrict Starlark to only interact with Starlark types #7281

c-parsons opened this issue Jan 29, 2019 · 5 comments
Assignees
Labels
P2 We'll consider working on this in future. (Assignee optional) type: process

Comments

@c-parsons
Copy link
Contributor

Over the past year or so a number of bugs have arisen from Starlark functions/methods/struct fields returning natives types that are not proper Starlark types.
One concrete example: specifying a license attr (using attr.license()) on a rule and then, from the rule context, calling ctx.attr.license, gets you an object which is not exposed to Starlark.
Trying to print this object prints <unknown object com.google.devtools.build.lib.packages.License>

Starlark needs to crack down on this -- we should throw a catastrophic exception whenever any expression evaluates to a non-Starlark type. Fixing these adhoc only gets us so far, as this sort of bug keeps reappearing.

For motivation as a P1, this particular license bug required me to rollback some progress on #5010. @SkylarkCallable-annotated methods do have return-type validation, and verify that methods return appropriate Starlark types. But struct() does not -- so the following (simplified) user code resulted in a catastrophic error when we migrated getattr to appropriate validation:

mystruct = struct(license = ctx.attr.license)
obj = getattr(mystruct, "license", None):

as getattr would try to return an invalid type.

For license itself, we will need to make a call on whether ctx.attr.license should return nothing or an opaque-but-valid Starlark object.

@c-parsons c-parsons added P1 I'll work on this now. (Assignee required) team-Starlark labels Jan 29, 2019
bazel-io pushed a commit that referenced this issue Feb 5, 2019
Prior to this change, attr.license() is already deprecated and subject to removal via --incompatible_no_attr_license. However, until this flag is fully flipped, the value of license attributes needs to be well defined. This change makes it so while providing nothing more than an opaque type.

Progress toward #7281.

RELNOTES: None.
PiperOrigin-RevId: 232535266
@laurentlb
Copy link
Contributor

Do you know how much work it is?

@c-parsons
Copy link
Contributor Author

The work is migrating all remaining uses of @SkylarkSignature to @SkylarkCallable, which should be a very small (< 1 week) amount of implementation work.
However, there may be some migration, which I find hard to estimate duration of.

Next step here is to deal with the implicit rule attribute "distrib", which has no Starlark type, but is assignable in Starlark.

bazel-io pushed a commit that referenced this issue Apr 12, 2019
These types are not Starlark-compatible, and, due to an existing bug, these are propagated anyway for native rule targets which include them as implicit attributes. Without this change, one may assign to these types in Starlark but later get a catastrophic error when an object of these types is passed to another function, such as dir().

Progress toward #7281.

RELNOTES: None.
PiperOrigin-RevId: 243336671
@c-parsons
Copy link
Contributor Author

Update: The work remaining here is to remove SkylarkSignature from PackageFactory, which is a tricky refactoring. To kill off this SkylarkSignature annotation from Bazel entirely (which would be a great refactoring win), we also need to swap out SkylarkSignature usages in Runtime for another annotation which is no-op except for providing documentation for docgen. (Useful for documenting top-level primitives such as None, True, and False.

@katre
Copy link
Member

katre commented May 13, 2020

Downgrading from P1 and sending to be re-triaged.

@katre katre added untriaged and removed P1 I'll work on this now. (Assignee required) labels May 13, 2020
@laurentlb laurentlb added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels May 13, 2020
@brandjon
Copy link
Member

I believe this work is mostly done in the interpreter refactoring. Starlark.fromJava() implements the conversion and validation at Java-to-Starlark boundaries, and is called at various places in the interpreter. It may be possible to sneak a bad value through, but more invasive checking (e.g. scanning list elements) likely comes with a performance penalty.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) type: process
Projects
None yet
Development

No branches or pull requests

4 participants