Add ProtoInfo constructor to Starlark

[cheister]

This is a follow up to #10966 to allow users to create a ProtoInfo provider in Starlark. The corresponding issue is #11060

It also allows the native proto_library to have deps and exports that are not proto_library rules.

[cheister]

cc: @Yannic

[cheister]

@lberki or @Yannic do you mind taking a look? Thanks.

[cheister]

@lberki are you still the right person to look at this PR? This feature would let us fix another problem with the IJ plugin not recognizing proto files because we have to make a custom ProtoInfo object for our Proto Rules because there is no way in StarLark to create a native ProtoInfo object.

[comius]

Have you considered implementing it as a default constructor? That is exposing ProtoInfo() call to Starlark, like it is done for JavaInfo, CcInfo.

Also rebase is needed, FakeApi has mostly been removed, so no changes are needed on FakeProtoCommon.

I haven't done a more thorough review, but the newly added signatures look ok.

[aiuto]

Friendly ping. This code has gone stale.
@cheister Can you address the concerns and update this or should we close it.

[cheister]

Thanks for looking at the change. The PR has been updated so it is now a ProtoInfo constructor instead of proto_common.createProtoInfo.

It probably still needs one or two more integration tests to make sure a ProtoInfo object built in Starlark will work along with native ProtoInfo objects.

[cheister]

I added an integration test that exposes a problem with the hidden :proto_compiler attribute on the native ProtoLibrary Rule. Apparently BazelJavaProtoAspect is expecting the :proto_compiler attribute to be defined on the rule that creates the ProtoInfo object but there is no way to create that attribute in Starlark because it starts with colon. I'm not sure if we should remove the :proto_compiler attribute or fix BazelJavaProtoAspect to fallback to a different protoc when the rule doesn't have the hidden :proto_compiler attribute.

[oquenchil]

Hello, thank you very much for this change. We are in the process of starlarkifying all the rules right now. We haven't reached proto rules yet but will do so in Q4. As part of that starlarkification process the ProtoInfo constructor will be in Starlark too. But for now in order not to interfere with that process we rather not expose any APIs publicly. Count on next year this being possible though.

Thanks!

[cheister]

@oquenchil The starlarkification process has been promised for 3 years now, is there any work actually started for the proto rules? The point of having a ProtoInfo constructor in Starlark is so you could transition from the native proto rules to the Starlark rules with the native ProtoInfo object. Is the plan to try to move everyone to a Starlark built ProtoInfo object all at once? That seems like that will break backwards compatibility.

[oquenchil]

Work on the proto rules should start during Q4 this year. Unlike previous Starlarkification promises we are actually moving forward this year and have already starlarkified a number of rules.

The plan is to have the current native rules in Starlark without any incompatible changes, the ProtoInfo constructor for the native provider will be in Starlark, then that will be replaced by a purely Starlark provider. We don't want to add public APIs before this happens precisely to avoid running into incompatible changes.

[cheister]

You say: the ProtoInfo constructor for the native provider will be in Starlark but that is exactly what this PR was trying to do? You need to add something to the Starlark API so you can create the native ProtoInfo provider in Starlark.

Unless you mean you're just going to move to a purely Starlark ProtoInfo provider all at once, which will break backwards compatibly with native rules.