Replace transitive_*_jars in java_common.create_provider with deps #3769

iirina · 2017-09-20T08:02:07Z

The current transitive_*_jars arguments in java_common.create_provider are hard to understand and present some shortcomings. For example, you cannot create a provider with transitive source jars or other transitive information that JavaInfo encapsulates (and might not even be exposed).

The current (old) API:

java_common.create_provider(
  actions,
  use_ijar = ...,
  compile_jars = [...],
  runtime_jars = [...],
  source_jars = [...],
  transitive_compile_time_jars = [...], 
  transitive_runtime_jars = [...],
  java_toolchain = ....
)

should be replaced by this:

java_common.create_provider(
  actions,
  use_ijar = ...,
  compile_jars = [...],
  runtime_jars = [...],
  source_jars = [...],
  deps = [...], # this should be a list/set of JavaInfo, to encapsulate the transitive information
  java_toolchain = ....
)

The text was updated successfully, but these errors were encountered:

iirina · 2017-09-20T08:02:40Z

cc: @ittaiz

ittaiz · 2017-09-20T08:43:04Z

Great, thanks. Just to make sure- this will respect neverlink semantics, right?

…

On Wed, 20 Sep 2017 at 11:02 Irina Iancu ***@***.***> wrote: cc: @ittaiz <https://github.com/ittaiz> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3769 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF3vfee26mnk4kL4in77VSl1nZX_Sks5skMamgaJpZM4PdfEx> .

ittaiz · 2017-10-15T10:39:07Z

Friendly ping

ittaiz · 2017-10-29T10:29:27Z

@iirina Stumbled on to a problem and I'm not sure if it should be a new issue or an expansion of this one.
The use_ijar=True in java_common.create_provider is currently unusable if one needs exports.
Tried a few alternatives and all of them failed:

compile_time_jars = outputs.full_jars, #error since provider doesn't propagate exports
compile_time_jars = outputs.full_jars + exports_jars.compile_jars.to_list(), #error since provider ijars existing ijars (e.g.g foo-ijar-ijar.jar)
compile_time_jars = outputs.full_jars + exports_jars.full_compile_jars.to_list(),# If passing full compile jars of the exports it fails since we generate outputs for a different target
#Output artifact 'external/com_twitter__scalding_date/jar/scalding-date_2.11-0.17.0-ijar.jar' not under package directory 'test' for target '//test:jar_export'.
#ERROR: Analysis of target '//test:jar_export' failed; build aborted: Analysis of target '//test:jar_export' failed; build aborted.

The reason I think this is related to this issue is because I think the real problem is the level of abstraction the current provider allows. Ideally I think it should support the API you mentioned above but also include runtime_deps, exports and neverlink.

WDYT?

You can see my attempt of using this feature in the following rules_scala branch

iirina · 2017-12-12T15:09:03Z

@dbabkin is currently implementing this.

ittaiz · 2017-12-12T20:05:18Z

Exciting! Thanks!

…

On Tue, 12 Dec 2017 at 17:09 Irina Iancu ***@***.***> wrote: @dbabkin <https://github.com/dbabkin> is currently implementing this. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3769 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF4BJEVLUufe4QaClsX2XVGU8UmOTks5s_pcWgaJpZM4PdfEx> .

ittaiz · 2017-12-24T05:59:56Z

Hi,
Any idea if this will hit the 0.10.0 release?

dbabkin · 2018-01-09T16:42:00Z

Hi,
Bazel contributor here :) Terribly sorry for delay in reply.

This task is in the progress right now.
First, very limited part of new API is about to be finished. Commit will be pushed today or tomorrow.
I'm not sure about release 0.10.0. Please follow the status of release 0.10 cut on this ticket:
#3958

New API supports creating JavaInfo with JavaCompilationArgsProvider only.
The next sub-task in scope of this story is to implement the list of all others providers.

ittaiz · 2018-01-09T17:20:31Z

Hi, Thanks for the update! This isn't exposed to skylark right? AFAIR JavaCompilationArgsProvider isn't exposed to skylark

…

On Tue, 9 Jan 2018 at 18:42 dbabkin ***@***.***> wrote: Hi, Bazel contributor here. :) Terribly sorry for delay in reply. This task is in the progress right now. First, very limited part of new API is about to be finished. Commit will be pushed today or tomorrow. I'm not sure about release 0.10.0. Please follow the status of release 0.10 cut on this ticket: #3958 <#3958> New API supports creating JavaInfo with JavaCompilationArgsProvider only. The next sub-task in scope of this story is to implement the list of all others providers. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3769 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIFx8cGLgaIQlFbGSa8_JheotxC3o-ks5tI5bcgaJpZM4PdfEx> .

iirina · 2018-01-10T10:29:47Z

No, it's not. It's just an implementation detail, meaning more work will be needed to make the new API 100% reliable.

I cannot tell right now if this will make the release, since it seems difficult to even cut an RC right now.

dbabkin · 2018-01-10T13:51:00Z

Hi Ittai,

We do not expose JavaCompilationArgsProvider itself.
If you take a look at JavaInfo class [1], it has 4 methods based on JavaCompilationArgsProvider
getCompileTimeJars()
getFullCompileTimeJars()
getTransitiveDeps()
getTransitiveRuntimeDeps()
This methods are exported already. And during creating JavaInfo using new API, JavaCompilationArgsProvider is populated now. So you can use this four methods now.

To create JavaInfo, you need to use constructor [2]. Please refer our tests for example [3]

javaInfo = JavaInfo(
    output_jar = ctx.file.output_jar,
    use_ijar = False,
    runtime_deps = dp
)
javaInfo.transitive_runtime_deps

Others methods from JavaInfo returns empty result now. To fix that I need to implement other providers. This is my priority at the moment.

[1] https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/java/JavaInfo.java;l=335

[2] https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/java/JavaInfo.java;l=113

[3] https://source.bazel.build/bazel/+/master:src/test/java/com/google/devtools/build/lib/rules/java/JavaSkylarkApiTest.java;l=1593

dbabkin · 2018-01-17T12:45:27Z

Hi,
Small update from my side:
JavaSourceJarsProvider had been created inside JavaInfo
source_jars - added to JavaSourceJarsProvider
sources - ignored at the moment.

dbabkin · 2018-01-30T15:02:32Z

Hi,
Update from my side:
Recently added JavaRuleOutputJarsProvider, JavaExportsProvider , JavaSourceJarsProvider.
Added support for 'exports' parameter.
I think we are pretty finish with functionality except minor things:

Input parameter 'sources' is ignored at the moment. ETA ~ few weeks
merge function for JavaInfo doesn't merge all providers. ETA ~1-2 days

johnynek · 2018-02-04T18:48:52Z

@dbabkin Thanks a lot! It sounds like there are a lot more moving pieces now. Will the documentation be updated when this ships?

dbabkin · 2018-02-12T08:47:03Z

Hi Oscar,

Yes I will add documentation to bazel.build web site after I finish implementation.

dbabkin · 2018-03-07T14:47:28Z

changes pushed.

ittaiz · 2018-03-08T04:31:55Z

Thanks!
Any chance for a link to the commit?

dbabkin · 2018-03-08T08:46:46Z

bb64d2f

cf79305

iirina added category: rules > java P2 We'll consider working on this in future. (Assignee optional) type: feature request labels Sep 20, 2017

iirina self-assigned this Sep 20, 2017

ittaiz mentioned this issue Sep 24, 2017

java_common.compile should support neverlink #3735

Closed

iirina assigned dbabkin Dec 12, 2017

This was referenced Dec 21, 2017

Scala import exports fixes bazelbuild/rules_scala#369

Merged

scala import should support neverlink bazelbuild/rules_scala#371

Closed

gkk-stripe mentioned this issue Jan 2, 2018

Converge on scala_import and java_import bazeltools/bazel-deps#102

Open

dbabkin mentioned this issue Feb 12, 2018

maven_jar .scala sources missing in IntelliJ bazelbuild/rules_scala#416

Open

dbabkin closed this as completed Mar 7, 2018

ittaiz mentioned this issue Apr 2, 2018

document how intellij bazel rules depend on scala rules bazelbuild/rules_scala#466

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace transitive_*_jars in java_common.create_provider with deps #3769

Replace transitive_*_jars in java_common.create_provider with deps #3769

iirina commented Sep 20, 2017

iirina commented Sep 20, 2017

ittaiz commented Sep 20, 2017 via email

ittaiz commented Oct 15, 2017

ittaiz commented Oct 29, 2017 •

edited

Loading

iirina commented Dec 12, 2017

ittaiz commented Dec 12, 2017 via email

ittaiz commented Dec 24, 2017

dbabkin commented Jan 9, 2018 •

edited

Loading

ittaiz commented Jan 9, 2018 via email

iirina commented Jan 10, 2018

dbabkin commented Jan 10, 2018

dbabkin commented Jan 17, 2018 •

edited

Loading

dbabkin commented Jan 30, 2018 •

edited

Loading

johnynek commented Feb 4, 2018

dbabkin commented Feb 12, 2018 •

edited

Loading

dbabkin commented Mar 7, 2018

ittaiz commented Mar 8, 2018

dbabkin commented Mar 8, 2018

Replace transitive_*_jars in java_common.create_provider with deps #3769

Replace transitive_*_jars in java_common.create_provider with deps #3769

Comments

iirina commented Sep 20, 2017

iirina commented Sep 20, 2017

ittaiz commented Sep 20, 2017 via email

ittaiz commented Oct 15, 2017

ittaiz commented Oct 29, 2017 • edited Loading

iirina commented Dec 12, 2017

ittaiz commented Dec 12, 2017 via email

ittaiz commented Dec 24, 2017

dbabkin commented Jan 9, 2018 • edited Loading

ittaiz commented Jan 9, 2018 via email

iirina commented Jan 10, 2018

dbabkin commented Jan 10, 2018

dbabkin commented Jan 17, 2018 • edited Loading

dbabkin commented Jan 30, 2018 • edited Loading

johnynek commented Feb 4, 2018

dbabkin commented Feb 12, 2018 • edited Loading

dbabkin commented Mar 7, 2018

ittaiz commented Mar 8, 2018

dbabkin commented Mar 8, 2018

ittaiz commented Oct 29, 2017 •

edited

Loading

dbabkin commented Jan 9, 2018 •

edited

Loading

dbabkin commented Jan 17, 2018 •

edited

Loading

dbabkin commented Jan 30, 2018 •

edited

Loading

dbabkin commented Feb 12, 2018 •

edited

Loading