Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom repository for Scala artifacts not propagated to bloop #2802

Closed
WojciechMazur opened this issue Mar 15, 2024 · 5 comments · Fixed by #2873
Closed

Custom repository for Scala artifacts not propagated to bloop #2802

WojciechMazur opened this issue Mar 15, 2024 · 5 comments · Fixed by #2873
Assignees
@Gedochao
Labels
Bloop Issues tied with Bloop integration. bug Something isn't working

Comments

@WojciechMazur
Copy link
Contributor

Version(s)
1.2.0

Describe the bug
In the Open Community Build we're building and publishing the Scala 3 compiler artefacts to the custom repository. These artifacts are used to check if projects can be build against given versions of the compiler. It's invoked like:

scala-cli compile . --scala-version=3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT --repository=https://custom/maven/repo/3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT

When compiling with --server=false it works correctly, however, when running with bloop we get exception:

Compiling project (Scala 3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT, JVM (11))
Error compiling project (Scala 3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT, JVM (11))
Error: Unexpected error when compiling repo_d49e65e4dd_6e665ca4ab: sbt.internal.inc.InvalidComponent: Couldn't retrieve module org.scala-lang:scala3-sbt-bridge:3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT:compile
        at sbt.internal.inc.BloopComponentCompiler.$anonfun$compileAndInstall$2(BloopComponentCompiler.scala:276)
        at sbt.internal.inc.BloopComponentCompiler.$anonfun$compileAndInstall$2$adapted(BloopComponentCompiler.scala:257)
        at sbt.io.IO$.withTemporaryDirectory(IO.scala:488)
        at sbt.io.IO$.withTemporaryDirectory(IO.scala:498)
        at sbt.internal.inc.BloopComponentCompiler.$anonfun$compileAndInstall$1(BloopComponentCompiler.scala:257)
        at sbt.internal.inc.BloopComponentCompiler.$anonfun$compileAndInstall$1$adapted(BloopComponentCompiler.scala:255)
        at sbt.io.IO$.withTemporaryDirectory(IO.scala:488)
        at sbt.io.IO$.withTemporaryDirectory(IO.scala:498)
        at sbt.internal.inc.BloopComponentCompiler.compileAndInstall(BloopComponentCompiler.scala:255)
        at sbt.internal.inc.BloopComponentCompiler.$anonfun$compiledBridgeJar$1(BloopComponentCompiler.scala:245)
        at sbt.internal.inc.IfMissing$Define.run(IfMissing.scala:19)
        at sbt.internal.inc.BloopComponentManager.createAndCache$1(BloopComponentManager.scala:46)
        at sbt.internal.inc.BloopComponentManager.$anonfun$files$3(BloopComponentManager.scala:60)
        at sbt.internal.inc.BloopComponentManager.getOrElse$1(BloopComponentManager.scala:38)
        at sbt.internal.inc.BloopComponentManager.$anonfun$files$2(BloopComponentManager.scala:60)
        at sbt.internal.inc.BloopComponentManager$$anon$2.call(BloopComponentManager.scala:89)
        at bloop.ComponentLock.apply(BloopComponentsLock.scala:10)
        at bloop.ComponentLock.apply$(BloopComponentsLock.scala:9)
        at bloop.BloopComponentsLock$.apply(BloopComponentsLock.scala:14)
        at sbt.internal.inc.BloopComponentManager.lock(BloopComponentManager.scala:89)
        at sbt.internal.inc.BloopComponentManager.$anonfun$lockSecondaryCache$1(BloopComponentManager.scala:84)
        at scala.Option.map(Option.scala:230)
        at sbt.internal.inc.BloopComponentManager.lockSecondaryCache(BloopComponentManager.scala:84)
        at sbt.internal.inc.BloopComponentManager.fromSecondary$1(BloopComponentManager.scala:58)
        at sbt.internal.inc.BloopComponentManager.$anonfun$files$6(BloopComponentManager.scala:64)
        at sbt.internal.inc.BloopComponentManager.getOrElse$1(BloopComponentManager.scala:38)
        at sbt.internal.inc.BloopComponentManager.$anonfun$files$5(BloopComponentManager.scala:64)
        at sbt.internal.inc.BloopComponentManager$$anon$2.call(BloopComponentManager.scala:89)
        at bloop.ComponentLock.apply(BloopComponentsLock.scala:10)
        at bloop.ComponentLock.apply$(BloopComponentsLock.scala:9)
        at bloop.BloopComponentsLock$.apply(BloopComponentsLock.scala:14)
        at sbt.internal.inc.BloopComponentManager.lock(BloopComponentManager.scala:89)
        at sbt.internal.inc.BloopComponentManager.lockLocalCache(BloopComponentManager.scala:82)
        at sbt.internal.inc.BloopComponentManager.files(BloopComponentManager.scala:64)
        at sbt.internal.inc.BloopComponentManager.file(BloopComponentManager.scala:72)
        at sbt.internal.inc.BloopComponentCompiler.compiledBridgeJar(BloopComponentCompiler.scala:245)
        at sbt.internal.inc.BloopComponentCompiler$BloopCompilerBridgeProvider.compiledBridge(BloopComponentCompiler.scala:123)
        at sbt.internal.inc.BloopComponentCompiler$BloopCompilerBridgeProvider.fetchCompiledBridge(BloopComponentCompiler.scala:129)
        at sbt.internal.inc.AnalyzingCompiler.getDualLoader(AnalyzingCompiler.scala:354)
        at sbt.internal.inc.AnalyzingCompiler.getCompilerLoader(AnalyzingCompiler.scala:343)
        at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:87)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.compileSources$1(BloopHighLevelCompiler.scala:133)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.$anonfun$compile$7(BloopHighLevelCompiler.scala:155)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.$anonfun$compile$1(BloopHighLevelCompiler.scala:71)
        at bloop.tracing.NoopTracer$.trace(BraveTracer.scala:53)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.timed$1(BloopHighLevelCompiler.scala:70)
        at sbt.internal.inc.bloop.internal.BloopHighLevelCompiler.$anonfun$compile$6(BloopHighLevelCompiler.scala:155)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at monix.eval.internal.TaskRunLoop$.startFull(TaskRunLoop.scala:81)
        at monix.eval.internal.TaskRestartCallback.syncOnSuccess(TaskRestartCallback.scala:101)
        at monix.eval.internal.TaskRestartCallback.onSuccess(TaskRestartCallback.scala:74)
        at monix.eval.internal.TaskExecuteOn$AsyncRegister$$anon$1.run(TaskExecuteOn.scala:71)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: coursierapi.error.SimpleResolutionError$1: Error downloading org.scala-lang:scala3-sbt-bridge:3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT
  not found: /Users/wmazur/.ivy2/local/org.scala-lang/scala3-sbt-bridge/3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT/ivys/ivy.xml
  not found: https://repo1.maven.org/maven2/org/scala-lang/scala3-sbt-bridge/3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT/scala3-sbt-bridge-3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT.pom
  not found: https://scala-ci.typesafe.com/artifactory/scala-integration/org/scala-lang/scala3-sbt-bridge/3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT/scala3-sbt-bridge-3.4.2-RC1-bin-8b1618573bdf0c9cd9704ed4772fcec9c213ca32-SNAPSHOT.pom
        at coursierapi.error.SimpleResolutionError.of(SimpleResolutionError.java:11)
        at coursierapi.shaded.coursier.internal.api.ApiHelper$.simpleResError(ApiHelper.scala:336)
        at coursierapi.shaded.coursier.internal.api.ApiHelper$.doFetch(ApiHelper.scala:380)
        at coursierapi.shaded.coursier.internal.api.ApiHelper.doFetch(ApiHelper.scala)
        at coursierapi.Fetch.fetchResult(Fetch.java:244)
        at coursierapi.Fetch.fetch(Fetch.java:239)
        at bloop.DependencyResolution$.resolveDependenciesWithErrors(DependencyResolution.scala:90)
        at bloop.DependencyResolution$.resolveWithErrors(DependencyResolution.scala:65)
        at sbt.internal.inc.BloopComponentCompiler.$anonfun$compileAndInstall$2(BloopComponentCompiler.scala:267)
        ... 58 more

Then investigating the logs you can see that custom repository https://custom/maven/repo/... was not included in the search of artifacts in the bloop

To Reproduce
Publish Scala 3 artifacts to custom remote repository, then try to use them in scala-cli

Expected behaviour
The custom repositories passed to the scala-cli should allow to find custom Scala 3 artefacts repositories.
@WojciechMazur WojciechMazur added the bug Something isn't working label Mar 15, 2024
@Quafadas Quafadas mentioned this issue Apr 2, 2024
Closed
@TomasMikula
Copy link

I'm also encountering this.

In the meantime, is it possible to set --server=false in some project settings rather than repeat every time on the command line?

@Gedochao Gedochao added the Bloop Issues tied with Bloop integration. label Apr 3, 2024
@tgodzik
Copy link
Member

tgodzik commented Apr 3, 2024

@TomasMikula is this the same issue with the sbt bridge or something else entirely? Bloop downloaded sbt bridges for a specific Scala version on demand and I think it doesn't have COURSIER_REPOSITORIES env variable set.

@WojciechMazur Did you try setting the variable for the entire VM?

Alternatively, to improve the situation we could set the env variable automatically when starting Bloop and if repositories were set.

@WojciechMazur
Copy link
Contributor Author

@tgodzik I wasn't aware of this env variable, setting it up seams to fix the problem, thank you.

It might be a good usability feature if we wouldn't need to set env var up, especially since we need to also add the standard repositories as well, but I guess it requires a sever changes to Bloop.

@tgodzik
Copy link
Member

tgodzik commented Apr 3, 2024

It's only an issue if you use a custom compiler version and it's a bit tricky to fix it otherwise. Probably, the easiest way to do it is to provide those env variables when starting bloop with non default repositories.

@TomasMikula
Copy link

Ah, upon inspecting the stack trace, it's a different problem:

Error fetching artifacts:
https://my.company.com/repo/ch/epfl/scala/com-microsoft-java-debug-core/0.34.0+20/com-microsoft-java-debug-core-0.34.0+20.jar: not found: https://my.company.com/repo/ch/epfl/scala/com-microsoft-java-debug-core/0.34.0+20/com-microsoft-java-debug-core-0.34.0+20.jar
https://my.company.com/repo/ch/epfl/scala/directory-watcher/0.8.0+6-f651bd93/directory-watcher-0.8.0+6-f651bd93.jar: not found: https://my.company.com/repo/ch/epfl/scala/directory-watcher/0.8.0+6-f651bd93/directory-watcher-0.8.0+6-f651bd93.jar

Usually, artifacts are blocked if there's a known vulnerability. I will try to find out whether that's the case here.

Sorry for the noise.

@Gedochao Gedochao self-assigned this Apr 23, 2024
@Gedochao Gedochao mentioned this issue Apr 25, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Gedochao Gedochao

Labels
Bloop Issues tied with Bloop integration. bug Something isn't working
Projects
Issue Board
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Explicitly pass scala3-sbt-bridge to Bloop Gedochao/scala-cli
4 participants
@TomasMikula @tgodzik @Gedochao @WojciechMazur