Make worksheets work on root level with the same classpath as sbt console

[bjornregnell]

Is your feature request related to a problem? Please describe.
This feature request is related to scalameta/metals#2363

With sbt you can put code on top level and get it to run without any src or together with stuff in src.

Currently, workshets in metals (server version 0.9.8) does not work outside src. This is problematic for the following reasons:

• it is inconsistent with how sbt works, compared to sbt console at root project level. So code that works in the REPL using sbt's console will not work in a metals worksheet if it is placed on top level.
• it makes it difficult for beginners; it is much easier to create a project without the src/main/scala ceremony and just put all code files on top level in the project dir.

Describe the solution you'd like

Relax the restriction that a worksheet must be inside src to see the code that gets compiled, by making worksheets at top-level also see the project code.

• If a worksheet, e.g. test.worksheet.sc, is placed at root level then it should have the same stuff on the classpath as sbt console and see all stuff compiled in top-level files such as Main.scala in such a project:

├── build.sbt
├── Main.scala
└── test.worksheet.sc

• If src/main/scala exists that code should also be on the classpath.

• If there are several sub-projects, then the root level build.sbt should define the classpath as in ThisBuild

Describe alternatives you've considered

1. Status quo. If it works as in v0.9.8, then you may fall in the pitfall of not understanding why your worksheet can't see your code. And you can't experiment with your top-level code in a worksheet without re-arranging your project and creating and src/main/scala etc. This means that many existing projects with code on top level will not work with worksheet out of the box.
2. Partial solution. It would be a step forward if a top-level worksheet sees the top-level code in the default package, even if it does not see code in src.

The Full solution means investigate what sbt console from project root level puts on the class path of the REPL-instance and use that classpath also on top-level worksheets in metals.

Search terms:

worksheet, project directory structure, metals

[bjornregnell]

Discussion on solutions are available here: scalameta/metals#2363

[tgodzik]

I did dig around a bit and this might be a bit more complex. Standalone files (outside source directories) are only compiled by Metals if a the build was reimported, because sbt treats them separately than source directories and they are not added if they do not exist. And if they have a package declared, a worksheet inside the source directories will actually pick them up, but not a worksheet outside them.

What we might do here is treat worksheet the same as sbt would have treated normal scala files. In essence we should be able to add an additional condition that for sbt targets if a file is located inside the module directory it should be treated as a source file for that module. We should also suggest reimporting the build whenever a new file is added at root of a module.

That however raises some questions:

• should we really replicate sbt's logic here?
• how to make it work sensibly without reimporting the project? Or maybe we should suggest actually using source directories in this case? Otherwise when using Bloop it will be far from a perfect experience (each new file will request a reimport)

I think it would be much safer to still teach people to use source directories, especially if they intend to work with sbt. Those directories are a bit complex for sure, but in that case we can use custom simpler source directories. Alternatively, they could use Mill, which does not follow the standard here and you can just use src.

[ckipp01]

should we really replicate sbt's logic here?

My opinion would probably be no. You hit on it in the last paragraph, but I think having a workspace that has a valid build plus random Scala files at the top level brings even more confusion since it gives the impression that it doesn't matter where your files are, which just isn't the case. As you also hit on, there is no guarantee that a user is using sbt, so not modeling worksheets off the way sbt console works also allows us to have consistency for worksheets across build tools.

[bjornregnell]

Hmmm. We also have the situation with a small projects with everything on top. This is very common among beginners (at least among my students). Another situation is when you just start a small project that does not need packages and you want to quickly get things done with minimal cruft.

I'm thinking like this: How can we make it super easy to get going? After sbt got really easy for beginners with minimal fuzz my student started to feel comfortable with using it. Imagine starting a clean vscode dir and adding scala files and worksheets and it just works. That's a good user experience IMHO. Even if we "teach people" to jump over hoops with the src/main/scala ceremony, the simple and most intuitive option would still be to just put it on top level if you don't need packages and there is not that much code. That's what happens now in Scala 3 with top level definitions and @main; it is the most simple way to start comparing to having to put a main-def in an object.

Also, the empty package is part of the language spec so I think it should work IMHO.

plus random Scala files at the top level brings even more confusion

I don't think scala-files on top are 'random'; they are/should be visible to the build as it inside the project dir. Consistency with how sbt interprets build.sbt would IMHO reduce confusion.

[bjornregnell]

Adding here for reference the text in the language spec about the empty package:
"Top-level definitions outside a packaging are assumed to be injected into a special empty package"
it is on page 108: https://scala-lang.org/files/archive/spec/2.13/spec.pdf
So in that spirit, a top level worksheet should also work IMHO...

[Quafadas]

As a note, if this issue was "looked into", it would be great to do alongside notebook support, which I assume would have a similar set of problems?