Issue #2929: implement inspect for modules

[Shri333]

Implement inspect for modules:

• Name of module with file/line info
• Scaladoc on module
• Inherited modules
• Module dependencies
• Default task
• Implemented tasks

[lihaoyi]

@Shri333 let's move the discussion for this PR here rather than the original issue

I played around a bit myself and it does seem problematic to get the Scala declared methods via java reflection. Rather than using scala reflection at runtime, we can instead use scala reflection in the Discover.scala macro where we already do this kind of thing.

Discover.scala currently looks for every class that is part of our module hierarchy, and generates a tuple of two lists:

1. A list of mainargs entrypoints, used to support Task.Commands, which we need statically generated so we can perform typeclass resolution on the parameter names
2. A list of all method names, inherited or not, used to perform "did you mean foo?" error reporting

It should be possible to extend that tuple to include a third list for each class: a list of method names that are directly declared by this particular object. You would then be able to use that information in inspect to return the list of directly declared methods.

This approach is a bit roundabout compared to using scala-reflect at runtime, but it should work, and help us preserve the "no scala reflect at runtime" invariant we've maintained so far

[Shri333]

@lihaoyi I pushed some changes (fb20fa9) to get all declared methods in Discover.scala and add it to the tuple. When I print out the mapping in Discover.scala, it prints the correct values. However, when I access it in MainModule.scala with discover.value, the Map is empty. Do you know why this could be occurring?

[lihaoyi]

@Shri333 when you say the Map is empty, are you referring to the entire Discover#value field? That needs to get populated by Mill for normal functionality to work, so it shouldn't be empty in the main branch.

You can try cherry-picking your minimal changes onto main to see whether it's still empty there. If it is empty after the cherry pick, it means your minimal changes are breaking something, so you'll need to bisect your minimal changes further to find out who is breaking it. If it's not empty after the cherry pick, then you'll need to bisect your other changes to figure out what is causing it to be empty

[Shri333]

@Shri333 when you say the Map is empty, are you referring to the entire Discover#value field? That needs to get populated by Mill for normal functionality to work, so it shouldn't be empty in the main branch.

You can try cherry-picking your minimal changes onto main to see whether it's still empty there. If it is empty after the cherry pick, it means your minimal changes are breaking something, so you'll need to bisect your minimal changes further to find out who is breaking it. If it's not empty after the cherry pick, then you'll need to bisect your other changes to figure out what is causing it to be empty

@lihaoyi I was able to fix the issue. Turns out I was supposed to use evaluator.rootModule.millDiscover rather than Discover[t.module.type]. In that case, are there any other features you want to add? I have implemented name of module with file/line info, scaladoc on module, directly inherited modules, module dependencies, default task, and directly implemented tasks. I also added doc annotation tests. The only thing remaining is to fix the resolve tests.

[Shri333]

@lihaoyi if I am not mistaken, resolve itself seems to be working incorrectly (on main). If I have the following build.mill:

package build
import mill._, javalib._

trait TypeA extends Module {
  def foo = Task { "foo" }
}
trait TypeB extends Module {
  def bar = Task { "bar" }
}
trait TypeC extends Module {
  def baz = Task { "baz" }
}
trait TypeAB extends TypeA with TypeB

object typeA extends TypeA
object typeB extends TypeB
object typeC extends TypeC {
  object typeA extends TypeA
}
object typeAB extends TypeAB

Running mill resolve _:TypeA._ produces the following output:

typeA.foo
typeAB.bar
typeAB.foo

But, running mill resolve __:TypeA._ produces the following output:

clean
init
inspect
path
plan
resolve
show
showNamed
shutdown
typeA
typeA.foo
typeAB
typeAB.bar
typeAB.foo
typeB
typeC
typeC.typeA.foo
version
visualize
visualizePlan

Is this expected? This was also the weird behavior I was seeing in the resolve tests. I know that __ matches any # of segments while _ matches one segment, but I was confused about this output.

[lihaoyi]

@Shri333 that looks like it might be a bug in the :TypeA syntax. I think just open an issue to track it, but no need to fix it as part of this PR since it's unrelated and pre-existing

[Shri333]

@lihaoyi the PR should be ready to review (green CI)

[lihaoyi]

Thanks @Shri333 ! I'm traveling now but I will try to find time to review this PR as soon as possible

[lihaoyi]

Went through the code and tried it out locally. This looks great!

Nice work @Shri333, and thanks for bearing with me and with the Mill codebase the last 2-3 weeks! Email me your bank transfer details and I will close out the bounty