internal: Add Cargo-style project discovery for Buck and Bazel Users

[davidbarsky]

This feature requires the user to add a command that generates a rust-project.json from a set of files. Project discovery can be invoked in two ways:

1. At extension activation time, which includes the generated rust-project.json as part of the linkedProjects argument in InitializeParams.
2. Through a new command titled "rust-analyzer: Add current file to workspace", which makes use of a new, rust-analyzer-specific LSP request that adds the workspace without erasing any existing workspaces. Note that there is no mechanism to remove workspaces other than "quit the rust-analyzer server".

Few notes:

• I think that the command-running functionality could merit being placed into its own extension (and expose it via extension contribution points) to provide build-system idiomatic progress reporting and status handling, but I haven't (yet) made an extension that does this nor does Buck expose this sort of functionality.
• This approach would just work for Bazel. I'll try and get the tool that's responsible for Buck integration open-sourced soon.
• On the testing side of things, I've used this in around my employer's Buck-powered monorepo and it's a nice experience. That being said, I can't think of an open-source repository where this can be tested in public, so you might need to trust me on this one.

I'd love to get feedback on:

• Naming of LSP extensions/new commands. I'm not too pleased with how "rust-analyzer: Add current file to workspace" is named, in that it's creating a new workspace. I think that this command being added should be gated on rust-analyzer.discoverProjectCommand on being set, so I can add this in sequent commits.
• My Typescript. It's not particularly good.
• Suggestions on handling folders with both Cargo and non-Cargo build systems and if I make activation a bit better.

(I previously tried to add this functionality entirely within rust-analyzer-the-LSP server itself, but matklad was right—an extension side approach is much, much easier.)

[Veykril]

I feel like an addProject extension isn't the right tool here, instead I think the client should track all linked projects (including those of virtual project-json's) itself and tell the server to reload with the fresh set of supplied projects on change of these projects.

cc #13446

[davidbarsky]

I feel like an addProject extension isn't the right tool here, instead I think the client should track all linked projects (including those of virtual project-json's) itself and tell the server to reload with the fresh set of supplied projects on change of these projects.

I can see how that approach is better. I think it'd allow for a Buck/Bazel extension to manage linked projects and (potentially) surface them through a UI and mapping each linked project to a BUILD/BUCK file. One question, though: linkedProjects only gets sent once, during the initial server setup as a part of InitializeParams. I don't think it's a good idea to fully reinitialize r-a on each reload, but I can change rust-analyzer/reload to accept linkedProjects (or even a full ConfigData!) instead of just (). Is that approach okay with you?

[Veykril]

Yes, I forgot to add that, we'd still need this to be a request, repurposing reloadWorkspace sounds good.

or even a full ConfigData

LSP already has a configuration changed notification so we could also re-use this and just update the linked projects key there. The main problem here is figuring out how to shove an "inline" rust project specification into there since right not this takes a list of file paths only

[davidbarsky]

Yes, I forgot to add that, we'd still need this to be a request, repurposing reloadWorkspace sounds good.

or even a full ConfigData

LSP already has a configuration changed notification so we could also re-use this and just update the linked projects key there. The main problem here is figuring out how to shove an "inline" rust project specification into there since right not this takes a list of file paths only

Thankfully, we don't need to figure this out! linkedProjects already accepts an inline rust-project. It's what I'm doing in this PR (sorry, I've spent a lot of time staring at this part of the codebase...)

Just to make sure I understand, we want to do:

1. Send a configuration changed notification to r-a.
2. Send the actual, full new configuration (with the updated linkedProjects).

[davidbarsky]

I've added a workspace configuration-based approach to reloading the workspace in cef2d62. I haven't removed the rust-analyzer/AddProject-based approach yet because the way I've implementing reloading feels like an abuse of vscode.WorkspaceConfiguration. Let me know what approach y'all prefer: I'm happy to change this PR accordingly.

[davidbarsky]

I think I've addressed all the comments:

• I've rebased against the master branch.
• I've removed rust-analyzer/addProject in favor of notifying the server that configuration has changed. Note that the rust-project.json is never written to disk; it's synthesized entirely in-memory and placed into the InitializeParams. Actually updating VS Code's configuration would result in writing to disk into the workspace-level .vscode/ folder, but I think exposing this detail to users is a poor UX.

[Veykril]

Will take a proper look tomorrow but

Actually updating VS Code's configuration would result in writing to disk into the workspace-level .vscode/ folder, but I think exposing this detail to users is a poor UX.

fully agree with this, we should not be changing the user config. We only need to synthesize the linkedProjects field when passing it to the server, this way we could in the future also expose an API for the vscode extension that other extensions can hook into (which I think is what matklad was describing in the other issue at some point)

[davidbarsky]

Will take a proper look tomorrow

No worries, take care!

fully agree with this, we should not be changing the user config. We only need to synthesize the linkedProjects field when passing it to the server, this way we could in the future also expose an API for the vscode extension that other extensions can hook into

Yup, that's the approach that I took here. It seems like the config was written out to the workspace-level .vscode/ when key was updated in vscode.WorkspaceConfiguration; there doesn't seem to be a way to keep that in the extension alone.

which I think is what matklad was describing in the other issue at some point

He was. I tried a different approach, and when that I didn't work out, I was able to get this approach working in about 2 days. I'm a bit annoyed with myself that I didn't listen to him first!

[Veykril]

One more thing

[Veykril]

I am not sure whether this is the right way we want to tackle this (that is having this command be part of r-a in long term), but for the time being I'd say that's okay.

[davidbarsky]

I am not sure whether this is the right way we want to tackle this (that is having this command be part of r-a in long term), but for the time being I'd say that's okay.

My feeling is that I'd like to get some user feedback first, but I strongly suspect that this should live in an external extension that rust-analyzer coordinates with. I don't expect rust-analyzer to worry about maintaining this for an extended period of time.

[davidbarsky]

Sorry about the back and forth, but thank you for your patience :)

[Veykril]

@bors r+

[bors]

📌 Commit 6e7bc07 has been approved by Veykril

It is now in the queue for this repository.

[bors]

⌛ Testing commit 6e7bc07 with merge c15335c...

[bors]

☀️ Test successful - checks-actions
Approved by: Veykril
Pushing c15335c to master...

[woody77]

When is the discovery command run? Is it run in lieu of opening a rust-project.json file?

[davidbarsky]

When is the discovery command run? Is it run in lieu of opening a rust-project.json file?

@woody77 At the moment, it can run in two situations:

• when the extension is activating (from either an editor restart or a Rust file being opened for the first time), or
• when the user runs the "Add Project to Current Workspace" command through the command palette.

To be entirely honest, I think I was a bit too conservative with the latter option. I think that we should be probably be running the discovery command each time the user opens a Rust file that isn't part of the current workspace—rust-analyzer is stupidly fast at loading projects, and getting the results from Buck takes a few seconds at worst.