Visual Studio does not reload source generator assemblies when they change on disk

[vatsan-madhavan]

I'm just started writing a source generator, and I'm finding that Visual Studio is caching source generators aggressively, and it's making it very hard to do iterative development.

This is what I'm having to do to make even small changes.

• Close Visual Studio (devenv.exe) instance in which the source generator project and its consumer (i.e., the project that ProjectReference's the source generator)
• pskill /t servicehub.roslyncodeanalysisservice
• del $env:TEMP\VS\AnalyzerAssemblyLoader -Recurse -Force <- This is where the source generator binaries seem to get cached. Both devenv and servicehub.roslyncodeanalysisservice seems to hold handles to files under this location

Am I missing something simple to get the development process working more seamlessly ?

I'm on 5.0.100-rc.2.20473.20 + Visual Studio 2019 Enterprise 16.8.0 Preview 4.0 [30517.14.main

[vatsan-madhavan]

Is there a way to view the generated sources except through VS/intellisense ?

[Youssef1313]

@vatsan-madhavan

For the purpose of testing/debugging, I was viewing them with a simple hack (which isn't good, but did the purpose for me).

Youssef1313/PrintMembersGenerator@31e31ce

Then after things got stable with me, I moved to unit testing to confirm the correctness of the generated sources.

https://github.com/Youssef1313/PrintMembersGenerator/blob/master/src/PrintMembersGeneratorTests/PrintMembersGeneratorTest.cs

[CyrusNajmabadi]

Is there a way to view the generated sources except through VS/intellisense ?

As of yesterday, you can now emit generated files to disk so you can inspect them: #47047

[vatsan-madhavan]

This is awesome! This means I can get by with commandline builds while VS catches up!

[jasonmalinowski]

So the one problem @vatsan-madhavan you're probably running into is once we've loaded your assembly...the CLR doesn't give us sane ways to unload it or load a different version if the assembly version hasn't changed. @chsienki or @cartermp any chance somebody already has some MSBuild magic to work around this in some way?

[jaredpar]

Curious: if the generators in this case are strong name signed could we potentially manipulate the version here and load the new copy?

[jasonmalinowski]

@jaredpar Potentially. As crazy as the feature request is, I almost wish the compiler had a feature where it'd generate a (determinstic) but effectively random version and stuff that into the assembly version. 😄

[jaredpar]

Honestly this could be done as a simple post-build sttep. Load the binary in memory, use the metadata writer to flip the strong name bit, change the version and then load that vs. the one on disk.

[jasonmalinowski]

@jaredpar Thanks for volunteering!

[jaredpar]

The best part about being a lead, maybe the only good part, is the ability to delegate ... @chsienki

😉

[PathogenDavid]

I almost wish the compiler had a feature where it'd generate a (determinstic) but effectively random version and stuff that into the assembly version. 😄

The module version ID would be good for this. Just kludge it into the version number. (It's been a hot minute since I've looked into it, but if I remember right with determinism enabled it's the SHA1/SHA256 of all the compiler inputs or something along those lines.)

Right now specifying a wildcard for the assembly version results in CS8357 if determinism is enabled. Maybe determinism + wildcard could mean "kludge MVID into version number".

[MisinformedDNA]

Is the Roslyn team not going through the same pains as us?

[jaredpar]

We use source generators inside our main solution hence we are dogfooding the experience every day. The source generators aren't iterated on as frequently though hence we don't hit the specific reload problem.

This is a problem we are taking a look at. It's existed since Roslyn 1.0 with analyzers (so roughly five years now) hence it's not a new problem, generators has just shined a new light on it. There are some ideas on how to work around this (see my comments above). At the moment though Roslyn does most of the evaluation in process and given we are still on .NET Desktop that limits our options a bit because of the inability to load multiple copies of a DLL into the same process space unless it's strong name signed + changes versions on every build.

[tmat]

@PathogenDavid Using hash for version is not viable since it's not monotonic and versions are expected to be.

I'd suggest that instead of running source generator in VS when developing it it's better to run it in a unit test. Write a unit test that runs the generator and produces output. In that setting you can iterate fast - even using Edit and Continue to modify the generator code as you are debugging it. There is no need to mess with versions/reloading/VS complexity etc.

[tmat]

@jasonmalinowski

So the one problem @vatsan-madhavan you're probably running into is once we've loaded your assembly...the CLR doesn't give us sane ways to unload it or load a different version if the assembly version hasn't changed. @chsienki or @cartermp any chance somebody already has some MSBuild magic to work around this in some way?

The magic is called Core CLR ;-).

[PathogenDavid]

Using hash for version is not viable since it's not monotonic and versions are expected to be.

Fair enough for making that the default behavior of *, but for the purposes of this discussion the version number being monotonic doesn't matter.

I'd suggest that instead of running source generator in VS when developing it it's better to run it in a unit test.

This is essentially what I've been doing. (Except running dotnet build outside of Visual Studio with EmitCompilerGeneratedFiles enabled.)

The main frustration for me has been that once I have finished working on my source generator, convincing Visual Studio to relinquish whatever hidden cached source generator it is seemingly impossible to do reliably. At the very least I'd expect restarting Visual Studio to fix things, but even stopping all instances of Visual Studio, killing any lingering service hubs, deleting the cache folder named in the main issue, deleting all folders starting with vs in my temp folder, making a blood sacrifice, deleting my .vs folder, and finally restarting Visual Studio: My old generator still sometimes (somehow) sticks around. (But only sometimes.)

[MisinformedDNA]

I added a documentation issue to add unit testing information. Any tutorial for creating source generators should have unit testing guidance included.

[Eli-Black-Work]

• For other cases (which should be the majority of your time), you are not actively working on the source generator implementation. In this case, the source generator behavior is stable inside Visual Studio and unexpected errors will not be occurring.

@sharwell The issue my team runs into is:

1. I make changes to our source generator and merge them into our main branch.
2. My coworkers pull main and then their code won't compile (because Visual Studio is still using the old version of the source generator).

This means that whenever I change our code generator, I need to ping everyone to let them know that after they pull the latest changes, they need to restart Visual Studio 😆

This also happens when switching branches, if one branch is based on newer code and the other is based on older code, so we end up needing to rebase our branches more than we normally would 🙂

[weltkante]

What works for me is setting up an internal nuget repo (can be as simple as a network share folder where the packages go) and put versioned packages with the analyzer/codegen there. Thats the only reliable way I found to get things working in a team.

[pavlexander]

Can't we just have a button in visual studio that restarts roslyn / refresh intellisense? So many intellisense bugs over the years. Why do we always have to restart visual studio?

I had a horrible first experience trying to use the new source generators. Until i realized that even adding a property to a class requires me to restart visual studio before intellisense picks it up. How does something like this get released and goes by unnoticed?

experienced the same frustration today. Heard lots of positive reviews of code-gen and about a new world of possibilities, then tried it out myself, then spent 1.5 hours trying to figure out why the most simplistic, basic, trivial hello-world generator's not working.. turns out you have to restart VS on each update.. Extremely non-friendly approach, must say.. I sincerely hope the situation will be better with next VS releases.. Kudos to Rider team for actually caring about the end users.

[CyrusNajmabadi]

@pavlexander the best way to do this development is through unit testing.

This is fully supported, fast, and what we advise people do.

[AdmiralSnyder]

@pavlexander
have a look at my extension - it works well and the amount of additional procedure is totally appropriate.
https://marketplace.visualstudio.com/items?itemName=AlexanderGayko.AutoUpdateAssemblyName

[JeremyTCD]

I'd just like to say that unit testing does not solve the problem for some of us: tests do allow us to avoid restarts while working on generators, but for those of us who are heavily dependent on generators (e.g. dozens of generators within a solution), we often need to make small modifications or create new generators - even if we develop against unit tests, we still face multiple Visual Studio restarts per day. Would be beneficial to keep an open mind about how devs are using generators.

[weltkante]

If you override GetProjectCompilationAsync on the verifier test class you can, after calling the base implementation, extract the generated source trees from the compilation and write them to the file system. You can then have a second project in the same solution that doesn't use the generator but consumes the source generated from the unit test to do incremental development.

[MiheevN]

I have to restart VS 50 times a day...
Considering that I don’t turn off the PC, this is more than usual in a YEAR!

It's strange that such an obvious and important functionality has not been implemented for so long.
I hope everything will be fixed in the next version.

When you write a huge generator that simply produces uncompiled code when errors occur, what tests will help?
The code won't run.
You need to look at the resulting code with one eye and at the generator with the other.

[CyrusNajmabadi]

what tests will help?

The tests validate the behavior on inputs and ensure you don't regress how it behaves as you make changes.

You need to look at the resulting code with one eye and at the generator with the other.

Tests do exactly this.

[AdmiralSnyder]

@MiheevN
have a look at my extension - it solves that exact problem. it works well and the amount of additional procedure is totally appropriate. - oh, and it's free.
https://marketplace.visualstudio.com/items?itemName=AlexanderGayko.AutoUpdateAssemblyName

[ZzZombo]

How exactly do you use NuGet packages for this? I did make one for my SG, but in order to make the IDE pick the changes up, I need to unload the package by removing it from the project dependencies, and then re-add it. Simply updating the package doesn't seem to work.

Also, why does VS not have a problem with NuGet packaged SGs, but loose assemblies can't be updated w/o restarting?

[CyrusNajmabadi]

Neither will work unless you change some part of the assembly identity. It's a limitation of net framework (which vs still uses).

[yoziv]

is this still the behaviour now?
not even debugging a SG , just consuming a working SG in a project took me like 5 VS restarts

[CyrusNajmabadi]

@yoziv please file an issue with a repro. Thanks!

[VirRus77]

Microsoft Visual Studio Professional 2019
Version 16.11.39
The behavior is repeated.

[CyrusNajmabadi]

Microsoft Visual Studio Professional 2019
Version 16.11.39
The behavior is repeated.

This was only just changed in the latest version of roslyn, and will only be in the upcoming 17.x version of VS, not 16.11

[333fred]

Needed to revert this for now due to some exceptions in VS insertions. We'll work on getting it revised and re-merged shortly.