Update to Roslyn 1.2

[sharwell]

• Update to xUnit.net 2.2.0-beta1
• Update to Roslyn 1.2.0-rc
• Replace generated code detection with Roslyn's built-in functionality
• Replace StyleCopSettings cache with Roslyn's built-in functionality
• Enable concurrent analyzer execution
• Remove workarounds for Roslyn 1.0 and/or 1.1

[codecov-io]

Current coverage is 93.46%

Merging #2102 into master will increase coverage by +0.10% as of 1a27559

@@            master   #2102   diff @@
======================================
  Files          566     564     -2
  Stmts        35937   35496   -441
  Branches      2315    2283    -32
  Methods                          
======================================
- Hit          33551   33178   -373
+ Partial        761     744    -17
+ Missed        1625    1574    -51

Review entire Coverage Diff as of 1a27559

Powered by Codecov. Updated on successful CI builds.

[vweijsters]

Looks good so far. What needs to be done before this is ready for merge?

[vweijsters]

version.ps1 should probably be updated to 1.1.0-dev

[sharwell]

What needs to be done before this is ready for merge?

• It would be good to wait until Visual Studio 2015 Update 2 is available. Otherwise people will not be able to debug this branch inside the IDE.
• I'd like to spend more time testing performance of this branch.

[sharwell]

My initial performance testing did not yield the expected results. For the test, I simply ran the following command using dotnet/roslyn@01ba99a.

StyleCopTester ......\roslyn\Roslyn.sln

Commit	Analysis Time (sec)
7e69f3e (Roslyn 1.0)	173
64d3935 (Roslyn 1.2.0-rc)	244
44bc173 (Roslyn 1.2.0-rc, using new Roslyn APIs)	251
cb0e3af (Roslyn 1.2.1, using new Roslyn APIs)	244

[pdelvo]

Thats weird. @mavasani Are we doing something wrong here or is this a regression in roslyn? Thats 45% slower in 1.2 usi g the new APIs compared to 1.0.

[mavasani]

We have made a bunch of perf fixes post RC. So I would recommend trying out later bits. Additionally, if you can provide ETL trace with RC bits, we can investigate further.

Did you also measure perf with RC bits but not enabling new features? That might give us more context.

[mavasani]

Oh, I see that you did measure with and without new APIs. So seems unrelated to new APIs.

[sharwell]

@mavasani Using 1.2.1 did improve things a bit. Haven't run the profiler yet to see where the remaining problem spots are (still much slower than 1.0.0 was).

[mavasani]

Thanks @sharwell - would also help if you can let me know how to run the test locally (if its via StyleCopTester, I can dig out your old response).

Also adding @basoundr, @srivatsn and @dotnet/roslyn-analysis as an FYI

[sharwell]

@mavasani I'm using StyleCopTester against a specific commit of Roslyn as described above. I run the test twice and take the analysis time from the second run.

[mavasani]

@sharwell do you have numbers against 1.1.0 bits of Roslyn

[mavasani]

@sharwell I notice that StyleCopTester isn't turning on concurrent execution. Can you change this to use the new overload and ensure that the ConcurrentAnalysis is enabled via CompilationWithAnalyzersOptions?

[sharwell]

@mavasani I enabled concurrent execution in cc7deaf, but that just slowed it down from 244 seconds to 254 seconds.

[mavasani]

@sharwell Would it be possible to measure the performance against Roslyn 1.1.0 bits? I am guessing this regression happened with the new IDE analyzer driver added in Update1.

I am going to attempt to repro this locally and confirm the root cause. Meanwhile I have also filed dotnet/roslyn#10530 to track the work on our side.

[sharwell]

I can get those numbers tonight. Essentially it will be a test like commit 64d3935, but using the 1.1 packages instead of 1.2.0-rc.

[sharwell]

@mavasani I ran the test with Roslyn 1.1.1. Some main items to note:

1. Solution load in Roslyn 1.2.1 (OpenSolutionAsync) takes 136% longer than it takes in Roslyn 1.1.1 (as in more than doubled).
2. Analysis time with Roslyn 1.1.1 is 460 seconds. So 1.2.1 is much faster than 1.1.1 but still loses out to 1.0.0.

[mavasani]

Thanks a lot @sharwell

1. Can you file a separate issue for OpenSolutionAsync regression?
2. This basically confirms that the regression happened with 1.1.1, but went unnoticed because neither the IDE diagnostic service nor the batch compiler uses this API for diagnostic computation. The background analysis in the IDE probably takes much longer to compute all diagnostics in Update1/Update2 then in RTM - we are restoring our rolling diagnostics perf tests to avoid such regressions in future.

I am working on dotnet/roslyn#10530 which should improve the analyzer execution time for both the IDE analyzer execution and this public API - I'll measure the analysis time with RTM, Update2 and bits with the fix for this issue and update here.