Perf: create a tracker per module and use indexes to improve performance #181
Conversation
|
Side note: this dosen't solve instrumentation of System.Private.CoreLib, but gets a path to a possible fix: when instrumenting corelib/mscorlib instead of generating the tracker code as
we need to generate
|
|
Testing this PR against one of the bigger unit test projects in the code I'm working on at the day job the performance improvements are very good:
Impressive, @pjanotti! |
| @@ -14,8 +14,9 @@ namespace coverlet.core.performancetest | |||
| /// </summary> | |||
| public class PerformanceTest | |||
| { | |||
| [Theory(Skip = "Only enabled when explicitly testing performance.")] | |||
| [InlineData(150)] | |||
| [Theory(/*Skip = "Only enabled when explicitly testing performance."*/)] | |||
There was a problem hiding this comment.
Should the test be enabled by default now, or remain Skip in the repo code?
There was a problem hiding this comment.
The perf test is not run by default so we don't need the skip in place (avoids us changing the source just to test perf). That said I should have removed the skip instead of commenting it, I'll clean it up on my next change.
| foreach (TypeDefinition type in types) | ||
| { | ||
| var actualType = type.DeclaringType ?? type; | ||
| if (!actualType.CustomAttributes.Any(IsExcludeAttribute) | ||
| && actualType.Namespace != "Coverlet.Core.Instrumentation.Tracker" |
There was a problem hiding this comment.
Why not add an ExcludeFromCodeCoverage attribute to all the methods in that class?
There was a problem hiding this comment.
I see that you already removed it 👍 Thanks, it was not needed as you noticed
…er.module.rebase Perf: create a tracker per module and use indexes to improve performance
This change came from the discusson on PR #172.
This change brings the performance for System.Text.Encondings.Web.Tests in CoreFx from around 113 seconds to around 19 seconds. The coverlet performance test project now runs for 20,000 iterations in about the same time that it previously was running for 200 iterations, now at 200 iterations the time is dominated by instrumentation and report generation.
The idea of the change is that by creating a distinct static tracker type per module the dictionaries used before to track execution can be replaced with an array to represent the hit locations. This array is written to the disk in binary format, only the number of hits per index, avoiding the need to parse to process the hit files. The trade-off is that this should represent a bit more consumption of memory for projects with very low coverage numbers.