Revisit default Quality used by BrotliStream for CompressionLevel.Optimal

[adamsitnik]

It has been noted by @stephentoub in #46595 (comment):

The quality values are set as:

        public const int Quality_Min = 0;
        public const int Quality_Default = 11;
        public const int Quality_Max = 11;

With the introduction of SmallestSize (#1549 (comment)) and the rationale employed for it that Optimal should really be a good balance between speed and size, we should define it closer to 6 rather than 11. The docs will also need to be updated.

The person who is willing to work on this issue should:

• choose a set of metrics used to determine which value is optimal. Preferably compressed output size and time it takes to perform the compression.
• choose a set of representative input files for the experiment while keeping in mind that .NET users might use this API for compressing all kinds of files.
• run the experiment for all the files and quality levels, share the results and source code (so others can run the tests for different configs like ARM64).
• send a PR with code change
• send a PR with docs change to https://github.com/dotnet/dotnet-api-docs

Nice to have:

• verify if 22 is the best default value for Window
• see if there are any places in our code base where we are using Brotli for compression and we know the input size up-front. If so, consider using lower quality for small inputs.

The recommended tool for benchmarking the performance is BenchmarkDotNet. Since getting BDN to print some extra metrics like compressed output size requires some hacks, I am providing a link to source code that shows how to do it: adamsitnik/performance@b2e21df...f0c222b

Sample results:

Method	quality	window	file	Mean	Size
Compress_WithoutState	4	21	alice29.txt	2,487,362.9 ns	54467
Decompress_WithoutState	4	21	alice29.txt	392,504.2 ns	54467
Compress_WithoutState	4	22	alice29.txt	2,511,040.1 ns	54467
Decompress_WithoutState	4	22	alice29.txt	391,846.0 ns	54467
Compress_WithoutState	4	23	alice29.txt	2,502,379.3 ns	54467
Decompress_WithoutState	4	23	alice29.txt	392,153.3 ns	54467
Compress_WithoutState	5	21	alice29.txt	4,250,425.3 ns	51989
Decompress_WithoutState	5	21	alice29.txt	378,212.0 ns	51989
Compress_WithoutState	5	22	alice29.txt	4,218,672.9 ns	51989
Decompress_WithoutState	5	22	alice29.txt	377,680.4 ns	51989
Compress_WithoutState	5	23	alice29.txt	4,251,750.8 ns	51989
Decompress_WithoutState	5	23	alice29.txt	377,350.3 ns	51989
Compress_WithoutState	6	21	alice29.txt	5,068,046.1 ns	51186
Decompress_WithoutState	6	21	alice29.txt	368,488.6 ns	51186
Compress_WithoutState	6	22	alice29.txt	5,110,149.7 ns	51186
Decompress_WithoutState	6	22	alice29.txt	371,180.5 ns	51186
Compress_WithoutState	6	23	alice29.txt	5,095,344.6 ns	51186
Decompress_WithoutState	6	23	alice29.txt	371,466.2 ns	51186
Compress_WithoutState	7	21	alice29.txt	7,774,881.0 ns	50700
Decompress_WithoutState	7	21	alice29.txt	365,068.9 ns	50700
Compress_WithoutState	7	22	alice29.txt	7,757,002.4 ns	50700
Decompress_WithoutState	7	22	alice29.txt	365,038.5 ns	50700
Compress_WithoutState	7	23	alice29.txt	7,754,481.9 ns	50700
Decompress_WithoutState	7	23	alice29.txt	364,923.5 ns	50700

Please consider using charts to visualize the results of the experiment. You can do that by using RPlotExporter or by getting BDN to export the data to CSV and importing it in Excel (or other tool of your choice)

[ghost]

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

Issue Details

---

It has been noted by @stephentoub in #46595 (comment):

The quality values are set as:

        public const int Quality_Min = 0;
        public const int Quality_Default = 11;
        public const int Quality_Max = 11;

With the introduction of SmallestSize (#1549 (comment)) and the rationale employed for it that Optimal should really be a good balance between speed and size, we should define it closer to 6 rather than 11. The docs will also need to be updated.

The person who is willing to work on this issue should:

• choose a set of metrics used to determine which value is optimal. Preferably compressed output size and time it takes to perform the compression.
• choose a set of representative input files for the experiment while keeping in mind that .NET users might use this API for compressing all kinds of files.
• run the experiment for all the files and quality levels, share the results and source code (so others can run the tests for different configs like ARM64).
• send a PR with code change
• send a PR with docs change to https://github.com/dotnet/dotnet-api-docs

The recommended tool for benchmarking the performance is BenchmarkDotNet. Since getting BDN to print some extra metrics like compressed output size requires some hacks, I am providing a link to source code that shows how to do it: adamsitnik/performance@b2e21df...f0c222b

Sample results:

Method	quality	window	file	Mean	Size
Compress_WithoutState	4	21	alice29.txt	2,487,362.9 ns	54467
Decompress_WithoutState	4	21	alice29.txt	392,504.2 ns	54467
Compress_WithoutState	4	22	alice29.txt	2,511,040.1 ns	54467
Decompress_WithoutState	4	22	alice29.txt	391,846.0 ns	54467
Compress_WithoutState	4	23	alice29.txt	2,502,379.3 ns	54467
Decompress_WithoutState	4	23	alice29.txt	392,153.3 ns	54467
Compress_WithoutState	5	21	alice29.txt	4,250,425.3 ns	51989
Decompress_WithoutState	5	21	alice29.txt	378,212.0 ns	51989
Compress_WithoutState	5	22	alice29.txt	4,218,672.9 ns	51989
Decompress_WithoutState	5	22	alice29.txt	377,680.4 ns	51989
Compress_WithoutState	5	23	alice29.txt	4,251,750.8 ns	51989
Decompress_WithoutState	5	23	alice29.txt	377,350.3 ns	51989
Compress_WithoutState	6	21	alice29.txt	5,068,046.1 ns	51186
Decompress_WithoutState	6	21	alice29.txt	368,488.6 ns	51186
Compress_WithoutState	6	22	alice29.txt	5,110,149.7 ns	51186
Decompress_WithoutState	6	22	alice29.txt	371,180.5 ns	51186
Compress_WithoutState	6	23	alice29.txt	5,095,344.6 ns	51186
Decompress_WithoutState	6	23	alice29.txt	371,466.2 ns	51186
Compress_WithoutState	7	21	alice29.txt	7,774,881.0 ns	50700
Decompress_WithoutState	7	21	alice29.txt	365,068.9 ns	50700
Compress_WithoutState	7	22	alice29.txt	7,757,002.4 ns	50700
Decompress_WithoutState	7	22	alice29.txt	365,038.5 ns	50700
Compress_WithoutState	7	23	alice29.txt	7,754,481.9 ns	50700
Decompress_WithoutState	7	23	alice29.txt	364,923.5 ns	50700

Please consider using charts to visualize the results of the experiment. You can do that by using RPlotExporter or by getting BDN to export the data to CSV and importing it in Excel (or other tool of your choice)

Author:	adamsitnik
Assignees:	-
Labels:	area-System.IO.Compression, tenet-performance, up-for-grabs
Milestone:	7.0.0

[EgorBo]

I always thought that there was some special secret meaning under Optimal being the same as Max like "Brotli only makes sense to use with -q11" 😄