Error reading from large streams using YamlStream / Parser

[whatatripp]

This is related to #407 but with a very simple reproduction case. I just briefly toured the code, but I suspect the error is an off-by-one error (or something) in the LookAheadBuffer?

Note that, depending the stream reader's underlying source, it'll work or not work:.

• responseStream and new BufferedStream(responseStream, 64 * 1024) fail with a gives a duplicate key error
• new StreamReader(responseStream).ReadToEnd() works fine

This is for .NET 452, though I doubt it would matter.

using System;
using System.IO;
using System.Net;
using YamlDotNet.RepresentationModel;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls12;

            for (int i = 0; i < 5; i++)
            {
                Console.WriteLine("Attempt #" + i);
                
                var stream = new YamlStream(); 
                var request = WebRequest.CreateHttp("https://helm.elastic.co/index.yaml");
                using (var response = (HttpWebResponse)request.GetResponse())
                using (var responseStream = response.GetResponseStream())
                {
                    //Fails
                    var index = new StreamReader(new BufferedStream(responseStream, 64 * 1024));
                    //var index = new StreamReader(responseStream);

                    // Works
                    //var index = new StringReader(new StreamReader(responseStream).ReadToEnd());
                    stream.Load(index);
                }                
            }            
        }
    }
}

[aaubry]

Thanks for the repro. I'll look into it.

[aaubry]

I managed to find the cause of the error. Thanks to your repro it was easy.

The code that read from the stream was assuming that it would always get the amount of bytes that it requested unless the end of file was reached. In the case of network streams, that's not the case. I'm surprised that this bug did not come up sooner. I guess that most people parse from files, and that the implementation happens to always return the requested amount of bytes.

Fixing the issue was just a matter of adding a loop to ensure that we read as much as we need.

I'll publish a release now so that you can try the fix.

[aaubry]

I have published a pre-release version of the package that should fix aother related error. You may prefer to test with that version?
The package is unlisted, so you will need to set the version explicitly:

Install-Package YamlDotNet -Version 9.1.4-fix-issues-553-562-0001

Thanks.

[EdwardCooke]

I just tested this with the latest library using the sample code provided and could not reproduce it, since aaubry mentioned he already fixed it and no response in the past year and a half, I'm going to close this issue.