RegexOptions.Multiline should be same as line by line

[VS-ux]

Using RegexOptions.Multiline should be the same as reading a file line by line I think.

When reading an entire file, and the Regexing it with RegexOptions.Multiline should be same as reading it line by line
Here is a code sample:

`
string path = @"PathToFile";
using (StreamReader reader = new StreamReader(path))
{
//string s = reader.ReadToEnd();
//int matchCount = Regex.Matches(s, @"^(.+)/([^/]+)$", RegexOptions.Compiled | RegexOptions.Multiline).Count; //I think this should be the same as below:

int matchCount = 0;
string s;
while ((s = reader.ReadLine()) != null)
{
    matchCount += Regex.Matches(s, @"^(.+)\/([^/]+)$", RegexOptions.Compiled).Count;
}

Console.WriteLine(matchCount);

}
`

This is probably related to #25598
This may be I'm just completely doing it wrong, and I apologize if this is very stupid.

[ghost]

Tagging subscribers to this area: @eerhardt, @pgovind
See info in area-owners.md if you want to be subscribed.

[danmoseley]

What are the line endings in your test file? \n or \r\n? If the latter, in the first code snippet above, the $ will match the \n after each \r (since we don’t have the AnyNewLine setting yet). This means your pattern will capture the \r. In the second snippet, the reader removes all \r and \n so it would not match the \r. However I wouldn’t expect your pattern to match a different number of times in this case.

What result do you see? Can your share a test file?

[VS-ux]

@danmosemsft Thank you for your reply!
My results were with reading the entire file at once, and using RegexOptions.Multiline | RegexOptions.Compiled I got 125797 matches. Reading the same file line by line with just RegexOptions.Compiled I got 176948 matches.

The test file I used is actually a random binary file, but the results can be seen in a simple test case scenario:

const string regex = @"^(?:[a-zA-Z]\:|\\\\[\w\.]+\\[\w.$]+)\\(?:[\w]+\\)*\w([\w.])+$";
string s = "C:\\users\\user\\test.txt\r\nC:\\user2\\test2.txt";
int matchCount = Regex.Matches(s, regex, RegexOptions.Compiled | RegexOptions.Multiline).Count;

int matchCountLineByLine = Regex.Matches("C:\\users\\user\\test.txt", regex, RegexOptions.Compiled).Count;
matchCountLineByLine += Regex.Matches("C:\\users\\user2\\test2.txt", regex, RegexOptions.Compiled).Count;

Console.WriteLine(matchCount); // Results in 1. If you swap to \n, we get 2
Console.WriteLine(matchCountLineByLine); // Results in 2

In the "matchCountLineByLine" section, I try to simulate reading the text file line by line, which contains 2 file paths.

[VS-ux]

Hi! OK, so I uploaded a System.Memory.dll binary test file that I did some tests on.
Here is the repo:

Here is the code:

using StreamReader reader = new StreamReader(@"PathTo\System.Memory.dll");

string s = reader.ReadToEnd();
int matchCount = Regex.Matches(s, @"^(.+)\/([^/]+)$", RegexOptions.Compiled | RegexOptions.Multiline).Count;

//int matchCount = 0;
//string s;
//int lineCount = 1;
//while ((s = reader.ReadLine()) != null)
//{
//    matchCount += Regex.Matches(s, @"^(.+)\/([^/]+)$", RegexOptions.Compiled | RegexOptions.Singleline).Count;
//    writer.WriteLine(lineCount.ToString());
//    lineCount++;
//}

Console.WriteLine(matchCount);

For the code the is NOT commented, I get 148 results. For code that is commented (Reading line by line), I get 166 matches.

[danmoseley]

I suggest you creates a fixed test file then progressively reduce your test file until it is as small as possible yet still repros the problem and the reason will become clear. You may need a hex editor to see line endings.

[VS-ux]

@danmosemsft I think the reason why is in binary file like System.Memory.dll there are some lone \r line endings without \r\n. It then is unable to match it.

[danmoseley]

Aha yes! In StreamReader i see it will match a lone \r. That is unusual in real text, I believe it was used by old iMacs. It will not be matched by the ^ and $ (again, until we have AnyNewLine)

I think this problem is explained, can we close the issue now?

[VS-ux]

@danmosemsft Yes sure.