Custom logging instead of NLog-based one (OSOE-402)

[Piedone]

Currently, the UI Testing Toolbox depends on NLog. In OrchardApplicationFactory we configure NLog for logging, and OrchardCoreInstance.GetLogs() we load the log files. Instead, while retaining support for the app under test to use NLog logging for its own reasons, we could have our custom in-memory logger just for testing. That'd collect log messages about errors and warnings, and we'd use that to check for log entries to fail the test.

Decoupling the project from NLog would also prevent issues like this when Orchard updates its NLog version: OrchardCMS/OrchardCore#11194 (comment)

Related: Lombiq/Open-Source-Orchard-Core-Extensions#264 (comment)

Jira issue

[rikbosch]

Could something like https://github.com/horsdal/InMemoryLogger/blob/master/src/InMemoryLogger.cs be a solution?

[Piedone]

Yep, looks suitable. Through InMemoryLogger.RecordedLogs we could also access the log entries.

In any case, the FailureDump should still contain a full log file.

[rikbosch]

What about using OpenTelemetry for capturing logs / traces and metrics of a specific failed test?

[Piedone]

That could be useful for tracing tests in general, though well beyond the scope of this issue, which is about substituting the ILogger implementation.

See also OrchardCMS/OrchardCore#12251.

[Piedone]

Perhaps the best would be to use FakeLogger. Then FakeLogCollector can be used to retrieve the log messages.

[Piedone]

This would also allow us to better filter log messages. Thus, we wouldn't log ErrorController's exception, to begin with, instead of filtering it out during assertion.

[Piedone]

It's important that we need to log URLs too, since this greatly improved troubleshooting from a failure dump.

[Piedone]

For the record:

FakeLogger is really simple, and it can only provide very rudimentary filtering based on log levels (or the type that logs, but that's only an allow list, i.e. we can only configure it to log for specific classes). The log level can't be set per type, nor can there be any custom filtering. But it works.

InMemoryLogger is unmaintained and also very rudimentary.

Logging.Memory has custom filtering (though this seems only for category and level), but is unmaintained, with 3 PRs being open that address issues all important for this use case.

Finally, `Meziantou.Extensions.Logging.InMemory is also very rudimentary.