Logging ordering fixes and feedback

[JamesNK]

Description

Following feedback from the previous PR and discussion with Karol about what options we have for getting logs from DCP, it's clear that there isn't a robust, fast, simple way to stream correctly ordered console logs from the host. The only way to produce the correct order is to do a final sort on the client.

This PR:

• Removes flaky time based delay from the server. It isn't possible to perfectly order log lines before they are returned from the service. Don't attempt to delay sending while ordering.

• Shares code for how log entries are internally represented on the client (dashboard) and server (host). Parsing and sorting logic is shared to ensure the backlog and client results match in content and order.

• Improves how sorting is implemented. Log lines with dates are placed in sorted order, while those without dates are placed in the order they were received. This mades ordering somewhat complicated, but there are tests, and I think it's important for the dashboard to be resilient to receiving whatever data it gets given as logs. In saying that, I want to make sure DCP always includes log lines with a prefixed date.

• Changes ResourceLoggerService to use one lock. I didn't see the benefit of multiple. It makes locking easier to reason with and reduces the chance of accidential deadlock.

• Resource logger state backlog is cleared immediately when there are no longer any subscribers. Previously they were cleared via a notificated sent through a number of channels. A race condition could cause logs to be watched again before they were cleared. That would cause logs to be double returned: the backlog + all logs from the start again.

Fixes #4893

Checklist

• Is this feature complete?
  • Yes. Ready to ship.
  • No. Follow-up changes expected.
• Are you including unit tests for the changes and scenario tests if relevant?
  • Yes
  • No
• Did you add public API?
  • Yes
    • If yes, did you have an API Review for it?
      • Yes
      • No
    • Did you add <remarks /> and <code /> elements on your triple slash comments?
      • Yes
      • No
  • No
• Does the change make any security assumptions or guarantees?
  • Yes
    • If yes, have you done a threat model and had a security review?
      • Yes
      • No
  • No
• Does the change require an update in our Aspire docs?
  • Yes
    • Link to aspire-docs issue:
  • No

Microsoft Reviewers: Open in CodeFlow

[eerhardt]

Why do we parse the timestamp out here, just to turn around and put them back together in the ResourceLoggerService? Why not just leave the timestamp in the content, and pass it all the way up to the dashboard. And only need to parse it out once in the UI?

[JamesNK]

I did this so the content in the host LogEntry was consistent with the client LogEntry.

[eerhardt]

Why do we need them to be consistent? The host can just deal with the logs as lines of content.

it’s only the UI that wants them to be sorted by date time, so why would we force the host to parse the time out?

[JamesNK]

They don't need to be consistent; I just think it's simpler to reason about everything when it's consistent and it allows more code to be shared between the host and dashboard.

In the host, either the ILogger logs need to add a timestamp to log line content or DCP logs need to remove a timestamp from log line content. Either way, it's slightly weird.

I chose to make the DCP logs remove the time stamp over the alternative for the reasons above. The downside there are some extra allocations as logs are received and sent by the host, the upside is storing the backlog uses less memory. DateTime is smaller than vs 15ish UTF16 chars in the log message.