Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate logging testing #19545

Merged
merged 26 commits into from
Mar 8, 2020
Merged

Migrate logging testing #19545

merged 26 commits into from
Mar 8, 2020

Conversation

JunTaoLuo
Copy link
Contributor

This is part of #19254.

We want to move our LoggedTest infrastructure to aspnetcore. However, this is going to be difficult to make progress on due to namespace collisions and cyclic dependencies (AspNetCore.Testing (in aspnetcore) => Logging.Testing => AspNetCore.Testing (in extensions)). I'm looking into possible ways to solve this.

Note that we might also want to move the [CollectDump] attribute.

Nate McMaster and others added 18 commits November 6, 2018 14:13
* Add attribute for collecting dumps for failed LoggedTest
* Remove obsolete targets, properties, and scripts
* Replace IsProductComponent with IsShipping
* Undo bad merge to version.props
* Update documentation, and put workarounds into a common file
* Replace usages of RepositoryRoot with RepoRoot
* Remove API baselines
* Remove unnecessary restore feeds and split workarounds into two files
* Enable PR checks on all branches, and disable autocancel
Adds our own hook for before/after logic that's more usable, called
`ITestMethodLifecycle`. This provides access to a context object
including the information about the test and the output helper. This can
be implemented by attributes or by the class itself.

The goal (and result) of this, is that we have a single *test executor*
extensibility point that provides all of the features we need. We should
use this everywhere we need features xUnit doesn't have.

Adding a new extensibility point (`ITestMethodLifecycle`) allows us to
do this without turning all of these features into a giant monolith.

---

Also updated our existing extensibility to use this new hook.

I did as much cleanup as a could to remove duplication from logging and
keep it loosly coupled. I didn't want to tease this apart completely
because the scope of this PR is already pretty large.
* Add option to preserve function test logs

* Upload test logs as artifacts

* Preserve binlogs

* Add target to ensure all functional test logs preserved
@JunTaoLuo
Copy link
Contributor Author

FYI this isn't going to work just yet and may need some work in dotnet/extensions and dotnet/runtime before the issues are resolved.

@Pilchie Pilchie added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Mar 4, 2020
@JunTaoLuo JunTaoLuo force-pushed the johluo/migrate-logging-testing branch from ece503f to c6c856e Compare March 5, 2020 02:06
@JunTaoLuo JunTaoLuo force-pushed the johluo/migrate-logging-testing branch from c6c856e to c181547 Compare March 5, 2020 06:10
@JunTaoLuo
Copy link
Contributor Author

This PR is ready for review. The changes here will require dotnet/extensions#3040 to be merged and the new packages from extensions to be ingested here. Note that the ingestion will fail without these changes.

@anurse I think the risk here is that even though I'm able to build and test locally, it's possible that we'll encounter some difficulties on the CI. I think we can work out all the issues in about 1 day. Should I wait for a low traffic time (evening or weekend?) to make this pair of changes?

@JunTaoLuo JunTaoLuo marked this pull request as ready for review March 5, 2020 06:15
@JunTaoLuo
Copy link
Contributor Author

FYI, the only commit that needs review is c181547

@dougbu
Copy link
Member

dougbu commented Mar 5, 2020

FYI, the only commit that needs review is c181547

Why? And why are the validation builds on the floor?

@JunTaoLuo
Copy link
Contributor Author

All other changes are merges from dotnet/extensions which we are bringing over to dotnet/aspnetcore. You can think of it as additional APIs we wanted to migrate from extensions to aspnetcore.

The changes here will require dotnet/extensions#3040 to be merged and the new packages from extensions to be ingested here. Note that the ingestion will fail without these changes.

These changes will not pass until we ingest new extensions packages due to a cyclic dependency.

@JunTaoLuo
Copy link
Contributor Author

@dougbu 🆙 📅

@dougbu
Copy link
Member

dougbu commented Mar 6, 2020

Hmm, I approved before I noticed this wasn't building ☹️

@JunTaoLuo
Copy link
Contributor Author

It won't build until I merge dotnet/extensions#3040 and get a dependency update from extensions

dotnet-maestro bot and others added 2 commits March 7, 2020 21:00
…build 20200307.1

- Microsoft.AspNetCore.Mvc.Razor.Extensions - 5.0.0-preview.3.20157.1
- Microsoft.AspNetCore.Razor.Language - 5.0.0-preview.3.20157.1
- Microsoft.CodeAnalysis.Razor - 5.0.0-preview.3.20157.1
- Microsoft.NET.Sdk.Razor - 5.0.0-preview.3.20157.1

Dependency coherency updates

- Microsoft.AspNetCore.Analyzer.Testing - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Caching.Abstractions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Caching.Memory - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Caching.SqlServer - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Caching.StackExchangeRedis - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.Abstractions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.AzureKeyVault - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.Binder - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.CommandLine - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.EnvironmentVariables - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.FileExtensions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.Ini - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.Json - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.UserSecrets - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration.Xml - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Configuration - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.DependencyInjection.Abstractions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.DependencyInjection - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.DiagnosticAdapter - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.FileProviders.Abstractions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.FileProviders.Composite - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.FileProviders.Physical - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.FileSystemGlobbing - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Hosting.Abstractions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Hosting - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Http - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.Abstractions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.AzureAppServices - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.Configuration - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.Console - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.Debug - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.EventSource - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.EventLog - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.TraceSource - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging.Testing - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Options.ConfigurationExtensions - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Options.DataAnnotations - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Options - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Primitives - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Internal.Extensions.Refs - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Microsoft.Extensions.Logging - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
- Internal.AspNetCore.Analyzers - 5.0.0-preview.3.20156.3 (parent: Microsoft.AspNetCore.Razor.Language)
@JunTaoLuo JunTaoLuo force-pushed the johluo/migrate-logging-testing branch from e061ecf to 54b506e Compare March 8, 2020 01:45
@JunTaoLuo
Copy link
Contributor Author

I need to rework 3 AssemblyTestLogTests (#19683) but I'm merging this to unblock dependency updates.

@JunTaoLuo JunTaoLuo merged commit a8e7ffb into master Mar 8, 2020
@JunTaoLuo JunTaoLuo deleted the johluo/migrate-logging-testing branch March 8, 2020 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants