Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tests] Move more tests to helix by overriding dcp path from assembly metadata #5215

Merged
merged 21 commits into from
Aug 8, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
9b54a02
DcpOptions: Add support for overriding dcp path via environment varia…
radical Aug 7, 2024
882c954
[ci] Send dcp as helix payload, and use for running tests
radical Aug 7, 2024
23e182e
[tests] Run more tests on helix
radical Aug 7, 2024
73a5b44
rename ASPIRE_DCP_DIR to DOTNET_ASPIRE_DCP_DIR
radical Aug 7, 2024
1488996
Address review feedback from @ eerhardt
radical Aug 7, 2024
7c32dd6
Address review feedback from @ eerhardt - use configuration to access…
radical Aug 7, 2024
2ad24b1
Merge remote-tracking branch 'origin/main' into patch-dcp-path-for-helix
radical Aug 7, 2024
b5517cf
Fix dcp path set for helix
radical Aug 7, 2024
7b0c277
Address review feedback from @ eerhardt - make RunTestsOnHelix defaul…
radical Aug 7, 2024
4b842ea
Cleanup computing the paths
radical Aug 7, 2024
f847361
[tests] Don't try to run test app projects on helix
radical Aug 8, 2024
2bb5753
Merge remote-tracking branch 'origin/main' into patch-dcp-path-for-helix
radical Aug 8, 2024
63f1d69
Always emit the test nuget package versions for helix, in the generated
radical Aug 6, 2024
63baf92
fix e2e tests
radical Aug 8, 2024
4b19cc9
Address review feedback from @ eerhardt, and don't remove the existin…
radical Aug 8, 2024
75d1924
[ci] download helix test result trx files
radical Aug 8, 2024
9e19a0c
[ci] Gather the helix result trx files so they can be published to azdo
radical Aug 8, 2024
ddba7ec
collect all the trx files in downloadable artifacts
radical Aug 8, 2024
bd6df92
avoid double publishing helix test results
radical Aug 8, 2024
357a0b2
Don't archive test support projects like Aspire.Components.Common.Tests
radical Aug 8, 2024
1a14351
Fix Milvus and Qdrant tests to run on Helix.
eerhardt Aug 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 2 additions & 10 deletions src/Aspire.Hosting.AppHost/build/Aspire.Hosting.AppHost.targets
Original file line number Diff line number Diff line change
Expand Up @@ -198,16 +198,8 @@ namespace Projects%3B

<ItemGroup>
<AssemblyAttribute Include="System.Reflection.AssemblyMetadata">
<_Parameter1>dcpclipath</_Parameter1>
<_Parameter2>$(DcpCliPath)</_Parameter2>
</AssemblyAttribute>
<AssemblyAttribute Include="System.Reflection.AssemblyMetadata">
<_Parameter1>dcpextensionpaths</_Parameter1>
<_Parameter2>$(DcpExtensionsDir)</_Parameter2>
</AssemblyAttribute>
<AssemblyAttribute Include="System.Reflection.AssemblyMetadata">
<_Parameter1>dcpbinpath</_Parameter1>
<_Parameter2>$(DcpBinDir)</_Parameter2>
<_Parameter1>dcpdir</_Parameter1>
<_Parameter2>$(DcpDir)</_Parameter2>
</AssemblyAttribute>
<AssemblyAttribute Include="System.Reflection.AssemblyMetadata">
<_Parameter1>apphostprojectpath</_Parameter1>
Expand Down
27 changes: 19 additions & 8 deletions src/Aspire.Hosting/Dcp/DcpOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,7 @@ internal class ConfigureDefaultDcpOptions(
DistributedApplicationOptions appOptions,
IConfiguration configuration) : IConfigureOptions<DcpOptions>
{
private const string DcpCliPathMetadataKey = "DcpCliPath";
private const string DcpExtensionsPathMetadataKey = "DcpExtensionsPath";
private const string DcpBinPathMetadataKey = "DcpBinPath";
private const string DcpDirMetadataKey = "DcpDir";
private const string DashboardPathMetadataKey = "aspiredashboardpath";

public static string DcpPublisher = nameof(DcpPublisher);
Expand All @@ -113,16 +111,22 @@ public void Configure(DcpOptions options)
var dcpPublisherConfiguration = configuration.GetSection(DcpPublisher);
var assemblyMetadata = appOptions.Assembly?.GetCustomAttributes<AssemblyMetadataAttribute>();

if (!string.IsNullOrEmpty(dcpPublisherConfiguration[nameof(options.CliPath)]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any backwards compatibility concerns we need to maintain here? For example, if someone used an 8.1 Aspire.Hosting.AppHost package and an 8.2 Aspire.Hosting package, I think this would break. Are we concerned about that level of mismatch?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One option would be to retain the 3 paths in the metadata as-is, but when DcpPublisher__CliPath is set then override the 3 paths based on the cliPath.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made the change.

// Find dcp tools path in the following order:
// 1. Environment variable ASPIRE_DCP_DIR
// 2. Configuration value
// 3. Assembly metadata
if (Environment.GetEnvironmentVariable("ASPIRE_DCP_DIR") is var dcpDirEnvVar && !string.IsNullOrEmpty(dcpDirEnvVar))
{
SetDcpPathsFromDcpDir(dcpDirEnvVar);
}
else if (!string.IsNullOrEmpty(dcpPublisherConfiguration[nameof(options.CliPath)]))
{
// If an explicit path to DCP was provided from configuration, don't try to resolve via assembly attributes
options.CliPath = dcpPublisherConfiguration[nameof(options.CliPath)];
}
else
else if (GetMetadataValue(assemblyMetadata, DcpDirMetadataKey) is var dcpDir && !string.IsNullOrEmpty(dcpDir))
{
options.CliPath = GetMetadataValue(assemblyMetadata, DcpCliPathMetadataKey);
options.ExtensionsPath = GetMetadataValue(assemblyMetadata, DcpExtensionsPathMetadataKey);
options.BinPath = GetMetadataValue(assemblyMetadata, DcpBinPathMetadataKey);
SetDcpPathsFromDcpDir(dcpDir);
}

if (!string.IsNullOrEmpty(dcpPublisherConfiguration[nameof(options.DashboardPath)]))
Expand Down Expand Up @@ -171,6 +175,13 @@ public void Configure(DcpOptions options)
options.DeleteResourcesOnShutdown = dcpPublisherConfiguration.GetValue(nameof(options.DeleteResourcesOnShutdown), options.DeleteResourcesOnShutdown);
options.RandomizePorts = dcpPublisherConfiguration.GetValue(nameof(options.RandomizePorts), options.RandomizePorts);
options.ServiceStartupWatchTimeout = configuration.GetValue("DOTNET_ASPIRE_SERVICE_STARTUP_WATCH_TIMEOUT", options.ServiceStartupWatchTimeout);

void SetDcpPathsFromDcpDir(string dcpDir)
{
options.CliPath = Path.Combine(dcpDir, "dcp") + (OperatingSystem.IsWindows() ? ".exe" : "");
options.ExtensionsPath = Path.Combine(dcpDir, "ext");
options.BinPath = Path.Combine(options.ExtensionsPath, "bin");
}
}

private static string? GetMetadataValue(IEnumerable<AssemblyMetadataAttribute>? assemblyMetadata, string key)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,12 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
<None Include="$(RepoRoot)src\Components\Aspire.Azure.Search.Documents\ConfigurationSchema.json" CopyToOutputDirectory="PreserveNewest" />

<ProjectReference Include="..\..\src\Components\Aspire.Azure.Search.Documents\Aspire.Azure.Search.Documents.csproj" />
<ProjectReference Include="..\Aspire.Components.Common.Tests\Aspire.Components.Common.Tests.csproj" />
</ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,12 @@
CS8002: Referenced assembly does not have a strong name
-->
<NoWarn>$(NoWarn);CS8002</NoWarn>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
<ProjectReference Include="..\..\src\Aspire.Dashboard\Aspire.Dashboard.csproj" />

<PackageReference Include="bUnit" />
</ItemGroup>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>false</RunTestsOnHelix>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>false</RunTestsOnHelix>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<NoWarn>$(NoWarn);CS8002</NoWarn> <!-- Milvus.Client packages are not signed -->
<!-- Cannot be moved to helix as this uses TestProject -->
<!--<RunTestsOnHelix>true</RunTestsOnHelix>-->
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<NoWarn>$(NoWarn);CS8002</NoWarn> <!-- MongoDB packages are not signed -->
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<!-- Cannot be moved to helix as this uses TestProject -->
<!--<RunTestsOnHelix>true</RunTestsOnHelix>-->
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

<PropertyGroup>
<TargetFramework>$(NetCurrent)</TargetFramework>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,14 @@
<TargetFramework>$(NetCurrent)</TargetFramework>
<PreserveCompilationContext>true</PreserveCompilationContext>
<CopyDocumentationFilesFromPackages>true</CopyDocumentationFilesFromPackages>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
<Compile Include="..\..\src\Components\Common\ConfigurationSchemaAttributes.cs" Link="ConfigurationSchemaAttributes.cs">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Compile>

<Content Include="Baselines\**\*;">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
<TargetFramework>$(NetCurrent)</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
<TargetFramework>$(NetCurrent)</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
<TargetFramework>$(NetCurrent)</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<RunTestsOnHelix>true</RunTestsOnHelix>
</PropertyGroup>

<ItemGroup>
Expand Down
6 changes: 6 additions & 0 deletions tests/helix/send-to-helix-basictests.targets
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,15 @@
<PropertyGroup>
<WorkItemArchiveWildCard>$(TestArchiveTestsDir)**/*.zip</WorkItemArchiveWildCard>
<BuildHelixWorkItemsDependsOn>$(BuildHelixWorkItemsDependsOn);BuildHelixWorkItemsForDefaultTests</BuildHelixWorkItemsDependsOn>
<NeedsDcpPathOverride>true</NeedsDcpPathOverride>
</PropertyGroup>

<Target Name="BuildHelixWorkItemsForDefaultTests">
<ItemGroup>
<!-- needed for Aspire.Hosting.Container.Tests -->
<HelixPreCommand Include="$(_EnvVarSetKeyword) DOCKER_BUILDKIT=1" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to put this in the Hosting.Container.Tests so it doesn't need to be set everywhere?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would that be useful? Right now it would have to be by conditionally adding it on the work item itself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking is that we wouldn't need a comment like <!-- needed for Aspire.Hosting.Container.Tests --> if it was in the Aspire.Hosting.Container.Tests project.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. In the follow up I'm generating run scripts which would allow individual test projects to control their own setups directly.

</ItemGroup>

<PropertyGroup>
<_TestRunCommand Condition="'$(RunWithCodeCoverage)' == 'true'">@(_TestCoverageCommand, ' ') &quot;@(_TestRunCommandArguments, ' ')&quot;</_TestRunCommand>
<_TestRunCommand Condition="'$(RunWithCodeCoverage)' != 'true'">@(_TestRunCommandArguments, ' ')</_TestRunCommand>
Expand Down
Loading