Skip to content

Commit

Permalink
Merge fsharp47 + dev16.3 into Dim interop (#7)
Browse files Browse the repository at this point in the history
* Copy sources from Versions.props to NuGet.config (dotnet#7191)

* Only check distinct errors (dotnet#7140)

* Use 1-based column numbers in tests (dotnet#7141)

* Use 1-based column numbers in tests

* Helper that can check for multiple type errors

* Update dependencies from https://github.com/dotnet/arcade build 20190710.8 (dotnet#7200)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19360.8

* Better record and value formatting in tools (dotnet#7021)

* Remove semicolons from record tooltips

* Update to put a space between braces

* Update formatting as best I can, plus some tests I guess

* More baseline updates

* Anonymous records

* Update anon records tests

* Add vsbsl lol

* Update baselines and reduce a simple filter

* Update baselines maybe last time

* Update fsharpqa test

* make tests pass

* Add formatting for values

* Update tests

* Update test

* Update fsharpqa tests

* tryit

* lol

* get yote

* shlerp

* Update tests again I guess

* more update

* mother of pearl

* this is a real turd

* fix portable PDBs for anon records (dotnet#7099)

* fix portable PDBs for anon records (dotnet#7099)

* Moving ElseBranchHasWrongTypeTests over to NUnit (dotnet#7104)

* Port tests for missing else branch to NUnit (dotnet#7209)

* Update dependencies from https://github.com/dotnet/arcade build 20190711.7 (dotnet#7216)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19361.7

* Update dependencies from https://github.com/dotnet/arcade build 20190712.5

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19362.5

* Fix Supplementary Character / Surrogate Pair info (no code changes) (dotnet#7221)

Terminology and info regarding Supplementary Characters / Surrogate Pairs is either incorrect, or at least incomplete (which then leads to incorrect statements and/or code).

1. Introduce the term "Supplementary Character" since that is often what we are dealing with, not "surrogate pairs" since that is an encoding-specific concept (UTF-16 only).

2. Add comment re: Supplementary Character code point range, which helps to explain the `elif high > 0x10 then Invalid` condition (line 173).

3. Fix URI for Unicode Standard PDF, Chapter 3, and specify the name of the section (i.e. "Surrogates") instead of the section number (i.e. 3.8) since the section number was 3.7 but is now 3.8 (line 174).

4. Add comment for definition of a valid "surrogate pair" because why make the reader guess or have to go look it up when it will never change? (line 175)

5. Correct and expand comment with example long Unicode escape sequence (line 64): `"\UDEADBEEF"` is _not_ a valid escape sequence. Usage of the `\U` escape has been misstated from the very beginning, both in this documentation as well as the C# Specification documentation, and the language references for "String" for both F# and C#:
    1. `\U` is used to specify a Unicode code point (or UTF-32 code unit, which maps 1:1 with all Unicode code points, hence they are synonymous), not surrogate pairs. Hence the valid range is `00000000` - `0010FFFF`, hence the first two digits are static `0`s, and the third digit can only ever be a `0` or `1`. This escape sequence can specify either a BMP character or a Supplementary character. Supplementary characters are then encoded as a surrogate pair in UTF-16 only, not in UTF-8 or UTF-32. If you want to specify an actual surrogate pair, then use the `\u` escape, e.g. `\uD83D\uDC7D` == `\U0001F47D`.
    2. Even if you could specify a surrogate pair using `\U`, "DEADBEEF" is not valid. U+DEAD is a valid surrogate, _but_ it's a low surrogate code point and cannot be specified first in the pair (meaning, at best one could use `\UxxxxDEAD`). Also, U+BEEF is _not_ a valid surrogate code point, high or low. Surrogate code points are in the range of U+D800 to U+DFFF.

For more info, please see:

https://sqlquantumleap.com/2019/06/26/unicode-escape-sequences-across-various-languages-and-platforms-including-supplementary-characters/#fsharp

* Update IlxGen.fs (dotnet#7227)

* Check for exit code in compiler tests (dotnet#7211)

* Moving AccessOfTypeAbbreviationTests over to NUnit (dotnet#7226)

* Moving AccessOfTypeAbbreviationTests over to NUnit

* ha! now I know what this `1` means =)

* Error range updated and removed `exit 0`

* Error message prefixed by "This construct is deprecated."

* Error messages changed based on current state of FSI

* Moving ConstructorTests over to NUnit (dotnet#7236)

* [master] Update dependencies from dotnet/arcade (dotnet#7233)

* Update dependencies from https://github.com/dotnet/arcade build 20190713.1

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19363.1

* Update dependencies from https://github.com/dotnet/arcade build 20190714.1

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19364.1

* Update dependencies from https://github.com/dotnet/arcade build 20190715.4 (dotnet#7240)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19365.4

* Moving UpcastDowncastTests over to NUnit (dotnet#7229)

* Moving UpcastDowncastTests over to NUnit

* missing new line

* Update dependencies from https://github.com/dotnet/arcade build 20190716.4 (dotnet#7245)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19366.4

* Move ErrorMessages/NameResolution Tests to NUnit (dotnet#7237)

* Mark name resoltion error message tests for port to NUnit

* Add Nunit tests for FieldNotInRecord, RecordFieldProposal, and GlobalQualifierAfterDot

* Fix expected error message in FieldNotInRecord Compiler test

* Change global Qualifier after dot test back to original fsharpqa version.
Needs seperate assert function for parsing errors instead of typechecking

* Remove unnecessary double ticks from NameResolution tests

* Moving WarnExpressionTests over to NUnit (dotnet#7232)

* move some error and warning tests to NUnit (dotnet#7244)

* move some error and warning tests to NUnit

* CompilerAssert.ParseWithErrors now uses ParseFile instead of ParseAndCheckFileInProject

* merge conflicts

* Update dependencies from https://github.com/dotnet/arcade build 20190717.8

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19367.8

* Move UnitGenericAbstractType To Nunit (dotnet#7257)

* Update dependencies from https://github.com/dotnet/arcade build 20190718.7 (dotnet#7256)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19368.7

* Moving TypeMismatchTests over to NUnit (dotnet#7250)

* Moving TypeMismatchTests over to NUnit

* ci restart

* publish pdbs in FSharp.Core.nupkg (dotnet#7255)

Also publish native symbols so they can be archived later.

* Enable hash algorithm selection (dotnet#7252)

* Enable hash algorithm selection

* Feedback

* More feedback

* Revert "Feedback"

This reverts commit 6ab1b07.

* feedback

* Update dependencies from https://github.com/dotnet/arcade build 20190719.2 (dotnet#7260)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19369.2

* Improve netcore reference selection (dotnet#7263)

* Improve netcore reference selection

* Update baselines

* Moving Libraries Control tests to NUnit (dotnet#7234)

* Moving Libraries Control tests to NUnit

* Names tests as Async instead of Control

* Member constraints and PrimitiveConstraints (dotnet#7210)

* Update dependencies from https://github.com/dotnet/arcade build 20190722.10 (dotnet#7268)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19372.10

* fixes issue dotnet#6832 (dotnet#7259)

* fixes issue dotnet#6832

* Fix comment

* Forgot to remove pdbs from fsharp.compiler.nuget

* Rename Program.fs and exclude FSharpSdk from checks

* Embedded doesn't need to verify tmp or obj directories.

* typo

* typo

* Don't check FSharpSdk for hash

* Make comment match code

* Empty commit to force rebuild

* Color nameof as intrinsic (dotnet#7273)

* [master] Update dependencies from dotnet/arcade (dotnet#7269)

* Update dependencies from https://github.com/dotnet/arcade build 20190723.6

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19373.6

* Update dependencies from https://github.com/dotnet/arcade build 20190724.2

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19374.2

* Update dependencies from https://github.com/dotnet/arcade build 20190725.2

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19375.2

* Update dependencies from https://github.com/dotnet/arcade build 20190725.15 (dotnet#7282)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19375.15

* Fix test assert (dotnet#7283)

* disablewarningtests

* code cleanup prior to optional interop improvements (dotnet#7276)

* add test cases we need to make work

* cleanup method call argument processing

* Update dependencies from https://github.com/dotnet/arcade build 20190726.18 (dotnet#7285)

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19376.18

* Moving ClassesTests over to NUnit (dotnet#7264)

* Moving ClassesTests over to NUnit

* Fixed assertion for ParseWithErrors test

* Move Basic Constants to NUnit (dotnet#7262)

* Move BasicConstants.fs Tests To Nunit

* fix typo

* Another typo

* Moved  Don't Suggest Tests over to NUnit (dotnet#7288)

* [master] Update dependencies from dotnet/arcade (dotnet#7287)

* Update dependencies from https://github.com/dotnet/arcade build 20190727.2

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19377.2

* Update dependencies from https://github.com/dotnet/arcade build 20190728.1

- Microsoft.DotNet.Arcade.Sdk - 1.0.0-beta.19378.1

* Fix langversion with multiple projects (dotnet#7293)

* Resolve merge issues, and take care of Test Framework incompatabilities

* Tweak tests
  • Loading branch information
KevinRansom authored and TIHan committed Jul 31, 2019
1 parent 510cb0b commit 71194c7
Show file tree
Hide file tree
Showing 220 changed files with 5,307 additions and 3,370 deletions.
8 changes: 8 additions & 0 deletions .vsts-signed.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,14 @@ jobs:
continueOnError: true
condition: succeeded()

# Publish native PDBs for archiving
- task: PublishBuildArtifacts@1
displayName: Publish Artifact Symbols
inputs:
PathtoPublish: '$(Build.SourcesDirectory)/artifacts/SymStore/$(BuildConfiguration)'
ArtifactName: NativeSymbols
condition: succeeded()

# Execute cleanup tasks
- task: ms-vseng.MicroBuildTasks.521a94ea-9e68-468a-8167-6dcf361ea776.MicroBuildCleanup@1
displayName: Execute cleanup tasks
Expand Down
3 changes: 2 additions & 1 deletion FSharpBuild.Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@
<RepositoryUrl Condition="'$(RepositoryUrl)' == ''">https://github.com/Microsoft/visualfsharp</RepositoryUrl>
<RepositoryType Condition="'$(RepositoryType)' == ''">git</RepositoryType>
</PropertyGroup>

<PropertyGroup Condition="'$(FSharpSourceBuild)' == 'true' AND '$(RepositoryCommit)' == ''">
<_DotGitDir>$(RepoRoot).git</_DotGitDir>
<_HeadFileContent Condition="Exists('$(_DotGitDir)/HEAD')">$([System.IO.File]::ReadAllText('$(_DotGitDir)/HEAD').Trim())</_HeadFileContent>
Expand All @@ -87,7 +88,7 @@
<PropertyGroup>
<NoWarn Condition="'$(Language)' == 'F#'">$(NoWarn);FS2003</NoWarn><!-- warning when AssemblyInformationalVersion looks like '1.2.3-dev' -->
<NoCompilerStandardLib>true</NoCompilerStandardLib><!-- necessary for resource generation using csc.exe -->
<DebugType>portable</DebugType>
<DebugType>embedded</DebugType>
<MicroBuildAssemblyFileLanguage>fs</MicroBuildAssemblyFileLanguage>
<UseStandardResourceNames>false</UseStandardResourceNames>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
Expand Down
22 changes: 22 additions & 0 deletions NuGet.config
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,28 @@
<clear />
<add key="dotnet-core" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
<add key="fsharp-daily" value="https://www.myget.org/F/fsharp-daily/api/v3/index.json" />
<add key="roslyn-master-nightly" value="https://dotnet.myget.org/F/roslyn-master-nightly/api/v3/index.json" />
<add key="myget-dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json" />
<add key="dotnet-buildtools" value="https://dotnet.myget.org/F/dotnet-buildtools/api/v3/index.json" />
<add key="roslyn-tools" value="https://dotnet.myget.org/F/roslyn-tools/api/v3/index.json" />
<add key="roslyn" value="https://dotnet.myget.org/F/roslyn/api/v3/index.json" />
<add key="symreader-converter" value="https://dotnet.myget.org/F/symreader-converter/api/v3/index.json" />
<add key="interactive-window" value="https://dotnet.myget.org/F/interactive-window/api/v3/index.json" />
<add key="vs-devcore" value="https://myget.org/F/vs-devcore/api/v3/index.json" />
<add key="vs-editor" value="https://myget.org/F/vs-editor/api/v3/index.json" />
<add key="vssdk" value="https://vside.myget.org/F/vssdk/api/v3/index.json" />
<add key="vs-impl" value="https://vside.myget.org/F/vs-impl/api/v3/index.json" />
<add key="roslyn_concord" value="https://myget.org/F/roslyn_concord/api/v3/index.json" />
<add key="devcore" value="https://vside.myget.org/F/devcore/api/v3/index.json" />

<add key="dotnet-windowsdesktop" value="https://dotnetfeed.blob.core.windows.net/dotnet-windowsdesktop/index.json" />
<add key="aspnet-aspnetcore" value="https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json" />
<add key="aspnet-aspnetcore-tooling" value="https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore-tooling/index.json" />
<add key="aspnet-entityframeworkcore" value="https://dotnetfeed.blob.core.windows.net/aspnet-entityframeworkcore/index.json" />
<add key="aspnet-extensions" value="https://dotnetfeed.blob.core.windows.net/aspnet-extensions/index.json" />
<add key="gRPC repository" value="https://grpc.jfrog.io/grpc/api/nuget/v3/grpc-nuget-dev" />

</packageSources>
<disabledPackageSources>
<clear />
Expand Down
6 changes: 6 additions & 0 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,12 @@ jobs:
PathtoPublish: '$(Build.SourcesDirectory)\artifacts\VSSetup\$(_BuildConfig)\VisualFSharpFull.vsix'
ArtifactName: 'Nightly'
condition: succeeded()
- task: PublishBuildArtifacts@1
displayName: Publish Artifact Symbols
inputs:
PathtoPublish: '$(Build.SourcesDirectory)\artifacts\SymStore\$(_BuildConfig)'
ArtifactName: 'NativeSymbols'
condition: succeeded()

#---------------------------------------------------------------------------------------------------------------------#
# PR builds #
Expand Down
11 changes: 6 additions & 5 deletions eng/Build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -225,13 +225,14 @@ function UpdatePath() {
TestAndAddToPath "$ArtifactsDir\bin\fsiAnyCpu\$configuration\net472"
}

function VerifyAssemblyVersions() {
$fsiPath = Join-Path $ArtifactsDir "bin\fsi\Proto\net472\publish\fsi.exe"
function VerifyAssemblyVersionsAndSymbols() {
$assemblyVerCheckPath = Join-Path $ArtifactsDir "Bootstrap\AssemblyCheck\AssemblyCheck.dll"

# Only verify versions on CI or official build
if ($ci -or $official) {
$asmVerCheckPath = "$RepoRoot\scripts"
Exec-Console $fsiPath """$asmVerCheckPath\AssemblyVersionCheck.fsx"" -- ""$ArtifactsDir"""
$dotnetPath = InitializeDotNetCli
$dotnetExe = Join-Path $dotnetPath "dotnet.exe"
Exec-Console $dotnetExe """$assemblyVerCheckPath"" ""$ArtifactsDir"""
}
}

Expand Down Expand Up @@ -329,7 +330,7 @@ try {
}

if ($build) {
VerifyAssemblyVersions
VerifyAssemblyVersionsAndSymbols
}

$desktopTargetFramework = "net472"
Expand Down
4 changes: 2 additions & 2 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
<ProductDependencies>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19359.6">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="1.0.0-beta.19378.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>0f5dd7680174620f31c9a00cdb2ac0b0e70e631f</Sha>
<Sha>a8e982d3bac01d8f4f91a4c57191147570079448</Sha>
</Dependency>
</ToolsetDependencies>
</Dependencies>
3 changes: 2 additions & 1 deletion eng/build-utils.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -236,10 +236,11 @@ function Make-BootstrapBuild() {
Remove-Item -re $dir -ErrorAction SilentlyContinue
Create-Directory $dir

# prepare FsLex and Fsyacc
# prepare FsLex and Fsyacc and AssemblyCheck
Run-MSBuild "$RepoRoot\src\buildtools\buildtools.proj" "/restore /t:Publish" -logFileName "BuildTools" -configuration $bootstrapConfiguration
Copy-Item "$ArtifactsDir\bin\fslex\$bootstrapConfiguration\netcoreapp2.1\publish" -Destination "$dir\fslex" -Force -Recurse
Copy-Item "$ArtifactsDir\bin\fsyacc\$bootstrapConfiguration\netcoreapp2.1\publish" -Destination "$dir\fsyacc" -Force -Recurse
Copy-Item "$ArtifactsDir\bin\AssemblyCheck\$bootstrapConfiguration\netcoreapp2.1\publish" -Destination "$dir\AssemblyCheck" -Force -Recurse

# prepare compiler
$projectPath = "$RepoRoot\proto.proj"
Expand Down
2 changes: 1 addition & 1 deletion eng/common/SigningValidation.proj
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<!--
This MSBuild file is intended to be used as the body of the default
publishing release pipeline. The release pipeline will use this file
to invoke the the SignCheck tool to validate that packages about to
to invoke the SignCheck tool to validate that packages about to
be published are correctly signed.
Parameters:
Expand Down
Empty file modified eng/common/build.sh
100644 → 100755
Empty file.
Empty file modified eng/common/cibuild.sh
100644 → 100755
Empty file.
Empty file modified eng/common/cross/armel/tizen-build-rootfs.sh
100644 → 100755
Empty file.
Empty file modified eng/common/cross/armel/tizen-fetch.sh
100644 → 100755
Empty file.
Empty file modified eng/common/cross/build-android-rootfs.sh
100644 → 100755
Empty file.
Empty file modified eng/common/cross/build-rootfs.sh
100644 → 100755
Empty file.
Empty file modified eng/common/darc-init.sh
100644 → 100755
Empty file.
10 changes: 8 additions & 2 deletions eng/common/init-tools-native.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -98,12 +98,18 @@ try {
}

Write-Verbose "Installing $ToolName version $ToolVersion"
Write-Verbose "Executing '$InstallerPath $LocalInstallerArguments'"
Write-Verbose "Executing '$InstallerPath $($LocalInstallerArguments.Keys.ForEach({"-$_ '$($LocalInstallerArguments.$_)'"}) -join ' ')'"
& $InstallerPath @LocalInstallerArguments
if ($LASTEXITCODE -Ne "0") {
$errMsg = "$ToolName installation failed"
if ((Get-Variable 'DoNotAbortNativeToolsInstallationOnFailure' -ErrorAction 'SilentlyContinue') -and $DoNotAbortNativeToolsInstallationOnFailure) {
Write-Warning $errMsg
$showNativeToolsWarning = $true
if ((Get-Variable 'DoNotDisplayNativeToolsInstallationWarnings' -ErrorAction 'SilentlyContinue') -and $DoNotDisplayNativeToolsInstallationWarnings) {
$showNativeToolsWarning = $false
}
if ($showNativeToolsWarning) {
Write-Warning $errMsg
}
$toolInstallationFailure = $true
} else {
Write-Error $errMsg
Expand Down
2 changes: 1 addition & 1 deletion eng/common/init-tools-native.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ function ReadGlobalJsonNativeTools {
local native_tools_list=$(echo $native_tools_section | awk -F"[{}]" '{print $2}')
native_tools_list=${native_tools_list//[\" ]/}
native_tools_list=${native_tools_list//,/$'\n'}
native_tools_list="$(echo -e "${native_tools_list}" | tr -d '[:space:]')"
native_tools_list="$(echo -e "${native_tools_list}" | tr -d '[[:space:]]')"

local old_IFS=$IFS
while read -r line; do
Expand Down
Empty file modified eng/common/internal-feed-operations.sh
100644 → 100755
Empty file.
Empty file modified eng/common/msbuild.sh
100644 → 100755
Empty file.
33 changes: 31 additions & 2 deletions eng/common/native/CommonLibrary.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,38 @@ function DownloadAndExtract {
-Verbose:$Verbose

if ($UnzipStatus -Eq $False) {
Write-Error "Unzip failed"
return $False
# Retry Download one more time with Force=true
$DownloadRetryStatus = CommonLibrary\Get-File -Uri $Uri `
-Path $TempToolPath `
-DownloadRetries 1 `
-RetryWaitTimeInSeconds $RetryWaitTimeInSeconds `
-Force:$True `
-Verbose:$Verbose

if ($DownloadRetryStatus -Eq $False) {
Write-Error "Last attempt of download failed as well"
return $False
}

# Retry unzip again one more time with Force=true
$UnzipRetryStatus = CommonLibrary\Expand-Zip -ZipPath $TempToolPath `
-OutputDirectory $InstallDirectory `
-Force:$True `
-Verbose:$Verbose
if ($UnzipRetryStatus -Eq $False)
{
Write-Error "Last attempt of unzip failed as well"
# Clean up partial zips and extracts
if (Test-Path $TempToolPath) {
Remove-Item $TempToolPath -Force
}
if (Test-Path $InstallDirectory) {
Remove-Item $InstallDirectory -Force -Recurse
}
return $False
}
}

return $True
}

Expand Down
Empty file modified eng/common/native/common-library.sh
100644 → 100755
Empty file.
Empty file modified eng/common/native/install-cmake.sh
100644 → 100755
Empty file.
77 changes: 77 additions & 0 deletions eng/common/performance/perfhelixpublish.proj
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">

<PropertyGroup Condition="'$(AGENT_OS)' == 'Windows_NT'">
<WorkItemCommand>%HELIX_CORRELATION_PAYLOAD%\performance\scripts\benchmarks_ci.py --csproj %HELIX_CORRELATION_PAYLOAD%\performance\$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions %DOTNET_VERSION% --cli-source-info args --cli-branch %PERFLAB_BRANCH% --cli-commit-sha %PERFLAB_HASH% --cli-repository https://github.com/%PERFLAB_REPO% --cli-source-timestamp %PERFLAB_BUILDTIMESTAMP%</CliArguments>
<Python>py -3</Python>
<CoreRun>%HELIX_CORRELATION_PAYLOAD%\Core_Root\CoreRun.exe</CoreRun>
<HelixPreCommands>$(HelixPreCommands);call %HELIX_CORRELATION_PAYLOAD%\performance\tools\machine-setup.cmd</HelixPreCommands>
<ArtifactsDirectory>%HELIX_CORRELATION_PAYLOAD%\artifacts\BenchmarkDotNet.Artifacts</ArtifactsDirectory>
</PropertyGroup>

<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'false'">
<BaseDirectory>$HELIX_CORRELATION_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)/performance</PerformanceDirectory>
</PropertyGroup>

<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT' and '$(RunFromPerfRepo)' == 'true'">
<BaseDirectory>$HELIX_WORKITEM_PAYLOAD</BaseDirectory>
<PerformanceDirectory>$(BaseDirectory)</PerformanceDirectory>
</PropertyGroup>

<PropertyGroup Condition="'$(AGENT_OS)' != 'Windows_NT'">
<WorkItemCommand>$(PerformanceDirectory)/scripts/benchmarks_ci.py --csproj $(PerformanceDirectory)/$(TargetCsproj)</WorkItemCommand>
<CliArguments>--dotnet-versions $DOTNET_VERSION --cli-source-info args --cli-branch $PERFLAB_BRANCH --cli-commit-sha $PERFLAB_HASH --cli-repository https://github.com/$PERFLAB_REPO --cli-source-timestamp $PERFLAB_BUILDTIMESTAMP</CliArguments>
<Python>python3</Python>
<CoreRun>$(BaseDirectory)/Core_Root/corerun</CoreRun>
<HelixPreCommands>$(HelixPreCommands);chmod +x $(PerformanceDirectory)/tools/machine-setup.sh;. $(PerformanceDirectory)/tools/machine-setup.sh</HelixPreCommands>
<ArtifactsDirectory>$(BaseDirectory)/artifacts/BenchmarkDotNet.Artifacts</ArtifactsDirectory>
</PropertyGroup>

<PropertyGroup Condition="'$(UseCoreRun)' == 'true'">
<CoreRunArgument>--corerun $(CoreRun)</CoreRunArgument>
</PropertyGroup>

<PropertyGroup Condition="'$(WorkItemCommand)' != ''">
<WorkItemCommand>$(Python) $(WorkItemCommand) --incremental no --architecture $(Architecture) -f $(_Framework) $(PerfLabArguments)</WorkItemCommand>
</PropertyGroup>

<PropertyGroup Condition="'$(_Framework)' != 'net461'">
<WorkItemCommand>$(WorkItemCommand) $(CliArguments)</WorkItemCommand>
</PropertyGroup>

<ItemGroup>
<HelixCorrelationPayload Include="$(CorrelationPayloadDirectory)">
<PayloadDirectory>%(Identity)</PayloadDirectory>
</HelixCorrelationPayload>
</ItemGroup>

<PropertyGroup>
<PartitionCount>5</PartitionCount>
</PropertyGroup>
<ItemGroup>
<Partition Include="$(BuildConfig).Partition0" Index="0" />
<Partition Include="$(BuildConfig).Partition1" Index="1" />
<Partition Include="$(BuildConfig).Partition2" Index="2" />
<Partition Include="$(BuildConfig).Partition3" Index="3" />
<Partition Include="$(BuildConfig).Partition4" Index="4" />
</ItemGroup>

<!--
Partition the Microbenchmarks project, but nothing else
-->
<ItemGroup Condition="$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="@(Partition)">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --artifacts $(ArtifactsDirectory) --partition-count $(PartitionCount) --partition-index %(HelixWorkItem.Index)"</Command>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
<ItemGroup Condition="!$(TargetCsproj.Contains('MicroBenchmarks.csproj'))">
<HelixWorkItem Include="$(BuildConfig).WorkItem">
<PayloadDirectory>$(WorkItemDirectory)</PayloadDirectory>
<Command>$(WorkItemCommand) --bdn-arguments="--anyCategories $(BDNCategories) $(ExtraBenchmarkDotNetArguments) $(CoreRunArgument) --artifacts $(ArtifactsDirectory)"</Command>
<Timeout>4:00</Timeout>
</HelixWorkItem>
</ItemGroup>
</Project>
91 changes: 91 additions & 0 deletions eng/common/performance/performance-setup.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
Param(
[string] $SourceDirectory=$env:BUILD_SOURCESDIRECTORY,
[string] $CoreRootDirectory,
[string] $Architecture="x64",
[string] $Framework="netcoreapp3.0",
[string] $CompilationMode="Tiered",
[string] $Repository=$env:BUILD_REPOSITORY_NAME,
[string] $Branch=$env:BUILD_SOURCEBRANCH,
[string] $CommitSha=$env:BUILD_SOURCEVERSION,
[string] $BuildNumber=$env:BUILD_BUILDNUMBER,
[string] $RunCategories="coreclr corefx",
[string] $Csproj="src\benchmarks\micro\MicroBenchmarks.csproj",
[string] $Kind="micro",
[switch] $Internal,
[string] $Configurations="CompilationMode=$CompilationMode"
)

$RunFromPerformanceRepo = ($Repository -eq "dotnet/performance")
$UseCoreRun = ($CoreRootDirectory -ne [string]::Empty)

$PayloadDirectory = (Join-Path $SourceDirectory "Payload")
$PerformanceDirectory = (Join-Path $PayloadDirectory "performance")
$WorkItemDirectory = (Join-Path $SourceDirectory "workitem")
$ExtraBenchmarkDotNetArguments = "--iterationCount 1 --warmupCount 0 --invocationCount 1 --unrollFactor 1 --strategy ColdStart --stopOnFirstError true"
$Creator = $env:BUILD_DEFINITIONNAME
$PerfLabArguments = ""
$HelixSourcePrefix = "pr"

$Queue = "Windows.10.Amd64.ClientRS4.DevEx.15.8.Open"

if ($Framework.StartsWith("netcoreapp")) {
$Queue = "Windows.10.Amd64.ClientRS4.Open"
}

if ($Internal) {
$Queue = "Windows.10.Amd64.ClientRS5.Perf"
$PerfLabArguments = "--upload-to-perflab-container"
$ExtraBenchmarkDotNetArguments = ""
$Creator = ""
$HelixSourcePrefix = "official"
}

$CommonSetupArguments="--frameworks $Framework --queue $Queue --build-number $BuildNumber --build-configs $Configurations"
$SetupArguments = "--repository https://github.com/$Repository --branch $Branch --get-perf-hash --commit-sha $CommitSha $CommonSetupArguments"

if ($RunFromPerformanceRepo) {
$SetupArguments = "--perf-hash $CommitSha $CommonSetupArguments"

robocopy $SourceDirectory $PerformanceDirectory /E /XD $PayloadDirectory $SourceDirectory\artifacts $SourceDirectory\.git
}
else {
git clone --branch master --depth 1 --quiet https://github.com/dotnet/performance $PerformanceDirectory
}

if ($UseCoreRun) {
$NewCoreRoot = (Join-Path $PayloadDirectory "Core_Root")
Move-Item -Path $CoreRootDirectory -Destination $NewCoreRoot
}

$DocsDir = (Join-Path $PerformanceDirectory "docs")
robocopy $DocsDir $WorkItemDirectory

# Set variables that we will need to have in future steps
$ci = $true

. "$PSScriptRoot\..\pipeline-logging-functions.ps1"

# Directories
Write-PipelineSetVariable -Name 'PayloadDirectory' -Value "$PayloadDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerformanceDirectory' -Value "$PerformanceDirectory" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'WorkItemDirectory' -Value "$WorkItemDirectory" -IsMultiJobVariable $false

# Script Arguments
Write-PipelineSetVariable -Name 'Python' -Value "py -3" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'ExtraBenchmarkDotNetArguments' -Value "$ExtraBenchmarkDotNetArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'SetupArguments' -Value "$SetupArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'PerfLabArguments' -Value "$PerfLabArguments" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'BDNCategories' -Value "$RunCategories" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'TargetCsproj' -Value "$Csproj" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Kind' -Value "$Kind" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Architecture' -Value "$Architecture" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'UseCoreRun' -Value "$UseCoreRun" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'RunFromPerfRepo' -Value "$RunFromPerformanceRepo" -IsMultiJobVariable $false

# Helix Arguments
Write-PipelineSetVariable -Name 'Creator' -Value "$Creator" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'Queue' -Value "$Queue" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name 'HelixSourcePrefix' -Value "$HelixSourcePrefix" -IsMultiJobVariable $false
Write-PipelineSetVariable -Name '_BuildConfig' -Value "$Architecture.$Kind.$Framework" -IsMultiJobVariable $false

exit 0
Loading

0 comments on commit 71194c7

Please sign in to comment.