Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[main] Update dependencies from dotnet/arcade #874

Merged
merged 1 commit into from
Jun 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,25 @@
</Dependency>
</ProductDependencies>
<ToolsetDependencies>
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="6.0.0-beta.21304.1">
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="6.0.0-beta.21311.3">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>85a65ea1fca1d0867f699fed44d191358270bf6a</Sha>
<Sha>4a2b475948d498b89fedef7cf890883f49bc1ea3</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="6.0.0-beta.21304.1">
<Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="6.0.0-beta.21311.3">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>85a65ea1fca1d0867f699fed44d191358270bf6a</Sha>
<Sha>4a2b475948d498b89fedef7cf890883f49bc1ea3</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.SignTool" Version="6.0.0-beta.21304.1">
<Dependency Name="Microsoft.DotNet.SignTool" Version="6.0.0-beta.21311.3">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>85a65ea1fca1d0867f699fed44d191358270bf6a</Sha>
<Sha>4a2b475948d498b89fedef7cf890883f49bc1ea3</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="6.0.0-beta.21304.1">
<Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="6.0.0-beta.21311.3">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>85a65ea1fca1d0867f699fed44d191358270bf6a</Sha>
<Sha>4a2b475948d498b89fedef7cf890883f49bc1ea3</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.SwaggerGenerator.MSBuild" Version="6.0.0-beta.21304.1">
<Dependency Name="Microsoft.DotNet.SwaggerGenerator.MSBuild" Version="6.0.0-beta.21311.3">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>85a65ea1fca1d0867f699fed44d191358270bf6a</Sha>
<Sha>4a2b475948d498b89fedef7cf890883f49bc1ea3</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.Maestro.Client" Version="1.1.0-beta.20258.6">
<Uri>https://github.com/dotnet/arcade-services</Uri>
Expand Down
6 changes: 3 additions & 3 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,8 @@
<XUnitVersion>2.4.1</XUnitVersion>
<XUnitAbstractionsVersion>2.0.3</XUnitAbstractionsVersion>
<XUnitVSRunnerVersion>2.4.1</XUnitVSRunnerVersion>
<MicrosoftDotNetBuildTasksFeedVersion>6.0.0-beta.21304.1</MicrosoftDotNetBuildTasksFeedVersion>
<MicrosoftDotNetSignToolVersion>6.0.0-beta.21304.1</MicrosoftDotNetSignToolVersion>
<MicrosoftDotNetBuildTasksFeedVersion>6.0.0-beta.21311.3</MicrosoftDotNetBuildTasksFeedVersion>
<MicrosoftDotNetSignToolVersion>6.0.0-beta.21311.3</MicrosoftDotNetSignToolVersion>
<MicrosoftAzureDocumentDBVersion>1.22.0</MicrosoftAzureDocumentDBVersion>
<MicrosoftAzureCosmosDBTableVersion>1.1.2</MicrosoftAzureCosmosDBTableVersion>
<MicrosoftAspNetCoreAllVersion>2.0.0</MicrosoftAspNetCoreAllVersion>
Expand All @@ -108,7 +108,7 @@
<MicrosoftDotNetMaestroClientVersion>1.1.0-beta.20258.6</MicrosoftDotNetMaestroClientVersion>
<MicrosoftSourceLinkGitHubVersion>1.1.0-beta-21228-01</MicrosoftSourceLinkGitHubVersion>
<MicrosoftSourceLinkAzureReposGitVersion>1.1.0-beta-21228-01</MicrosoftSourceLinkAzureReposGitVersion>
<MicrosoftDotNetSwaggerGeneratorMSBuildVersion>6.0.0-beta.21304.1</MicrosoftDotNetSwaggerGeneratorMSBuildVersion>
<MicrosoftDotNetSwaggerGeneratorMSBuildVersion>6.0.0-beta.21311.3</MicrosoftDotNetSwaggerGeneratorMSBuildVersion>
<XliffTasksVersion>1.0.0-beta.21265.1</XliffTasksVersion>
<MicrosoftDotNetMaestroTasksVersion>1.1.0-beta.21228.1</MicrosoftDotNetMaestroTasksVersion>
<MicrosoftDotNetXHarnessCLIVersion>1.0.0-prerelease.21264.3</MicrosoftDotNetXHarnessCLIVersion>
Expand Down
2 changes: 1 addition & 1 deletion eng/common/dotnet-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ case $cpuname in
;;
esac

dotnetRoot="$repo_root/.dotnet"
dotnetRoot="${repo_root}.dotnet"
if [[ $architecture != "" ]] && [[ $architecture != $buildarch ]]; then
dotnetRoot="$dotnetRoot/$architecture"
fi
Expand Down
6 changes: 3 additions & 3 deletions eng/common/internal-feed-operations.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,11 @@ function SetupCredProvider {
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully

$nugetConfigPath = "$RepoRoot\NuGet.config"
$nugetConfigPath = Join-Path $RepoRoot "NuGet.config"

if (-Not (Test-Path -Path $nugetConfigPath)) {
Write-PipelineTelemetryError -Category 'Build' -Message 'NuGet.config file not found in repo root!'
ExitWithExitCode 1
ExitWithExitCode 1
}

$endpoints = New-Object System.Collections.ArrayList
Expand Down Expand Up @@ -85,7 +85,7 @@ function SetupCredProvider {

#Workaround for https://github.com/microsoft/msbuild/issues/4430
function InstallDotNetSdkAndRestoreArcade {
$dotnetTempDir = "$RepoRoot\dotnet"
$dotnetTempDir = Join-Path $RepoRoot "dotnet"
$dotnetSdkVersion="2.1.507" # After experimentation we know this version works when restoring the SDK (compared to 3.0.*)
$dotnet = "$dotnetTempDir\dotnet.exe"
$restoreProjPath = "$PSScriptRoot\restore.proj"
Expand Down
2 changes: 1 addition & 1 deletion eng/common/internal-feed-operations.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ function SetupCredProvider {
# Then, we set the 'VSS_NUGET_EXTERNAL_FEED_ENDPOINTS' environment variable to restore from the stable
# feeds successfully

local nugetConfigPath="$repo_root/NuGet.config"
local nugetConfigPath="{$repo_root}NuGet.config"

if [ ! "$nugetConfigPath" ]; then
Write-PipelineTelemetryError -category 'Build' "NuGet.config file not found in repo's root!"
Expand Down
4 changes: 2 additions & 2 deletions eng/common/sdk-task.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ function Print-Usage() {
function Build([string]$target) {
$logSuffix = if ($target -eq 'Execute') { '' } else { ".$target" }
$log = Join-Path $LogDir "$task$logSuffix.binlog"
$outputPath = Join-Path $ToolsetDir "$task\\"
$outputPath = Join-Path $ToolsetDir "$task\"

MSBuild $taskProject `
/bl:$log `
Expand Down Expand Up @@ -64,7 +64,7 @@ try {
$GlobalJson.tools | Add-Member -Name "vs" -Value (ConvertFrom-Json "{ `"version`": `"16.5`" }") -MemberType NoteProperty
}
if( -not ($GlobalJson.tools.PSObject.Properties.Name -match "xcopy-msbuild" )) {
$GlobalJson.tools | Add-Member -Name "xcopy-msbuild" -Value "16.8.0-preview3" -MemberType NoteProperty
$GlobalJson.tools | Add-Member -Name "xcopy-msbuild" -Value "16.10.0-preview2" -MemberType NoteProperty
}
if ($GlobalJson.tools."xcopy-msbuild".Trim() -ine "none") {
$xcopyMSBuildToolsFolder = InitializeXCopyMSBuild $GlobalJson.tools."xcopy-msbuild" -install $true
Expand Down
2 changes: 1 addition & 1 deletion eng/common/sdl/execute-all-sdl-tools.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ try {
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
$global:LASTEXITCODE = 0

# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
Expand Down
2 changes: 1 addition & 1 deletion eng/common/sdl/init-sdl.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Param(
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
$global:LASTEXITCODE = 0

# `tools.ps1` checks $ci to perform some actions. Since the SDL
# scripts don't necessarily execute in the same agent that run the
Expand Down
2 changes: 1 addition & 1 deletion eng/common/sdl/run-sdl.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Param(
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2.0
$disableConfigureToolsetImport = $true
$LASTEXITCODE = 0
$global:LASTEXITCODE = 0

try {
# `tools.ps1` checks $ci to perform some actions. Since the SDL
Expand Down
8 changes: 8 additions & 0 deletions eng/common/templates/job/onelocbuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ parameters:
LclSource: lclFilesInRepo
LclPackageId: ''
RepoType: gitHub
GitHubOrg: dotnet
MirrorRepo: ''
MirrorBranch: main
condition: ''

jobs:
Expand Down Expand Up @@ -66,6 +69,11 @@ jobs:
${{ if eq(parameters.RepoType, 'gitHub') }}:
repoType: ${{ parameters.RepoType }}
gitHubPatVariable: "${{ parameters.GithubPat }}"
${{ if ne(parameters.MirrorRepo, '') }}:
isMirrorRepoSelected: true
gitHubOrganization: ${{ parameters.GitHubOrg }}
mirrorRepo: ${{ parameters.MirrorRepo }}
mirrorBranch: ${{ parameters.MirrorBranch }}
condition: ${{ parameters.condition }}

- task: PublishBuildArtifacts@1
Expand Down
99 changes: 46 additions & 53 deletions eng/common/tools.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -193,38 +193,42 @@ function InitializeDotNetCli([bool]$install, [bool]$createSdkLocationFile) {
return $global:_DotNetInstallDir = $dotnetRoot
}

function Retry($downloadBlock, $maxRetries = 5) {
$retries = 1

while($true) {
try {
& $downloadBlock
break
}
catch {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
}

if (++$retries -le $maxRetries) {
$delayInSeconds = [math]::Pow(2, $retries) - 1 # Exponential backoff
Write-Host "Retrying. Waiting for $delayInSeconds seconds before next attempt ($retries of $maxRetries)."
Start-Sleep -Seconds $delayInSeconds
}
else {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unable to download file in $maxRetries attempts."
break
}

}
}

function GetDotNetInstallScript([string] $dotnetRoot) {
$installScript = Join-Path $dotnetRoot 'dotnet-install.ps1'
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit

$maxRetries = 5
$retries = 1

$uri = "https://dot.net/$dotnetInstallScriptVersion/dotnet-install.ps1"

while($true) {
try {
Write-Host "GET $uri"
Invoke-WebRequest $uri -OutFile $installScript
break
}
catch {
Write-Host "Failed to download '$uri'"
Write-Error $_.Exception.Message -ErrorAction Continue
}

if (++$retries -le $maxRetries) {
$delayInSeconds = [math]::Pow(2, $retries) - 1 # Exponential backoff
Write-Host "Retrying. Waiting for $delayInSeconds seconds before next attempt ($retries of $maxRetries)."
Start-Sleep -Seconds $delayInSeconds
}
else {
throw "Unable to download file in $maxRetries attempts."
}

}
Retry({
Write-Host "GET $uri"
Invoke-WebRequest $uri -OutFile $installScript
})
}

return $installScript
Expand Down Expand Up @@ -308,8 +312,8 @@ function InitializeVisualStudioMSBuild([bool]$install, [object]$vsRequirements =

# If the version of msbuild is going to be xcopied,
# use this version. Version matches a package here:
# https://dev.azure.com/dnceng/public/_packaging?_a=package&feed=dotnet-eng&package=RoslynTools.MSBuild&protocolType=NuGet&version=16.8.0-preview3&view=overview
$defaultXCopyMSBuildVersion = '16.8.0-preview3'
# https://dev.azure.com/dnceng/public/_packaging?_a=package&feed=dotnet-eng&package=RoslynTools.MSBuild&protocolType=NuGet&version=16.10.0-preview2&view=overview
$defaultXCopyMSBuildVersion = '16.10.0-preview2'

if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
$vsMinVersionStr = if ($vsRequirements.version) { $vsRequirements.version } else { $vsMinVersionReqdStr }
Expand Down Expand Up @@ -403,9 +407,13 @@ function InitializeXCopyMSBuild([string]$packageVersion, [bool]$install) {
}

Create-Directory $packageDir

Write-Host "Downloading $packageName $packageVersion"
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
Invoke-WebRequest "https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/flat2/$packageName/$packageVersion/$packageName.$packageVersion.nupkg" -OutFile $packagePath
Retry({
Invoke-WebRequest "https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/flat2/$packageName/$packageVersion/$packageName.$packageVersion.nupkg" -OutFile $packagePath
})

Unzip $packagePath $packageDir
}

Expand Down Expand Up @@ -442,27 +450,9 @@ function LocateVisualStudio([object]$vsRequirements = $null){
if (!(Test-Path $vsWhereExe)) {
Create-Directory $vsWhereDir
Write-Host 'Downloading vswhere'
$maxRetries = 5
$retries = 1

while($true) {
try {
Invoke-WebRequest "https://netcorenativeassets.blob.core.windows.net/resource-packages/external/windows/vswhere/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
break
}
catch{
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message $_
}

if (++$retries -le $maxRetries) {
$delayInSeconds = [math]::Pow(2, $retries) - 1 # Exponential backoff
Write-Host "Retrying. Waiting for $delayInSeconds seconds before next attempt ($retries of $maxRetries)."
Start-Sleep -Seconds $delayInSeconds
}
else {
Write-PipelineTelemetryError -Category 'InitializeToolset' -Message "Unable to download file in $maxRetries attempts."
}
}
Retry({
Invoke-WebRequest "https://netcorenativeassets.blob.core.windows.net/resource-packages/external/windows/vswhere/$vswhereVersion/vswhere.exe" -OutFile $vswhereExe
})
}

if (!$vsRequirements) { $vsRequirements = $GlobalJson.tools.vs }
Expand Down Expand Up @@ -498,7 +488,7 @@ function InitializeBuildTool() {
if (Test-Path variable:global:_BuildTool) {
# If the requested msbuild parameters do not match, clear the cached variables.
if($global:_BuildTool.Contains('ExcludePrereleaseVS') -and $global:_BuildTool.ExcludePrereleaseVS -ne $excludePrereleaseVS) {
Remove-Item variable:global:_BuildTool
Remove-Item variable:global:_BuildTool
Remove-Item variable:global:_MSBuildExe
} else {
return $global:_BuildTool
Expand Down Expand Up @@ -555,7 +545,7 @@ function GetDefaultMSBuildEngine() {

function GetNuGetPackageCachePath() {
if ($env:NUGET_PACKAGES -eq $null) {
# Use local cache on CI to ensure deterministic build.
# Use local cache on CI to ensure deterministic build.
# Avoid using the http cache as workaround for https://github.com/NuGet/Home/issues/3116
# use global cache in dev builds to avoid cost of downloading packages.
# For directory normalization, see also: https://github.com/NuGet/Home/issues/7968
Expand Down Expand Up @@ -712,7 +702,10 @@ function MSBuild-Core() {
}

foreach ($arg in $args) {
if ($arg -ne $null -and $arg.Trim() -ne "") {
if ($null -ne $arg -and $arg.Trim() -ne "") {
if ($arg.EndsWith('\')) {
$arg = $arg + "\"
}
$cmdArgs += " `"$arg`""
}
}
Expand Down Expand Up @@ -784,7 +777,7 @@ function Get-Darc($version) {

. $PSScriptRoot\pipeline-logging-functions.ps1

$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..\')
$EngRoot = Resolve-Path (Join-Path $PSScriptRoot '..')
$ArtifactsDir = Join-Path $RepoRoot 'artifacts'
$ToolsetDir = Join-Path $ArtifactsDir 'toolset'
Expand Down
9 changes: 5 additions & 4 deletions eng/common/tools.sh
Original file line number Diff line number Diff line change
Expand Up @@ -485,13 +485,14 @@ _script_dir=`dirname "$_ResolvePath"`

eng_root=`cd -P "$_script_dir/.." && pwd`
repo_root=`cd -P "$_script_dir/../.." && pwd`
artifacts_dir="$repo_root/artifacts"
repo_root="${repo_root}/"
artifacts_dir="${repo_root}artifacts"
toolset_dir="$artifacts_dir/toolset"
tools_dir="$repo_root/.tools"
tools_dir="${repo_root}.tools"
log_dir="$artifacts_dir/log/$configuration"
temp_dir="$artifacts_dir/tmp/$configuration"

global_json_file="$repo_root/global.json"
global_json_file="${repo_root}global.json"
# determine if global.json contains a "runtimes" entry
global_json_has_runtimes=false
if command -v jq &> /dev/null; then
Expand All @@ -504,7 +505,7 @@ fi

# HOME may not be defined in some scenarios, but it is required by NuGet
if [[ -z $HOME ]]; then
export HOME="$repo_root/artifacts/.home/"
export HOME="${repo_root}artifacts/.home/"
mkdir -p "$HOME"
fi

Expand Down
6 changes: 3 additions & 3 deletions global.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@
"Microsoft.VisualStudio.Component.VSSDK"
]
},
"dotnet": "6.0.100-preview.3.21202.5",
"dotnet": "6.0.100-preview.4.21255.9",
"vswhere": "2.5.2"
},
"msbuild-sdks": {
"Microsoft.DotNet.Arcade.Sdk": "6.0.0-beta.21304.1",
"Microsoft.DotNet.Helix.Sdk": "6.0.0-beta.21304.1"
"Microsoft.DotNet.Arcade.Sdk": "6.0.0-beta.21311.3",
"Microsoft.DotNet.Helix.Sdk": "6.0.0-beta.21311.3"
}
}