Applying comments from API Review (#18662)

* Removing return response for Pause/Resume/Stop recording

* Removing validations

* Fixes after merge

* Added downloadToFile operation

* Added downloadToFile test

* Updating API MD file

* Fixing format auto

* Renaming files

* [KeyVault] - Move MHSM to resource group location (#18664)

Moving the Managed HSM to the same location as our resource group now that the
limits have been expanded.

* Add default cloud configuration values to source (#18653)

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>

* simplify the commit history so that the patch can apply properly (#18665)

Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>

* Updating name for header

* [Event Hubs] Merge feature branch for buffered producer (#18590)

* [event-hubs] EventHubsBufferedProducerClient skeleton (#17761)

* temp transfer

* [event-hubs] initial EventHubBufferedProducerClient skeleton

* Change default value of BufferedCloseOptions.flush to true instead of false

* Reorder doc comment content for EventHubBufferedProducerClient

* remove eslint exception for @azure/azure-sdk/ts-naming-options in EventHubBufferedProducerClientOptions

* add enqueueEvent (singular) method

* update API view

* [WIP][event-hubs] EventHubBufferedProducerClient implementation (#18106)

* transfer

* [mock-hub] fix issue where calling stop() would not remove existing idle connection timeout watchers

* [event-hubs] reduce repetition in hubruntime.spec.ts tests

* Add support for flush,eventSuccess and eventError handlers, and backpressure to EventHubBufferedProducerClient

* add documentation and update API to align with .NET

* [Buffered Event Hubs Producer] Implements Parition Key to Partition ID mapping (#18331)

* [Buffered Event Hubs Producer] Implements Parition Key to Partition ID mapping

* fix the implementation

* factor out the hashing logic

* remove unused import

* the test pass but the implementation needs to be simplified

* simplified implementation

* address feedback

* address feedback

* fix format

* address feedback

Co-authored-by: chradek <51000525+chradek@users.noreply.github.com>

* [Event Hubs] Prepare release (#18672)

* [Event Hubs] Prepare release

* remove empty sections

* Node doesn't support some samples for smoke test (#18496)

* Node doesn't support some samples for smoke test

* [search-documents] reprocessed samples with exp. generator

Co-authored-by: Will Temple <will@wtemple.net>

* Add rlc quickstart guideline (#18503)

* add llc quickstart guideline

* update format

* add documentation about ci.yml

* update to resolve some comments

* update to resolve some comments

* update to resolve comments

* updates term

* update format

* update format

* Running Rush update

* Sync eng/common directory with azure-sdk-tools for PR 2265 (#18683)

* Improve devops logging for link checker

* Update eng/common/scripts/Verify-Links.ps1

Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

* Build and test predefined set of packages only in identity pipelines (#18686)

* Build and test predefined set of packages only in identity pipelines

* Fixing playback testing

* Run rushx format

Co-authored-by: Maor Leger <maorleger@users.noreply.github.com>
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Deyaaeldeen Almahallawi <dealmaha@microsoft.com>
Co-authored-by: chradek <51000525+chradek@users.noreply.github.com>
Co-authored-by: Sarangan Rajamanickam <sarajama@microsoft.com>
Co-authored-by: Will Temple <will@wtemple.net>
Co-authored-by: Qiaoqiao Zhang <55688292+qiaozha@users.noreply.github.com>
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: praveenkuttappan <55455725+praveenkuttappan@users.noreply.github.com>

eng/common/TestResources/New-TestResources.ps1

@@ -84,11 +84,13 @@ if (!$PSBoundParameters.ContainsKey('ErrorAction')) {
     $ErrorActionPreference = 'Stop'
 }
 
-function Log($Message) {
+function Log($Message)
+{
     Write-Host ('{0} - {1}' -f [DateTime]::Now.ToLongTimeString(), $Message)
 }
 
-function Retry([scriptblock] $Action, [int] $Attempts = 5) {
+function Retry([scriptblock] $Action, [int] $Attempts = 5)
+{
     $attempt = 0
     $sleep = 5
 
@@ -109,7 +111,20 @@ function Retry([scriptblock] $Action, [int] $Attempts = 5) {
     }
 }
 
-function MergeHashes([hashtable] $source, [psvariable] $dest) {
+function LoadCloudConfig([string] $env)
+{
+    $configPath = "$PSScriptRoot/clouds/$env.json"
+    if (!(Test-Path $configPath)) {
+        Write-Warning "Could not find cloud configuration for environment '$env'"
+        return @{}
+    }
+
+    $config = Get-Content $configPath | ConvertFrom-Json -AsHashtable
+    return $config
+}
+
+function MergeHashes([hashtable] $source, [psvariable] $dest)
+{
     foreach ($key in $source.Keys) {
         if ($dest.Value.ContainsKey($key) -and $dest.Value[$key] -ne $source[$key]) {
             Write-Warning ("Overwriting '$($dest.Name).$($key)' with value '$($dest.Value[$key])' " +
@@ -119,7 +134,8 @@ function MergeHashes([hashtable] $source, [psvariable] $dest) {
     }
 }
 
-function BuildBicepFile([System.IO.FileSystemInfo] $file) {
+function BuildBicepFile([System.IO.FileSystemInfo] $file)
+{
     if (!(Get-Command bicep -ErrorAction Ignore)) {
         Write-Error "A bicep file was found at '$($file.FullName)' but the Azure Bicep CLI is not installed. See https://aka.ms/install-bicep-pwsh"
         throw
@@ -492,6 +508,8 @@ try {
         $templateParameters.Add('testApplicationSecret', $TestApplicationSecret)
     }
 
+    $defaultCloudParameters = LoadCloudConfig $Environment
+    MergeHashes $defaultCloudParameters $(Get-Variable templateParameters)
     MergeHashes $ArmTemplateParameters $(Get-Variable templateParameters)
     MergeHashes $AdditionalParameters $(Get-Variable templateParameters)
 

eng/common/TestResources/clouds/AzureChinaCloud.json

@@ -0,0 +1,15 @@
+{
+    "azConfigEndpointSuffix": ".azconfig.azure.cn",
+    "azureAuthorityHost": "https://login.chinacloudapi.cn/",
+    "cognitiveServicesEndpointSuffix": ".cognitiveservices.azure.cn",
+    "containerRegistryEndpointSuffix": ".azurecr.cn",
+    "cosmosEndpointSuffix": "cosmos.azure.cn",
+    "enableStorageVersioning": false,
+    "keyVaultDomainSuffix": ".vault.azure.cn",
+    "keyVaultEndpointSuffix": ".vault.azure.cn",
+    "keyVaultSku": "standard",
+    "searchEndpointSuffix": "search.azure.cn",
+    "serviceBusEndpointSuffix": ".servicebus.chinacloudapi.cn",
+    "storageEndpointSuffix": "core.chinacloudapi.cn",
+    "textAnalyticsSku": "S"
+}

eng/common/TestResources/clouds/AzureCloud.json

@@ -0,0 +1,8 @@
+{
+    "azureAuthorityHost": "https://login.microsoftonline.com/",
+    "cognitiveServicesEndpointSuffix": ".cognitiveservices.azure.com",
+    "communicationServicesEndpointSuffix": ".communication.azure.com",
+    "keyVaultDomainSuffix": ".vault.azure.net",
+    "keyVaultEndpointSuffix": ".vault.azure.net",
+    "storageEndpointSuffix": "core.windows.net"
+}

eng/common/TestResources/clouds/AzureUSGovernment.json

@@ -0,0 +1,15 @@
+{
+    "azConfigEndpointSuffix": ".azconfig.azure.us",
+    "azureAuthorityHost": "https://login.microsoftonline.us/",
+    "cognitiveServicesEndpointSuffix": ".cognitiveservices.azure.us",
+    "containerRegistryEndpointSuffix": ".azurecr.us",
+    "cosmosEndpointSuffix": "cosmos.azure.us",
+    "enableStorageVersioning": false,
+    "formRecognizerLocation": "usgovvirginia",
+    "keyVaultDomainSuffix": ".vault.usgovcloudapi.net",
+    "keyVaultEndpointSuffix": ".vault.usgovcloudapi.net",
+    "keyVaultSku": "premium",
+    "searchEndpointSuffix": "search.azure.us",
+    "serviceBusEndpointSuffix": ".servicebus.usgovcloudapi.net",
+    "storageEndpointSuffix": "core.usgovcloudapi.net"
+}

eng/common/scripts/Verify-Links.ps1

@@ -244,10 +244,10 @@ function CheckLink ([System.Uri]$linkUri, $allowRetry=$true)
 
       if ($statusCode -in $errorStatusCodes) {
         if ($originalLinkUri -ne $linkUri) {
-          LogWarning "[$statusCode] broken link $originalLinkUri (resolved to $linkUri)"
+          LogError "[$statusCode] broken link $originalLinkUri (resolved to $linkUri)"
         }
         else {
-          LogWarning "[$statusCode] broken link $linkUri"
+          LogError "[$statusCode] broken link $linkUri"
         }
 
         $linkValid = $false
@@ -423,14 +423,17 @@ foreach ($url in $urls) {
   $pageUrisToCheck.Enqueue($uri);
 }
 
+if ($devOpsLogging) {
+  Write-Host "##[group]Link checking details"
+}
 while ($pageUrisToCheck.Count -ne 0)
 {
   $pageUri = $pageUrisToCheck.Dequeue();
   if ($checkedPages.ContainsKey($pageUri)) { continue }
   $checkedPages[$pageUri] = $true;
 
   $linkUris = GetLinks $pageUri
-  Write-Host "Found $($linkUris.Count) links on page $pageUri";
+  Write-Host "Checking $($linkUris.Count) links found on page $pageUri";
   $badLinksPerPage = @();
   foreach ($linkUri in $linkUris) {
     $isLinkValid = CheckLink $linkUri
@@ -450,6 +453,9 @@ while ($pageUrisToCheck.Count -ne 0)
     $badLinks[$pageUri] = $badLinksPerPage
   }
 }
+if ($devOpsLogging) {
+  Write-Host "##[endgroup]"
+}
 
 if ($badLinks.Count -gt 0) {
   Write-Host "Summary of broken links:"
@@ -464,7 +470,7 @@ foreach ($pageLink in $badLinks.Keys) {
 $linksChecked = $checkedLinks.Count - $cachedLinksCount
 
 if ($badLinks.Count -gt 0) {
-  LogError "Checked $linksChecked links with $($badLinks.Count) page(s) broken."
+  Write-Host "Checked $linksChecked links with $($badLinks.Count) broken link(s) found."
 }
 else {
   Write-Host "Checked $linksChecked links. No broken links found."

eng/common/testproxy/test-proxy-tool.yml

@@ -22,13 +22,13 @@ steps:
   - pwsh: |
       Start-Process $(Build.BinariesDirectory)/test-proxy/test-proxy.exe `
         -ArgumentList "--storage-location '${{ parameters.rootFolder }}'" `
-        -NoNewWindow -PassThru
+        -NoNewWindow -PassThru -RedirectStandardOutput $(Build.SourcesDirectory)/test-proxy.log
     displayName: 'Run the testproxy - windows'
     condition: and(succeeded(), eq(variables['Agent.OS'],'Windows_NT'))
 
   # nohup does NOT continue beyond the current session if you use it within powershell
   - bash: |
-      nohup $(Build.BinariesDirectory)/test-proxy/test-proxy &
+      nohup $(Build.BinariesDirectory)/test-proxy/test-proxy > $(Build.SourcesDirectory)/test-proxy.log &
     displayName: "Run the testproxy - linux/mac"
     condition: and(succeeded(), ne(variables['Agent.OS'],'Windows_NT'))
     workingDirectory: "${{ parameters.rootFolder }}"

eng/tools/rush-runner.js

@@ -34,7 +34,22 @@ const reducedDependencyTestMatrix = {
     '@azure/identity-vscode',
     '@azure/storage-file-share',
     '@azure/template'
-  ]
+  ],
+  'identity': [
+    '@azure-rest/core-client',
+    '@azure-rest/core-client-lro',
+    '@azure-rest/core-client-paging',
+    '@azure-rest/purview-account',
+    '@azure-tests/perf-storage-blob',
+    '@azure/ai-text-analytics',
+    '@azure/arm-compute',
+    '@azure/identity-cache-persistence',
+    '@azure/identity-vscode',
+    '@azure/service-bus',
+    '@azure/storage-blob',
+    '@azure/template',
+    '@azure/synapse-monitoring'
+  ],
 };
 
 const parseArgs = () => {

sdk/communication/communication-calling-server/package.json

@@ -9,6 +9,7 @@
     "stream": "./node_modules/stream-browserify/index.js",
     "./dist-esm/src/credentials/cryptoUtils.js": "./dist-esm/src/credentials/cryptoUtils.browser.js",
     "./dist-esm/src/RepeatableContentDownloadResponse.js": "./dist-esm/src/RepeatableContentDownloadResponse.browser.js",
+    "./dist-esm/src/utils/utils.node.js": "./dist-esm/src/utils/utils.browser.js",
     "./dist-esm/test/public/utils/index.js": "./dist-esm/test/public/utils/index.browser.js"
   },
   "types": "types/communication-calling-server.d.ts",