Import-Module Az.Accounts throws an exception if 2 environment vars differ only by case

[edyoung]

Description

If any two environment variables which differ only by case exist when importing the az.accounts module, it throws an exception. It doesn't matter if these variables will be referred to by Az, any ones will do.

This occurs because the code introduced in PR #18162 creates a dictionary from all the existing variables, but case-folds them in the process.

See method List() in https://github.com/Azure/azure-powershell/pull/18162/files#diff-adfe64b8657069009dc1abf7f6c0d9981943cd1ba5c943b59d2ef18be56f6b92

This issue can only be reproduced on linux, because on Windows environment variables are case-insensitive.
It is new in Az.Accounts 2.8.0, i could not repro with 2.7.1

Issue script & Debug output

PS /mnt/c/onebranch/HybridAgent> $DebugPreference="Continue"
PS /mnt/c/onebranch/HybridAgent> $env:foo="lowercase"
PS /mnt/c/onebranch/HybridAgent> $env:FOO="uppercase"
PS /mnt/c/onebranch/HybridAgent> ipmo az.accounts
DEBUG: Registering Az shared AssemblyLoadContext for path: '/home/edwin/.local/share/powershell/Modules/Az.Accounts/2.8.0/StartupScripts/../AzSharedAlcAssemblies'.
DEBUG: AssemblyLoadContext registered.
Import-Module: /home/edwin/.local/share/powershell/Modules/Az.Accounts/2.8.0/Az.Accounts.psm1:117
Line |
 117 |  Import-Module (Join-Path -Path $PSScriptRoot -ChildPath Microsoft.Azu …
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | An item with the same key has already been added. Key: foo

PS /mnt/c/onebranch/HybridAgent> $e = $Error[0]
PS /mnt/c/onebranch/HybridAgent> $e.Exception.StackTrace
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at Microsoft.Azure.Commands.Common.Authentication.Config.DefaultEnvironmentVariableProvider.List(EnvironmentVariableTarget target)
   at Microsoft.Azure.Commands.Common.Authentication.Config.Internal.Providers.EnvironmentVariablesConfigurationProvider.Load()
   at Microsoft.Azure.Commands.Common.Authentication.Config.Internal.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Azure.Commands.Common.Authentication.Config.Internal.ConfigurationBuilder.Build()
   at Microsoft.Azure.Commands.Common.Authentication.Config.ConfigManager.BuildConfig()
   at Microsoft.Azure.Commands.Common.Authentication.Config.ConfigInitializer.InitializeForAzureSession(AzureSession session)
   at Microsoft.Azure.Commands.Common.Authentication.AzureSessionInitializer.InitializeConfigs(AzureSession session, String profilePath)
   at Microsoft.Azure.Commands.Common.Authentication.AzureSessionInitializer.CreateInstance(IDataStore dataStore)
   at Microsoft.Azure.Commands.Common.Authentication.AzureSessionInitializer.<>c.<InitializeAzureSession>b__2_0()
   at Microsoft.Azure.Commands.Common.Authentication.AzureSession.Initialize(Func`1 instanceCreator, Boolean overwrite)
   at Microsoft.Azure.Commands.Common.Authentication.AzureSession.Initialize(Func`1 instanceCreator)
   at Microsoft.Azure.Commands.Common.Authentication.AzureSessionInitializer.InitializeAzureSession()
   at Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand.OnImport()
   at System.Management.Automation.Runspaces.PSSnapInHelpers.ExecuteModuleInitializer(Assembly assembly, IEnumerable`1 assemblyTypes) in /PowerShell/src/System.Management.Automation/engine/InitialSessionState.cs:line 5432
   at System.Management.Automation.Runspaces.PSSnapInHelpers.AnalyzeModuleAssemblyWithReflection(Assembly assembly, String name, PSSnapInInfo psSnapInInfo, PSModuleInfo moduleInfo, String helpFile, Dictionary`2& cmdlets, Dictionary`2& aliases, Dictionary`2& providers) in /PowerShell/src/System.Management.Automation/engine/InitialSessionState.cs:line 5209
   at System.Management.Automation.Runspaces.PSSnapInHelpers.AnalyzePSSnapInAssembly(Assembly assembly, String name, PSSnapInInfo psSnapInInfo, PSModuleInfo moduleInfo, Dictionary`2& cmdlets, Dictionary`2& aliases, Dictionary`2& providers, String& helpFile) in /PowerShell/src/System.Management.Automation/engine/InitialSessionState.cs:line 5154
   at System.Management.Automation.Runspaces.InitialSessionState.ImportCmdletsFromAssembly(Assembly assembly, PSModuleInfo module) in /PowerShell/src/System.Management.Automation/engine/InitialSessionState.cs:line 4058
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadBinaryModule(PSModuleInfo parentModule, Boolean trySnapInName, String moduleName, String fileName, Assembly assemblyToLoad, String moduleBase, SessionState ss, ImportModuleOptions options, ManifestProcessingFlags manifestProcessingFlags, String prefix, Boolean loadTypes, Boolean loadFormats, Boolean& found, String shortModuleName, Boolean disableFormatUpdates) in /PowerShell/src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs:line 6700
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadBinaryModule(Boolean trySnapInName, String moduleName, String fileName, Assembly assemblyToLoad, String moduleBase, SessionState ss, ImportModuleOptions options, ManifestProcessingFlags manifestProcessingFlags, String prefix, Boolean loadTypes, Boolean loadFormats, Boolean& found) in /PowerShell/src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs:line 6518
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModule(PSModuleInfo parentModule, String fileName, String moduleBase, String prefix, SessionState ss, Object privateData, ImportModuleOptions& options, ManifestProcessingFlags manifestProcessingFlags, Boolean& found, Boolean& moduleFileFound) in /PowerShell/src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs:line 5875
   at Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModule(String fileName, String moduleBase, String prefix, SessionState ss, ImportModuleOptions& options, ManifestProcessingFlags manifestProcessingFlags, Boolean& found) in /PowerShell/src/System.Management.Automation/engine/Modules/ModuleCmdletBase.cs:line 5529
   at Microsoft.PowerShell.Commands.ImportModuleCommand.ImportModule_LocallyViaName(ImportModuleOptions importModuleOptions, String name) in /PowerShell/src/System.Management.Automation/engine/Modules/ImportModuleCommand.cs:line 706
   at Microsoft.PowerShell.Commands.ImportModuleCommand.ImportModule_LocallyViaName_WithTelemetry(ImportModuleOptions importModuleOptions, String name) in /PowerShell/src/System.Management.Automation/engine/Modules/ImportModuleCommand.cs:line 596
   at Microsoft.PowerShell.Commands.ImportModuleCommand.ProcessRecord() in /PowerShell/src/System.Management.Automation/engine/Modules/ImportModuleCommand.cs:line 1862
   at System.Management.Automation.Cmdlet.DoProcessRecord() in /PowerShell/src/System.Management.Automation/engine/cmdlet.cs:line 173
   at System.Management.Automation.CommandProcessor.ProcessRecord() in /PowerShell/src/System.Management.Automation/engine/CommandProcessor.cs:line 388

Environment data

PS> $psversiontable

Name                           Value
----                           -----
PSVersion                      7.1.3
PSEdition                      Core
GitCommitId                    7.1.3
OS                             Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

PS> get-module Az.Accounts

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     2.8.0                 Az.Accounts

Error output

I don't appear to have this command?

[edyoung]

fyi @isra-fel

[dingmeng-xue]

@edyoung , Thanks for reporting and investigation. We should fix it as soon as possible.

[isra-fel]

Thanks for the feedback! The issue was resolved by #18312
The next release is planned for June 5th. At the meantime, we are working on improving the overall stability of Az to prevent similar issues from happening.

For now, to work-around the issue, we suggest using the previous version of Az v7.5.0 or Az.Accounts v2.7.6.

[edyoung]

Thanks!

[iambernics]

Thanks for the feedback! The issue was resolved by #18312 The next release is planned for June 5th. At the meantime, we are working on improving the overall stability of Az to prevent similar issues from happening.

For now, to work-around the issue, we suggest using the previous version of Az v7.5.0 or Az.Accounts v2.7.6.

Downgrade to 7.5.0 nor 7.4.0 or 7.3.2 isn't solve the issue.

Az 7.5.0 (last Updated: a month ago) depend on Az.ContainerRegistry 2.2.3 (last update: 5/25/2021) which depend on Az.Accounts 2.8.0 (last updated: 16 days ago).
Same with Az 7.4.0 (last updated: 2 months ago) > Az.ContainerRegistry 2.2.3 > Az.Accounts 2.8.0
Same with Az 7.3.2 (last updated: 3 months ago) > Az.ContainerRegistry 2.2.3 > Az.Accounts 2.8.0

As we can realize, sometimes there is no real option to mitigate a bug in a new release of Az with rollback to previous release.

My opinion, this is a critical bug without proper priority.
Pipelines on case-sensitive Linux CI/CD Agents widely-using Az.
Lot of Microsoft Azure DevOps Task working with uppercase environment variable of HTTP_PROXY and NO_PROXY only.
Other Linux basis tools are compatible with only the lowercase variant.
These important variables should be duplicated which cause error in all use-cases of Az.Accounts.
Moreover, AzureCLI@2 ADO Task is mapping the id of the tenant with duplication.

When will be published the fix of the issue with duplicated environment variables in 2.8.0?

[dbkhandelwal]

Hi

When will the new version be published?
The workaround of installing Az v7.5.0. v7.4.0 or 7.3.2 is not working for us. When we try do install Az 7.5.0, the installer keep finding the new version of Az.accounts and installs 2.8.0 for it.

so there is some dependency check somewhere for greater than or equal to 2.7.6 rather then equal to 2.7.6 which is making to go for latest version for az.accounts.

We also tried to install the az.accounts 2.7.6 before installing az 7.3.2 and it still download and install 2.8.0 accounts as part of az 7.3.2

[iambernics]

Hi

When will the new version be published? The workaround of installing Az v7.5.0. v7.4.0 or 7.3.2 is not working for us. When we try do install Az 7.5.0, the installer keep finding the new version of Az.accounts and installs 2.8.0 for it.

so there is some dependency check somewhere to only look for 2.7.6 instead rather then greater than or equal to 2.7.6.

We also tried to install the az.accounts 2.7.6 before installing az 7.3.2 and it still download and install 2.8.0 accounts as part of az 7.3.2

You can remove the Az.Accounts after the installation of Az and install an older version manually:

Install-Module -Name Az.Accounts -RequiredVersion 2.7.6

If you are using AKS, good luck..

[iambernics]

Hi
When will the new version be published? The workaround of installing Az v7.5.0. v7.4.0 or 7.3.2 is not working for us. When we try do install Az 7.5.0, the installer keep finding the new version of Az.accounts and installs 2.8.0 for it.
so there is some dependency check somewhere to only look for 2.7.6 instead rather then greater than or equal to 2.7.6.
We also tried to install the az.accounts 2.7.6 before installing az 7.3.2 and it still download and install 2.8.0 accounts as part of az 7.3.2

You can remove the Az.Accounts after the installation of Az and install an older version manually:

Install-Module -Name Az.Accounts -RequiredVersion 2.7.6

If you are using AKS, good luck..

Almost all functions of Az.Aks already tested with lower version than the original dependency.
There is no side effects of downgrade the Az.Accounts from 2.8.0 to 2.7.6.

Workaround for the issue of Az.Accounts with duplicated environment variables:

Install previous stable release of Az
Install-Module -Name Az -RequiredVersion 7.5.0

Remove the Az.Accounts 2.8.0 (installed automatically with Az)
Uninstall-Module -Name Az.Accounts

Install the previous release
Install-Module -Name Az.Accounts -RequiredVersion 2.7.6