[Bug] Calling TokenAcquisition.GetAccessTokenForUserAsync throws a NullReferenceException

[davidkarlsson]

Which version of Microsoft Identity Web are you using?
1.16.1

Where is the issue?

• Web app
  • Sign-in users
  • Sign-in users and call web APIs
• Web API
  • Protected web APIs (validating tokens)
  • Protected web APIs (validating scopes)
  • Protected web APIs call downstream web APIs
• Token cache serialization
  • In-memory caches
  • Session caches
  • Distributed caches
• Other (please describe)

Is this a new or an existing app?
The app is in production and I have upgraded to a new version of Microsoft Identity Web. This error started happening after upgrading from 1.15.2 to 1.16.1. I've verified this by downgrading the package to 1.15.2 again.

I think maybe this issue is related to #1372 but it has already been closed so I opened a new issue for this.

Repro

public class Startup
{
   public void ConfigureServices(IServiceCollection services)
   {
        services.AddAuthentication()
            .AddMicrosoftIdentityWebApp(azureAdSection, cookieScheme: null)
            .EnableTokenAcquisitionToCallDownstreamApi().AddInMemoryTokenCaches();
   }

    public void Configure(IApplicationBuilder app)
    {
        app.UseAuthentication();
        app.UseAuthorization();
    }
}

public class BaseController : Controller
{
    private readonly SignInManager<User> _signInManager;
    private readonly ITokenAcquisition _tokenAcquisition;
    private readonly IGraphApiClient _graphApiClient;

    public BaseController(
        SignInManager<User> signInManager,        
        ITokenAcquisition tokenAcquisition,
        IGraphApiClient graphApiClient
    )
    {
        _signInManager = signInManager;        
        _tokenAcquisition = tokenAcquisition;
        _graphApiClient = graphApiClient;
    }

    [Route("ExternalLoginCallback")]
    public async Task<IActionResult> ExternalLoginCallback(
        string returnUrl,
        string? remoteError = null
    )
    {
        var info = await _signInManager.GetExternalLoginInfoAsync();

        var accessToken = await _tokenAcquisition.GetAccessTokenForUserAsync(
            new[] { "User.Read" },
            user: info.Principal
        );
    }
}

Expected behavior
I expected ITokenAcquisition.GetAccessTokenForUserAsync() to return an access token like it did in 1.15.2 instead of throwing a NullReferenceException.

Actual behavior
The call to ITokenAcquisition.GetAccessTokenForUserAsync() throws the following exception:

System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Identity.Web.MergedOptions.PrepareAuthorityInstanceForMsal()
   at Microsoft.Identity.Web.TokenAcquisition.BuildConfidentialClientApplication(MergedOptions mergedOptions)
   at Microsoft.Identity.Web.TokenAcquisition.GetOrBuildConfidentialClientApplication(MergedOptions mergedOptions)
   at Microsoft.Identity.Web.TokenAcquisition.GetAuthenticationResultForUserAsync(IEnumerable`1 scopes, String authenticationScheme, String tenantId, String userFlow, ClaimsPrincipal user, TokenAcquisitionOptions tokenAcquisitionOptions)
   at Microsoft.Identity.Web.TokenAcquisition.GetAccessTokenForUserAsync(IEnumerable`1 scopes, String authenticationScheme, String tenantId, String userFlow, ClaimsPrincipal user, TokenAcquisitionOptions tokenAcquisitionOptions)
   at TEST.Web.Controllers.BaseController.ExternalLoginCallback(String returnUrl, String remoteError) in D:\Code\DotNet\Test\TEST.Web\Controllers\BaseController.cs:line 159
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

[jennyf19]

@davidkarlsson can you provide a link to your code or a basic repro i can run locally? thank you.

[davidkarlsson]

@jennyf19 I've written a repro you can run locally here now: https://github.com/davidkarlsson/identity-web-token-issue.

[lsiepel]

Same issue here. Downgraded Microsoft.Identity.Web 16.1 to 16.0 and all is fine.

System.NullReferenceException: Object reference not set to an instance of an object.
    at  Microsoft.Identity.Web.MergedOptions.PrepareAuthorityInstanceForMsal (Microsoft.Identity.Web, Version=1.16.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae)
    at  Microsoft.Identity.Web.TokenAcquisition.BuildConfidentialClientApplication (Microsoft.Identity.Web, Version=1.16.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae)
    at  Microsoft.Identity.Web.TokenAcquisition.GetOrBuildConfidentialClientApplication (Microsoft.Identity.Web, Version=1.16.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae)
    at  Microsoft.Identity.Web.TokenAcquisition+<GetAuthenticationResultForUserAsync>d__16.MoveNext (Microsoft.Identity.Web, Version=1.16.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae)
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  Microsoft.Identity.Web.TokenAcquisitionAuthenticationProvider+<AuthenticateRequestAsync>d__3.MoveNext (Microsoft.Identity.Web.MicrosoftGraph, Version=1.16.1.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae)
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  Microsoft.Graph.AuthenticationHandler+<SendAsync>d__16.MoveNext (Microsoft.Graph.Core, Version=2.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Net.Http.HttpClient+<FinishSendAsyncBuffered>d__70.MoveNext (System.Net.Http, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
    at  Microsoft.Graph.HttpProvider+<SendRequestAsync>d__19.MoveNext (Microsoft.Graph.Core, Version=2.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)

Startup.cs:

services.AddAuthentication()
                .AddMicrosoftIdentityWebApp(options =>
                {
                    options.ResponseType = OpenIdConnectResponseType.CodeIdToken;
                    options.Instance = Configuration["AzureAd:Instance"];
                    options.TenantId = Configuration["AzureAd:TenantId"];
                    options.ClientId = Configuration["AzureAd:ClientId"];
                    options.CallbackPath = Configuration["AzureAd:CallbackPath"];
                    options.ClientSecret = Configuration["AzureAd:ClientSecret"];
                    options.Events.OnAccessDenied = OnAccessDenied;
                    options.Events.OnTokenValidated = OnTokenValidated;
                    options.Events.OnRemoteSignOut = OnRemoteSignOut;
                    options.Events.OnSignedOutCallbackRedirect = OnSignedOutCallbackRedirect;
                    options.Events.OnRedirectToIdentityProvider = OnRedirectToIdentityProvider;
                })
                .EnableTokenAcquisitionToCallDownstreamApi()
                .AddMicrosoftGraph(defaultScopes: GraphScopes)
                .AddDistributedTokenCaches();

            services.AddAuthentication()
                .AddMicrosoftIdentityWebApi(Configuration.GetSection("AzureAd"))
                .EnableTokenAcquisitionToCallDownstreamApi()
                .AddMicrosoftGraph(defaultScopes: GraphScopes)
                .AddDistributedTokenCaches();

            services.AddDistributedSqlServerCache(options =>
            {
                options.ConnectionString = _connectionString;
                options.SchemaName = "dbo";
                options.TableName = "TokenCache";
                options.DefaultSlidingExpiration = TimeSpan.FromMinutes(90);
            });

Unfortunately i'm unable to provide a runnable demo at this moment, but if the provided details are not enough, i could try to spend some time later this week.

When same code is used with package 16.0 all is fine.

[jennyf19]

@lsiepel can you include a default authentication scheme? Which authentication scheme do you have in your controller, when you call token acquisition?

[lsiepel]

Ah sorry, the authentication schemepart was left out. Both schemes are used in the controllers, both lead to the same problem

services.AddAuthentication(sharedOptions =>
            {
                sharedOptions.DefaultScheme = "smart";
                sharedOptions.DefaultChallengeScheme = "smart";
            })
             .AddPolicyScheme("smart", "Authorization Bearer or OIDC", options =>
             {
                 options.ForwardDefaultSelector = context =>
                 {
                     string authHeader = context.Request.Headers["Authorization"].FirstOrDefault();
                     if (authHeader?.StartsWith("Bearer ") == true)
                     {
                         return JwtBearerDefaults.AuthenticationScheme;
                     }
                     return OpenIdConnectDefaults.AuthenticationScheme;
                 };
             });

[GeorgeTTD]

Hi @davidkarlsson,
Not sure if this is any help but I have found that using AuthorizeForScopesAttribute has solved a similar sounding issue for me. Use [AuthorizeForScopes] on the pages you wish to use token acquisition or add it as a filter if you want to apply to all routes.

[davidkarlsson]

Hey @GeorgeTTD! Thanks for the suggestion but as far as I can tell I don't really want the functionality the [AuthorizeForScopes] attribute provides. The scopes I'm using either don't require incremental consent or has already been authorized by an admin for the application in Azure AD.

Also calling ITokenAcquisition.GetAccessTokenForUserAsync() is working fine from the controller action without this attribute in 1.15.2 but throws a NullReferenceException in 1.16.1 so I think it is meant to work without it.

[lsiepel]

Will be testing if this issue still persists for the latest version. Is there anything i can do to help fix this issue?

[davidkarlsson]

@jennyf19 Were you able to reproduce this error with 1.16.1 or later with the test repo I created?

[Choonster]

Similar to #1385, this seems to be caused by the options merge not being triggered. I worked around this by getting the IOptionsMonitor<OpenIdConnectOptions> instance (from HttpContext.RequestServices in my case) and calling Get(OpenIdConnectDefaults.AuthenticationScheme) to trigger the merge.

[jennyf19]

sorry for the delay in responses. will take a look.

[jmprieur]

See PR: #1672

[jennyf19]

Released in 1.24.0.

[davidkarlsson]

After having updated from 1.15.2 to 1.24.1 I now have to pass OpenIdConnectDefaults.AuthenticationScheme as the authenticationScheme parameter to GetAccessTokenForUserAsync for this to work otherwise it throws this new exception the PR added:

System.InvalidOperationException: IDW10503: Cannot determine the cloud Instance. The provided authentication scheme was ''. Microsoft.Identity.Web inferred 'Identity.Application' as the authentication scheme. Available authentication schemes are 'Identity.Application,Identity.External,Identity.TwoFactorRememberMe,Identity.TwoFactorUserId,OpenIdConnect'. See https://aka.ms/id-web/authSchemes. 
   at Microsoft.Identity.Web.TokenAcquisition.GetOptions(String authenticationScheme, String& effectiveAuthenticationScheme)

Is this a required parameter now or is this a bug? The documentation for the parameter says "Authentication scheme. If null, will use OpenIdConnectDefault.AuthenticationScheme if called from a web app..." so I thought it should work without passing a scheme to it? There's also a spelling mistake there as it should say OpenIdConnectDefaults.AuthenticationScheme and not OpenIdConnectDefault.AuthenticationScheme.

I'm not sure if this matters but I'm not passing OpenIdConnect as the default authentication scheme when I configure authentication because I'm using both cookie and OpenID Connect authentication:

builder.Services.AddAuthentication()
    .AddMicrosoftIdentityWebApp(azureAdSettings, cookieScheme: null)
    .EnableTokenAcquisitionToCallDownstreamApi()
    .AddInMemoryTokenCaches();

[FelipeCostaGualberto]

Hi, I lost a lot of time with this problem because the latest version (1.25.0) doesn't work with JwtBearerDefaults scheme. Downgrading to 1.16.0 made it work. Please fix this. This is how I register my service:

    var scopes = new List<string>(configuration["AzureAd:Scopes"].Split(new char[] { ' ' }));
    services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.RequireHttpsMetadata = false;
        options.SaveToken = true;
        options.TokenValidationParameters = new TokenValidationParameters()
        {
            NameClaimType = "sub",
            RoleClaimType = "role",
            ValidateIssuerSigningKey = true,
            IssuerSigningKey = new SymmetricSecurityKey(Encoding.ASCII.GetBytes(configuration["Jwt:SecurityKey"])),
            ValidateIssuer = false,
            ValidateAudience = false,
            ClockSkew = TimeSpan.Zero,
        };
    })
    .AddMicrosoftIdentityWebApp(configuration)
        .EnableTokenAcquisitionToCallDownstreamApi(scopes)
        .AddDistributedTokenCaches();
    }

[jmprieur]

What does not work @FelipeCostaGualberto
Can you please provide a full repro?

@davidkarlsson.

this a required parameter now or is this a bug?
It's a required parameter when you are not using the default authentication scheme.

The documentation for the parameter says "Authentication scheme. If null, will use OpenIdConnectDefault.AuthenticationScheme if called from a web app..." so I thought it should work without passing a scheme to it? There's also a spelling mistake there as it should say OpenIdConnectDefaults.AuthenticationScheme and not OpenIdConnectDefault.AuthenticationScheme.

Thanks for the update.

[FelipeCostaGualberto]

Hi @jmprieur , as soon as I can create a minimal project I'll provide a repo, thanks for investigating the issue.
What not work is: if I use Microsoft.Identity.Web 1.25.0, the method ITokenAcquisition.GetAccessTokenForUserAsync throws NullReferenceException. If you use version 1.16.0, it works normally.

[xperiandri]

Here is the issue

microsoft-identity-web/src/Microsoft.Identity.Web/ITokenAcquisition.cs

Line 41 in 34cfae8

null,

[rkd-xom]

Is this fixed? I'm facing the same issue and I am using JwtBearerDefaults scheme as well. I tried using OpenIdConnectDefaults.AuthenticationScheme as well and got the same error System.InvalidOperationException: IDW10503: Cannot determine the cloud Instance. The provided authentication scheme was ''. I'm using version 1.25.1

[dawid-kraczkowski]

Ah sorry, the authentication schemepart was left out. Both schemes are used in the controllers, both lead to the same problem

services.AddAuthentication(sharedOptions =>
            {
                sharedOptions.DefaultScheme = "smart";
                sharedOptions.DefaultChallengeScheme = "smart";
            })
             .AddPolicyScheme("smart", "Authorization Bearer or OIDC", options =>
             {
                 options.ForwardDefaultSelector = context =>
                 {
                     string authHeader = context.Request.Headers["Authorization"].FirstOrDefault();
                     if (authHeader?.StartsWith("Bearer ") == true)
                     {
                         return JwtBearerDefaults.AuthenticationScheme;
                     }
                     return OpenIdConnectDefaults.AuthenticationScheme;
                 };
             });

I solved this problem by implementing custom AuthenticationSchemeProvider according to this thread: https://stackoverflow.com/questions/46464469/how-to-configureservices-authentication-based-on-routes-in-asp-net-core-2-0