Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derived method 'TryCreateLogger' in type 'OpenTelemetry.Logs.LoggerProviderSdk' from assembly 'OpenTelemetry, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c' cannot reduce access. #5741

Closed
ChuanGoing opened this issue Jul 5, 2024 · 13 comments
Labels
bug Something isn't working pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package

Comments

@ChuanGoing
Copy link

Package

OpenTelemetry

Package Version

Package Name Version
OpenTelemetry.Api 1.9.0
OpenTelemetry.Api.ProviderBuilderExtensions 1.9.0
OpenTelemetry.Instrumentation.EntityFrameworkCore 1.0.0-beta.12

Runtime Version

.NET6.0

Description

When recompiling and running my application, an error occurs during the startup phase:
The derived method 'TryCreateLogger' in type 'OpenTelemetry.Logs.LoggerProviderSdk' from assembly 'OpenTelemetry, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c' cannot reduce access.

Description: The application relies on OpenTelemetry related packages to provide APM functionality. After encountering the error, I compared it with another application (which starts normally), which references OpenTelemetry.Api version 1.8.1. This led me to suspect that there might have been changes in the OpenTelemetry.Api version 1.9.0 update.

Steps to Reproduce

Upgrading from OpenTelemetry.Api 1.8.1 to 1.9.0 can reproduce the issue.

Expected Result

I hope to locate the problem as soon as possible.

Actual Result

The above exception message is encountered.

Additional Context

No response

@ChuanGoing ChuanGoing added the bug Something isn't working label Jul 5, 2024
@github-actions github-actions bot added the pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package label Jul 5, 2024
@Kielek
Copy link
Contributor

Kielek commented Jul 8, 2024

@ChuanGoing, could you please provide us Minimal, Reproducible Example? It can be separate repository with the problematic code. It should not include any internal business logic, just scaffolding of code to reproduce issue.

@jovnas-inet
Copy link

jovnas-inet commented Jul 9, 2024

We have the same problem, using the following packages. (Downgrade to 1.8.X fixes the problem.)
OpenTelemetry.Instrumentation.Runtime v1.9.0
OpenTelemetry.Instrumentation.SqlClient v1.9.0-beta.1

My guess is that this change is the reason:
2710c42#diff-725c4458d2e5e52fc9f80dc087141999cce9d3b213c2fa30ed2488e1cec3fa06L92-R104

EDIT:
If it matters, for us, it is when Autofac tries to build the services that the error occurs.

Unhandled exception. Autofac.Core.DependencyResolutionException: An exception was thrown while activating ?:Microsoft.Extensions.Hosting.IHost -> Microsoft.Extensions.Hosting.Internal.ApplicationLifetime -> Microsoft.Extensions.Logging.Logger`1[[Microsoft.Extensions.Hosting.Internal.ApplicationLifetime, Microsoft.Extensions.Hosting, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]] -> Microsoft.Extensions.Logging.LoggerFactory -> ?:Microsoft.Extensions.Logging.ILoggerProvider[] -> ?:Microsoft.Extensions.Logging.ILoggerProvider.
 ---> System.TypeLoadException: Derived method 'TryCreateLogger' in type 'OpenTelemetry.Logs.LoggerProviderSdk' from assembly 'OpenTelemetry, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c' cannot reduce access.

@jiawbao
Copy link

jiawbao commented Jul 9, 2024

I also met this issue after we added this package :
<PackageVersion Include="OpenTelemetry.Instrumentation.Quartz" Version="1.0.0-beta.3" />

Below are all packages we are using :

    <PackageVersion Include="OpenTelemetry.Audit.Geneva" Version="2.1.1" />

    <PackageVersion Include="OpenTelemetry.Exporter.Console" Version="1.5.1" />

    <PackageVersion Include="OpenTelemetry.Exporter.Geneva" Version="1.5.1" />

    <PackageVersion Include="OpenTelemetry.Extensions.Hosting" Version="1.5.1" />

    <PackageVersion Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.8.1" />

    <PackageVersion Include="OpenTelemetry.Instrumentation.GrpcNetClient" Version="1.8.0-beta.1" />

    <PackageVersion Include="OpenTelemetry.Instrumentation.Http" Version="1.8.1" />

    <PackageVersion Include="OpenTelemetry.Instrumentation.Quartz" Version="1.0.0-beta.3" />`

@damien-c-d
Copy link

We are having the same problem with the following packages installed:

OpenTelemetry.Instrumentation.AspNetCore v1.9.0
OpenTelemetry.Instrumentation.Http v1.9.0
OpenTelemetry.Instrumentation.Runtime v1.9.0

Downgrading them ALL to 1.8.1 fixes the problem. Likewise, upgrading any one of them to 1.9.0 reintroduces the problem.

If this method is not called, then the problem does not occur:

    public static WebApplicationBuilder WithOpenTelemetry(this WebApplicationBuilder builder)
    {
        const string serviceName = "my-web-app";

        builder.Logging.AddApplicationInsights();

        builder.Logging.AddOpenTelemetry(options =>
        {
            options.IncludeScopes = true;
            options.IncludeFormattedMessage = true;

            options.SetResourceBuilder(ResourceBuilder.CreateDefault()
                .AddService(serviceName));
        });

        builder.Services.AddOpenTelemetry()
            .ConfigureResource(resource =>
            {
                resource.AddService(serviceName);
            })
            .WithTracing(tracing =>
            {
                tracing.AddAspNetCoreInstrumentation();
            })
            .WithMetrics(metrics =>
            {
                metrics.AddMeter("My.WebApp");
                metrics.AddAspNetCoreInstrumentation()
                       .AddRuntimeInstrumentation();
            })
            .UseAzureMonitor(options =>
            {
                options.ConnectionString = builder.Configuration["ApplicationInsights:ConnectionString"];
            });

        return builder;
    }

@TimothyMothra
Copy link
Contributor

TimothyMothra commented Jul 11, 2024

I've narrowed this to an incompatibility between OpenTelemetry v1.8.1 and OpenTelemetry.Api.ProviderBuilderExtensions v1.9.0.
On the surface that may look obvious, but these can be transient dependencies from other combinations of packages.

Here's a minimal repro:

WebApplication1.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="OpenTelemetry" Version="1.8.1" />
    <PackageReference Include="OpenTelemetry.Api.ProviderBuilderExtensions" Version="1.9.0" />
  </ItemGroup>

</Project>

Program.cs

namespace WebApplication1
{
    using Microsoft.AspNetCore.Builder;
    using Microsoft.Extensions.DependencyInjection;
    using Microsoft.Extensions.Logging;

    public class Program
    {
        public static void Main(string[] args)
        {
            var builder = WebApplication.CreateBuilder(args);

            builder.Services.AddLogging(logging => logging.AddOpenTelemetry());

            var app = builder.Build();

            app.MapGet("/", () => "Hello World!");

            app.Run();
        }
    }
}

Additional Repro

This combination may be more common for users:

csproj

In this example, these are the latest versions of these two packages.

  <ItemGroup>
    <PackageReference Include="Azure.Monitor.OpenTelemetry.AspNetCore" Version="1.2.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.9.0" />
  </ItemGroup>

Program.cs

In this example, UseAzureMonitor() internally uses builder.Services.AddLogging(logging => logging.AddOpenTelemetry());

builder.Services.AddOpenTelemetry().UseAzureMonitor();

@vishweshbankwar
Copy link
Member

Upgrading OpenTelemetry to 1.9.0 should fix the issue.

@CodeBlanch - FYI

@ChuanGoing
Copy link
Author

I'm sorry for the late response. After encountering the issue last time, I resolved it by downgrading OpenTelemetry.Instrumentation.EntityFrameworkCore to version 1.0.0-beta.11, but I didn't personally investigate the root cause

@damien-c-d
Copy link

damien-c-d commented Jul 14, 2024

@vishweshbankwar That's not a fix. Upgrading the packages I mentioned above to 1.9.0 is what causes this issue for us.

Edit:

These 3 are the only ones we have installed:
OpenTelemetry.Instrumentation.AspNetCore v1.9.0 OpenTelemetry.Instrumentation.Http v1.9.0 OpenTelemetry.Instrumentation.Runtime v1.9.0

All other packages are up to date in the solution.

@CodeBlanch
Copy link
Member

CodeBlanch commented Jul 15, 2024

Lots to unpack here! 🤣

Let's start with root cause. In 1.9.0 the LoggerProvider class inside OpenTelemetry.Api was promoted to public from internal but the Logger class was left internal. This creates issues for LoggerProvider.TryCreateLogger which hands back a Logger instance. TryCreateLogger had to be made internal:

API 1.9.0:

public class LoggerProvider : BaseProvider
{
    internal virtual bool TryCreateLogger(string? name, out Logger? logger) {}
}

API previously:

internal class LoggerProvider : BaseProvider
{
    protected virtual bool TryCreateLogger(string? name, out Logger? logger) {}
}

The SDK (OpenTelemetry.dll) contains an implementation of this type.

SDK 1.9.0:

internal sealed class LoggerProviderSdk : LoggerProvider
{
    internal override bool TryCreateLogger(string? name, out Logger? logger) {}
}

SDK previous:

internal sealed class LoggerProviderSdk : LoggerProvider
{
    protected override bool TryCreateLogger(string? name, out Logger? logger) {}
}

The issue is going to manifest if you have OpenTelemetry.Api.dll >= 1.9.0 and OpenTelemetry.dll < 1.9.0.

The fix is to add a reference to OpenTelemetry (SDK) version 1.9.0 (or greater). A direct reference will win over transitive ones.

This report:

These 3 are the only ones we have installed:
OpenTelemetry.Instrumentation.AspNetCore v1.9.0
OpenTelemetry.Instrumentation.Http v1.9.0
OpenTelemetry.Instrumentation.Runtime v1.9.0

I'm having trouble reconciling that. Because there is no OpenTelemetry (SDK) reference at all in that graph. The goal with all OTel implementations is explicitly that instrumentation libraries should NOT need an SDK reference. If you only have those 3 packages, there is no "Sdk.Create*" API available or AddOpenTelemetry API available to even turn on OTel. There must be some other reference brining in OpenTelemetry at an older version.

My guess would be it is something more like...

OpenTelemetry.Extensions.Hosting v1.8.1 <- Issue is here
OpenTelemetry.Instrumentation.AspNetCore v1.9.0
OpenTelemetry.Instrumentation.Http v1.9.0
OpenTelemetry.Instrumentation.Runtime v1.9.0

Getting the SDK transitively is the issue here.

OpenTelemetry.Extensions.Hosting brings in the SDK. So if you are using that type of reference structure (getting the SDK transitively), make sure to bump OpenTelemetry.Extensions.Hosting:

OpenTelemetry.Extensions.Hosting v1.9.0
OpenTelemetry.Instrumentation.AspNetCore v1.9.0
OpenTelemetry.Instrumentation.Http v1.9.0
OpenTelemetry.Instrumentation.Runtime v1.9.0

Or do the direct reference.

What you want to see in your graph is SDK + API versions matching:

Bad:

image

Good:

image

Also good (direct reference added to SDK):

image

What could be done to improve this?

We have run into other issues before with users on newer instrumentation libraries and older SDKs. @rajkumar-rangaraj @vishweshbankwar @Kielek The way our CI is set up we always test SDK using latest API. Perhaps we should have some steps which test latest API packages using previous SDK? We sort of assume users will be on same version of API & SDK but the way instrumentation libraries work it is easy for graphs to become mismatched.

@Kielek
Copy link
Contributor

Kielek commented Jul 16, 2024

@CodeBlanch, correct me if I am wrong, but it is the root cause of the issue:

[assembly: InternalsVisibleTo("OpenTelemetry" + AssemblyInfo.PublicKey)]
.

I suppose that we could add some msbuild task to detect such cases and rise warning if there is incompatibility version between OpenTelemetry.Api and OpenTelemetry. The other option is to have more strict requirements (support for exact versions) on the nuget package level. In such case we should cover all new releases.

@xaviergxf
Copy link

I'm also having this issue, can this be reopened?

@CodeBlanch
Copy link
Member

@Kielek

@CodeBlanch, correct me if I am wrong, but it is the root cause of the issue:

InternalsVisibleTo is part of it but isn't the root cause. If we didn't use InternalsVisibleTo we wouldn't have this issue, but we also wouldn't have experimental APIs and we would need a lot of other internal APIs to be exposed public. We definitely abuse InternalsVisibleTo 😄

The issue is something which was previously protected became internal in 1.9.0 so the type system is complaining. Basically a breaking change to an API contract but it slipped through our tooling because of the internal nature.

The other option is to have more strict requirements (support for exact versions) on the nuget package level. In such case we should cover all new releases.

I'm not opposed to this but I would be concerned the net result long term would be users getting stuck on old versions. We can discuss more on SIG?

@xaviergxf

I'm also having this issue, can this be reopened?

We could reopen this but I'm not sure what could be done. To fix update your graph so API + SDK packages are on the same version. Let's say we release a new OpenTelemetry.Api with some fix. You would need a package upgrade for that so the process would look pretty much the same 😄 I'll mess with things to see if there is a possible way to even fix it through API upgrade.

@ggimenes
Copy link

@Kielek

@CodeBlanch, correct me if I am wrong, but it is the root cause of the issue:

InternalsVisibleTo is part of it but isn't the root cause. If we didn't use InternalsVisibleTo we wouldn't have this issue, but we also wouldn't have experimental APIs and we would need a lot of other internal APIs to be exposed public. We definitely abuse InternalsVisibleTo 😄

The issue is something which was previously protected became internal in 1.9.0 so the type system is complaining. Basically a breaking change to an API contract but it slipped through our tooling because of the internal nature.

The other option is to have more strict requirements (support for exact versions) on the nuget package level. In such case we should cover all new releases.

I'm not opposed to this but I would be concerned the net result long term would be users getting stuck on old versions. We can discuss more on SIG?

@xaviergxf

I'm also having this issue, can this be reopened?

We could reopen this but I'm not sure what could be done. To fix update your graph so API + SDK packages are on the same version. Let's say we release a new OpenTelemetry.Api with some fix. You would need a package upgrade for that so the process would look pretty much the same 😄 I'll mess with things to see if there is a possible way to even fix it through API upgrade.

I just faced this problem trying to use OpenTelemetry.Instrumentation.StackExchangeRedis version 1.9.0-beta.1
The application completely stopped sending traces to Jaeger 😢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package
Projects
None yet
Development

No branches or pull requests

10 participants