Upgrade to Confluent.SchemaRegistry.Serdes.Json 2.6.1 breaks serialisation

[TimButterfield]

We've created a library (net standard 2.0) that wraps the Confluent C# Kafka library and all was working until we recently upgraded to use the Confluent.SchemaRegistry.Serdes.Json library. The issues persists from version 2.6.1 onwards. We upgraded to this version as dependabot warned us about a denial of service vulnerability in Microsoft.Extensions.Caching.Memory in version 8.0.0

After the upgrade our stack continues to compile but we get runtime issues. The error occurs when we try to produce a message from a project that is targeting .net 8. For clarity, if we call into our net standard 2.0 wrapper from a project targeting .net 6.0, everything works fine. It's only when we're invoking our wrapper from a .net 8.x project that we get the following runtime error.

System.MissingMethodException
Method not found: 'Void Confluent.SchemaRegistry.Serdes.JsonValueSerializer'1..ctor(Confluent.SchemaRegistry.ISchemaRegistryClient, Confluent.SchemaRegistry.Serdes.JsonSerializerConfig, NJsonSchema.Generation.JsonSchemaGeneratorSettings, Confluent.SchemaRegistry.RuleRegistry)

If i change the wrapper library to target .net 6 and call it from a .net 8 project, I get a different error.

System.MissingMethodException
Method not found: 'Newtonsoft.Json.JsonSerializerSettings NJsonSchema.GenerationJsonSchemaGeneratorSettings.get_ActualSerilizerSettings

The invocation of this comes from https://github.com/confluentinc/confluent-kafka-dotnet/blob/master/src/Confluent.SchemaRegistry.Serdes.Json/JsonSerializer.cs#L85 which is called from https://github.com/confluentinc/confluent-kafka-dotnet/blob/master/src/Confluent.SchemaRegistry.Serdes.Json/JsonSerializer.cs#L255.

Our code is setting up the producer by setting up the DependentProducerBuilder before calling build, as so.

private IProducer<Tkey, TEvent> Initialise(
      DependentProducerBuilder<TKey, TEvent> builder, 
      ISchemaRegistryClient schemaRegistryClient)
{
      builder.SetValueSerializer(new JsonSerialzer<TEvent>(schemaRegistryClient)); 
}

I checked out https://github.com/confluentinc/confluent-kafka-dotnet/releases/tag/v2.6.1 of the code to see if I can identify the issue, but have only got as far as noting that it was introduced when supporting different dependencies as part of the change to support .net 8. The pre processor directives and the invocation call stack when being invoked from .net 8 via a net standard library is presently causing us problems.

Incidentally, we're having to target net standard 2 as a library as my client has some legacy apps still running on .net framework (4.7.2) with new builds targeting .net 8. There may be some net 6 apps using the library in time as well.

The events I've tested the above with are very simple POCO's and I've replicated the above with a very simple wrapper around that simply creates an IProducer from a dependent producer builder.

Would appreciate any assistance you can offer on this as we can't upgrade to 2.6.1 or beyond atm.