Fix to #33073 - JSON columns throws Invalid token type: 'StartObject' exception with AsNoTrackingWithIdentityResolution()

[maumar]

Problem was that we were treating NoTrackingWithIdentityResolution queries as regular NoTracking in the context of JSON queries. However due to how JSON is materialized (streaming + nested includes are part of the parent materialization, rather than each entity materialized separately) we have special provisions when ChangeTracker is being used. In NoTrackingWithIdentityResolution, ChangeTracker is being used but the materializer didn't adjust to that, which lead to errors.

Fix is to generate JSON materializer code based on whether query uses Change Tracker rather than if it's a Tracking/NoTracking query.

Fixes #33073

[maumar]

there is a problem with this fix, some tests regress into data corruption (from invalid token now), need to investigate in-depth.

failing tests for NTWIR

Json_branch_collection_distinct_and_other_collection - NRE
Json_collection_distinct_in_projection - NRE
Json_collection_filter_in_projection - NRE
Json_collection_SelectMany - unable to track
Json_collection_skip_take_in_projection - NRE
Json_multiple_collection_projections - NRE
Json_nested_collection_anonymous_projection_in_projection - DATA CORRUPTION
Json_nested_collection_filter_in_projection - NRE
Json_nested_collection_SelectMany - unable to track
Json_projection_deduplication_with_collection_in_original_and_collection_indexer_in_target - DATA CORRUPTION
Json_projection_deduplication_with_collection_indexer_in_target - DATA CORRUPTION

[maumar]

@roji @ajcvickers new version up

[aldrashan]

Semi-related: the same error gets thrown when it tries to deserialize a json object as a list of objects.

 public class Engravings
 {
     public Guid UidProduction { get; set; }
     public List<Engraving_Files>? Files { get; set; }
 }

modelBuilder.Entity<Engravings>(entity =>
 {
     entity.HasKey(x => x.UidProduction);
     entity.OwnsMany(x => x.Files, opt => { opt.ToJson(); });
 });

-> Database is expected to return something like the following for the Files property: [ {"id":"1"}, {"id":"2"} ]
If the database value is actually not an array, because of manual db manipulations, e.g. {"id":"3"}, the exception gets thrown.

Meanwhile, if you would try to deserialize that incorrect object as a List via Newtonsoft (haven't tried with System.Text.Json), you'd get a much clearer Exception message.

[ajcvickers]

@maumar Remind me again why this ordering is not needed for tracking queries?

[maumar]

@ajcvickers you can't project a "naked" owned type in a tracking query scenario. #31047, so we always have the parent entities and they include everything in the correct order.