Improve Execution Performance (#1392)

(note: descriptions that are quoted were intially authored by GitHub
Copilot and modified for brevity, correctness, and conciseness.
@seantleonard)

## Why this change?

Take advantage from Hot Chocolates execution pipeline to improve
execution of pre-acquired data. Without this change, there exists
inefficiencies in `JsonDocument` object allocations and disposal,
inefficiencies in GraphQL result processing in the `ResolverMiddleware`
class

## What is this change?

+ Introduction of `ResolverTypeInterceptor.cs` class which 
+ Introduction of `ExecutionHelper.cs` class w

+ Introduction of `ArrayPoolWriter.cs` class
> The `ArrayPoolWriter.cs` file provides a way to manage and interact
with pooled arrays, mainly byte arrays, in a more efficient and
controlled manner. Here are some key benefits:

> **Memory Efficiency**: It uses `ArrayPool<byte>`, a shared pool of
byte arrays. This is an efficient way to handle memory because it reuses
arrays, thereby reducing the frequency of garbage collection and the
memory footprint of the application.
>
> **Buffer Management**: The class ArrayPoolWriter implements the
`IBufferWriter<byte>` interface, providing a standard way to write to
the buffer. It also implements the `IDisposable` interface, ensuring
proper cleanup of resources.
>
> **Exception Handling**: The class includes robust error handling,
throwing exceptions when methods are used incorrectly, such as
attempting to advance the buffer past its capacity or trying to use the
writer after it has been disposed.
>
> **Dynamic Buffer Expansion**: If the required capacity exceeds the
current buffer's capacity, it automatically rents a larger buffer from
the pool, copies existing data, and returns the original buffer,
ensuring that the buffer size can grow as needed.
> Access Flexibility: It provides methods to access written data as both
`ReadOnlyMemory<byte>` and `ReadOnlySpan<byte>`, offering flexible ways
to interact with the data.
>
> This class can be beneficial in scenarios where you anticipate writing
to a byte buffer repeatedly or where the amount of data to write may
grow over time.

+ JsonObjectExtensions.cs
> **Easy Conversion**: Provides a straightforward way to convert
`JsonObject` instances to `JsonElement` or `JsonDocument`.
>
> **Memory Optimization**: The conversion process leverages the
ArrayPoolWriter for writing the JsonObject to a pooled buffer, which
avoids the need for serializing to a full JSON string. This can
potentially save memory and improve performance.

+ SqlQueryEngine updates
> The changes in the `SqlQueryEngine.cs` file can be summarized as
follows:
>
> The `ResolveInnerObject` method is now renamed to `ResolveObject`. The
new method handles the case where the JsonElement is of
JsonValueKind.String type and requires parsing into a JSON object. It
also handles the case where the JsonElement is of JsonValueKind.Object
type, directly returning the element.
>
> The `ResolveListType` method is now renamed to `ResolveList`. It has
been refactored to handle two types of `JsonElement`:
`JsonValueKind.Array` and `JsonValueKind.String`. It deserializes both
types into a list of JsonElements. The method now also checks and
handles the case where metadata is not null.
>
> The `public JsonDocument? ResolveInnerObject(JsonElement element,
IObjectField fieldSchema, ref IMetadata metadata)` method signature has
been changed to `public JsonElement ResolveObject(JsonElement element,
IObjectField fieldSchema, ref IMetadata metadata)` to account for no
longer passing back and forth `JsonDocument` objects.
>
> The `public object? ResolveListType(JsonElement element, IObjectField
fieldSchema, ref IMetadata metadata)` method signature has been changed
to `public object ResolveList(JsonElement array, IObjectField
fieldSchema, ref IMetadata? metadata)` to account for no longer passing
back and forth `JsonDocument` objects.
>
> The `ResolveObject` and `ResolveList` methods now have detailed
summary comments explaining their functionality.
An extra check for
`parentMetadata.Subqueries.TryGetValue(QueryBuilder.PAGINATION_FIELD_NAME,
out PaginationMetadata? paginationObjectMetadata)` has been added in the
`ResolveObject` method. If true, `parentMetadata` is updated with
`paginationObjectMetadata`.
>
> Overall, these changes focus on handling the cases where `JsonElement`
is in the form of a string and needs to be parsed into a JSON object or
array, as well as improving the metadata handling.

ExecutionHelper.cs
> The `ExecutionHelper` class contains methods that are primarily used
for interacting with the query engine and mutation engine to execute
queries and mutations, respectively.

> `ExecuteQueryAsync(...)`: Represents the root query resolver and
fetches initial data from the query engine. It accepts a context
parameter and uses the query engine to execute the query. If the
selection type is a list, it will execute a list query, register a
cleanup action to dispose the documents after the query, and set the
result. If not, it will execute a single item query and set the result.
>
> `ExecuteMutateAsync(...)`: Represents the root mutation resolver and
invokes the mutation on the query engine. Similar to the
ExecuteQueryAsync method, it handles both list and single item
mutations.
>
> The `ExecuteLeafField`, `ExecuteObjectField`, `ExecuteListField`
methods are HotChocolate coined "pure resolvers" used to resolve the
results of specific types of fields in the GraphQL request.
>
> The `GetMetadata`, `GetMetadataObjectField`, `SetNewMetadata`,
`SetNewMetadataChildren` methods handling storing and retreiving
SqlPaginationMetadata objects utilized by each level of depth in the
path of a GraphQL result.

- Added fix for DW JSON result differing from MSSQL json result. DW has
extraneous escape characters in the json which results in
JsonDocument/Element not resolving the expected JsonValueKind, and
requires additional JsonDocument.Parse operation to remediate.

## Testing

- Unit testing added for the ArrayPoolWriter functionality. This was
adopted from HotChocolate code.
- GraphQL integration tests that already existed were used to ensure the
refactor maintained the ability to handle request scenarios we test for.
There were tests that initially failed that were used to help validate
SqlPaginationMetadata usage, storage, and creation within the
PureResolver context. The following test classes are directly applicable
for testing query execution through the PureResolver code paths
established in `ExecutionHelper`
  - `Azure.DataApiBuilder.Service.Tests.SqlTests.GraphQLFilterTests`
  - `Azure.DataApiBuilder.Service.Tests.SqlTests.GraphQLMutationTests`
  - `Azure.DataApiBuilder.Service.Tests.SqlTests.GraphQLPaginationTests`
  - `Azure.DataApiBuilder.Service.Tests.SqlTests.GraphQLQueryTests`
-
`Azure.DataApiBuilder.Service.Tests.SqlTests.GraphQLSupportedTypesTests`

---------

Co-authored-by: Sean Leonard <sean.leonard@microsoft.com>

src/Core/Models/GraphQLFilterParsers.cs

@@ -11,6 +11,7 @@
 using Azure.DataApiBuilder.Service.Exceptions;
 using Azure.DataApiBuilder.Service.GraphQLBuilder.Directives;
 using Azure.DataApiBuilder.Service.GraphQLBuilder.Queries;
+using Azure.DataApiBuilder.Service.Services;
 using HotChocolate.Language;
 using HotChocolate.Resolvers;
 using Microsoft.AspNetCore.Http;
@@ -65,12 +66,12 @@ public Predicate Parse(
         string dataSourceName = _configProvider.GetConfig().GetDataSourceNameFromEntityName(entityName);
         ISqlMetadataProvider metadataProvider = _metadataProviderFactory.GetMetadataProvider(dataSourceName);
 
-        InputObjectType filterArgumentObject = ResolverMiddleware.InputObjectTypeFromIInputField(filterArgumentSchema);
+        InputObjectType filterArgumentObject = ExecutionHelper.InputObjectTypeFromIInputField(filterArgumentSchema);
 
         List<PredicateOperand> predicates = new();
         foreach (ObjectFieldNode field in fields)
         {
-            object? fieldValue = ResolverMiddleware.ExtractValueFromIValueNode(
+            object? fieldValue = ExecutionHelper.ExtractValueFromIValueNode(
                 value: field.Value,
                 argumentSchema: filterArgumentObject.Fields[field.Name.Value],
                 variables: ctx.Variables);
@@ -85,7 +86,7 @@ public Predicate Parse(
             bool fieldIsAnd = string.Equals(name, $"{PredicateOperation.AND}", StringComparison.OrdinalIgnoreCase);
             bool fieldIsOr = string.Equals(name, $"{PredicateOperation.OR}", StringComparison.OrdinalIgnoreCase);
 
-            InputObjectType filterInputObjectType = ResolverMiddleware.InputObjectTypeFromIInputField(filterArgumentObject.Fields[name]);
+            InputObjectType filterInputObjectType = ExecutionHelper.InputObjectTypeFromIInputField(filterArgumentObject.Fields[name]);
             if (fieldIsAnd || fieldIsOr)
             {
                 PredicateOperation op = fieldIsAnd ? PredicateOperation.AND : PredicateOperation.OR;
@@ -509,7 +510,7 @@ private Predicate ParseAndOr(
         List<PredicateOperand> operands = new();
         foreach (IValueNode field in fields)
         {
-            object? fieldValue = ResolverMiddleware.ExtractValueFromIValueNode(
+            object? fieldValue = ExecutionHelper.ExtractValueFromIValueNode(
                 value: field,
                 argumentSchema: argumentSchema,
                 ctx.Variables);
@@ -598,11 +599,11 @@ public static Predicate Parse(
     {
         List<PredicateOperand> predicates = new();
 
-        InputObjectType argumentObject = ResolverMiddleware.InputObjectTypeFromIInputField(argumentSchema);
+        InputObjectType argumentObject = ExecutionHelper.InputObjectTypeFromIInputField(argumentSchema);
         foreach (ObjectFieldNode field in fields)
         {
             string name = field.Name.ToString();
-            object? value = ResolverMiddleware.ExtractValueFromIValueNode(
+            object? value = ExecutionHelper.ExtractValueFromIValueNode(
                 value: field.Value,
                 argumentSchema: argumentObject.Fields[field.Name.Value],
                 variables: ctx.Variables);

src/Core/Resolvers/ArrayPoolWriter.cs

@@ -0,0 +1,188 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+using System.Buffers;
+
+/// <summary>
+/// A helper to write to pooled arrays.
+/// </summary>
+internal sealed class ArrayPoolWriter : IBufferWriter<byte>, IDisposable
+{
+    private const int INITIAL_BUFFER_SIZE = 512;
+    private byte[] _buffer;
+    private int _capacity;
+    private int _start;
+    private bool _disposed;
+
+    /// <summary>
+    /// Initializes a new instance of the <see cref="ArrayPoolWriter"/> class.
+    /// </summary>
+    public ArrayPoolWriter()
+    {
+        _buffer = ArrayPool<byte>.Shared.Rent(INITIAL_BUFFER_SIZE);
+        _capacity = _buffer.Length;
+        _start = 0;
+    }
+
+    /// <summary>
+    /// Gets the part of the buffer that has been written to.
+    /// </summary>
+    /// <returns>
+    /// A <see cref="ReadOnlyMemory{T}"/> of the written portion of the buffer.
+    /// </returns>
+    public ReadOnlyMemory<byte> GetWrittenMemory()
+        => _buffer.AsMemory()[.._start];
+
+    /// <summary>
+    /// Gets the part of the buffer that has been written to.
+    /// </summary>
+    /// <returns>
+    /// A <see cref="ReadOnlySpan{T}"/> of the written portion of the buffer.
+    /// </returns>
+    public ReadOnlySpan<byte> GetWrittenSpan()
+        => _buffer.AsSpan()[.._start];
+
+    /// <summary>
+    /// Advances the writer by the specified number of bytes.
+    /// </summary>
+    /// <param name="count">
+    /// The number of bytes to advance the writer by.
+    /// </param>
+    /// <exception cref="ArgumentOutOfRangeException">
+    /// Thrown if <paramref name="count"/> is negative or
+    /// if <paramref name="count"/> is greater than the
+    /// available capacity on the internal buffer.
+    /// </exception>
+    public void Advance(int count)
+    {
+        if (_disposed)
+        {
+            throw new ObjectDisposedException(nameof(ArrayPoolWriter));
+        }
+
+        if (count < 0)
+        {
+            throw new ArgumentOutOfRangeException(nameof(count));
+        }
+
+        if (count > _capacity)
+        {
+            throw new ArgumentOutOfRangeException(nameof(count), count, "Cannot advance past the end of the buffer.");
+        }
+
+        _start += count;
+        _capacity -= count;
+    }
+
+    /// <summary>
+    /// Gets a <see cref="Memory{T}"/> to write to.
+    /// </summary>
+    /// <param name="sizeHint">
+    /// The minimum size of the returned <see cref="Memory{T}"/>.
+    /// </param>
+    /// <returns>
+    /// A <see cref="Memory{T}"/> to write to.
+    /// </returns>
+    /// <exception cref="ArgumentOutOfRangeException">
+    /// Thrown if <paramref name="sizeHint"/> is negative.
+    /// </exception>
+    public Memory<byte> GetMemory(int sizeHint = 0)
+    {
+        if (_disposed)
+        {
+            throw new ObjectDisposedException(nameof(ArrayPoolWriter));
+        }
+
+        if (sizeHint < 0)
+        {
+            throw new ArgumentOutOfRangeException(nameof(sizeHint));
+        }
+
+        int size = sizeHint < 1 ? INITIAL_BUFFER_SIZE : sizeHint;
+        EnsureBufferCapacity(size);
+        return _buffer.AsMemory().Slice(_start, size);
+    }
+
+    /// <summary>
+    /// Gets a <see cref="Span{T}"/> to write to.
+    /// </summary>
+    /// <param name="sizeHint">
+    /// The minimum size of the returned <see cref="Span{T}"/>.
+    /// </param>
+    /// <returns>
+    /// A <see cref="Span{T}"/> to write to.
+    /// </returns>
+    /// <exception cref="ArgumentOutOfRangeException">
+    /// Thrown if <paramref name="sizeHint"/> is negative.
+    /// </exception>
+    public Span<byte> GetSpan(int sizeHint = 0)
+    {
+        if (_disposed)
+        {
+            throw new ObjectDisposedException(nameof(ArrayPoolWriter));
+        }
+
+        if (sizeHint < 0)
+        {
+            throw new ArgumentOutOfRangeException(nameof(sizeHint));
+        }
+
+        int size = sizeHint < 1 ? INITIAL_BUFFER_SIZE : sizeHint;
+        EnsureBufferCapacity(size);
+        return _buffer.AsSpan().Slice(_start, size);
+    }
+
+    /// <summary>
+    /// Ensures that the internal buffer has the needed capacity.
+    /// </summary>
+    /// <param name="neededCapacity">
+    /// The needed capacity on the internal buffer.
+    /// </param>
+    private void EnsureBufferCapacity(int neededCapacity)
+    {
+        // check if we have enough capacity available on the buffer.
+        if (_capacity < neededCapacity)
+        {
+            // if we need to expand the buffer we first capture the original buffer.
+            byte[] buffer = _buffer;
+
+            // next we determine the new size of the buffer, we at least double the size to avoid
+            // expanding the buffer too often.
+            int newSize = buffer.Length * 2;
+
+            // if that new buffer size is not enough to satisfy the needed capacity
+            // we add the needed capacity to the doubled buffer capacity.
+            if (neededCapacity > newSize - _start)
+            {
+                newSize += neededCapacity;
+            }
+
+            // next we will rent a new array from the array pool that supports
+            // the new capacity requirements.
+            _buffer = ArrayPool<byte>.Shared.Rent(newSize);
+
+            // the rented array might have a larger size than the needed capacity,
+            // so we will take the buffer length and calculate from that the free capacity.
+            _capacity += _buffer.Length - buffer.Length;
+
+            // finally we copy the data from the original buffer to the new buffer.
+            buffer.AsSpan().CopyTo(_buffer);
+
+            // last but not least we return the original buffer to the array pool.
+            ArrayPool<byte>.Shared.Return(buffer);
+        }
+    }
+
+    /// <inheritdoc/>
+    public void Dispose()
+    {
+        if (!_disposed)
+        {
+            ArrayPool<byte>.Shared.Return(_buffer);
+            _buffer = Array.Empty<byte>();
+            _capacity = 0;
+            _start = 0;
+            _disposed = true;
+        }
+    }
+}

src/Core/Resolvers/CosmosQueryEngine.cs

@@ -189,14 +189,14 @@ public Task<IActionResult> ExecuteAsync(StoredProcedureRequestContext context, s
         }
 
         /// <inheritdoc />
-        public JsonDocument ResolveInnerObject(JsonElement element, IObjectField fieldSchema, ref IMetadata metadata)
+        public JsonElement ResolveObject(JsonElement element, IObjectField fieldSchema, ref IMetadata metadata)
         {
-            //TODO: Try to avoid additional deserialization/serialization here.
-            return JsonDocument.Parse(element.ToString());
+            return element;
         }
 
         /// <inheritdoc />
-        public object ResolveListType(JsonElement element, IObjectField fieldSchema, ref IMetadata metadata)
+        /// metadata is not used in this method, but it is required by the interface.
+        public object ResolveList(JsonElement array, IObjectField fieldSchema, ref IMetadata metadata)
         {
             IType listType = fieldSchema.Type;
             // Is the List type nullable? [...]! vs [...]
@@ -217,10 +217,10 @@ public object ResolveListType(JsonElement element, IObjectField fieldSchema, ref
 
             if (listType.IsObjectType())
             {
-                return JsonSerializer.Deserialize<List<JsonElement>>(element);
+                return JsonSerializer.Deserialize<List<JsonElement>>(array);
             }
 
-            return JsonSerializer.Deserialize(element, fieldSchema.RuntimeType);
+            return JsonSerializer.Deserialize(array, fieldSchema.RuntimeType);
         }
 
         /// <summary>

src/Core/Resolvers/IQueryEngine.cs

@@ -44,11 +44,11 @@ public interface IQueryEngine
         /// <summary>
         /// Resolves a jsonElement representing an inner object based on the field's schema and metadata
         /// </summary>
-        public JsonDocument? ResolveInnerObject(JsonElement element, IObjectField fieldSchema, ref IMetadata metadata);
+        public JsonElement ResolveObject(JsonElement element, IObjectField fieldSchema, ref IMetadata metadata);
 
         /// <summary>
         /// Resolves a jsonElement representing a list type based on the field's schema and metadata
         /// </summary>
-        public object? ResolveListType(JsonElement element, IObjectField fieldSchema, ref IMetadata metadata);
+        public object ResolveList(JsonElement array, IObjectField fieldSchema, ref IMetadata? metadata);
     }
 }

src/Core/Resolvers/JsonObjectExtensions.cs

@@ -0,0 +1,91 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+using System.Buffers;
+using System.Text.Json;
+using System.Text.Json.Nodes;
+
+/// <summary>
+/// This extension class provides helpers to convert a mutable JSON object
+/// to a JSON element or JSON document.
+/// </summary>
+internal static class JsonObjectExtensions
+{
+    /// <summary>
+    /// Converts a mutable JSON object to an immutable JSON element.
+    /// </summary>
+    /// <param name="obj">
+    /// The mutable JSON object to convert.
+    /// </param>
+    /// <returns>
+    /// An immutable JSON element.
+    /// </returns>
+    /// <exception cref="ArgumentNullException">
+    /// Thrown if <paramref name="obj"/> is <see langword="null"/>.
+    /// </exception>
+    public static JsonElement ToJsonElement(this JsonObject obj)
+    {
+        if (obj == null)
+        {
+            throw new ArgumentNullException(nameof(obj));
+        }
+
+        // we first write the mutable JsonObject to the pooled buffer and avoid serializing
+        // to a full JSON string.
+        using ArrayPoolWriter buffer = new();
+        obj.WriteTo(buffer);
+
+        // next we take the reader here and parse the JSON element from the buffer.
+        Utf8JsonReader reader = new(buffer.GetWrittenSpan());
+
+        // the underlying JsonDocument will not use pooled arrays to store metadata on it ...
+        // this JSON element can be safely returned.
+        return JsonElement.ParseValue(ref reader);
+    }
+
+    /// <summary>
+    /// Converts a mutable JSON object to an immutable JSON document.
+    /// </summary>
+    /// <param name="obj">
+    /// The mutable JSON object to convert.
+    /// </param>
+    /// <returns>
+    /// An immutable JSON document.
+    /// </returns>
+    /// <exception cref="ArgumentNullException">
+    /// Thrown if <paramref name="obj"/> is <see langword="null"/>.
+    /// </exception>
+    public static JsonDocument ToJsonDocument(this JsonObject obj)
+    {
+        if (obj == null)
+        {
+            throw new ArgumentNullException(nameof(obj));
+        }
+
+        // we first write the mutable JsonObject to the pooled buffer and avoid serializing
+        // to a full JSON string.
+        using ArrayPoolWriter buffer = new();
+        obj.WriteTo(buffer);
+
+        // next we parse the JSON document from the buffer.
+        // this JSON document will be disposed by the GraphQL execution engine.
+        return JsonDocument.Parse(buffer.GetWrittenMemory());
+    }
+
+    private static void WriteTo(this JsonObject obj, IBufferWriter<byte> bufferWriter)
+    {
+        if (obj == null)
+        {
+            throw new ArgumentNullException(nameof(obj));
+        }
+
+        if (bufferWriter == null)
+        {
+            throw new ArgumentNullException(nameof(bufferWriter));
+        }
+
+        using Utf8JsonWriter writer = new(bufferWriter);
+        obj.WriteTo(writer);
+        writer.Flush();
+    }
+}