C++/CLI x86 marshalling misses copy ctor/dtor calls resulting in C++ STL iterator debugging breaking

[tgani-msft]

Description

This issue is a mirror of DTS bug 1949402, meant to track it on Github.

@jkoritzinsky is looking at this issue.

Reproduction Steps

The DTS bug contains the repro. Essentially, a copy ctor of std::vector<>::iterator is not called when pushing the argument onto the stack for the call to the native method, which takes an argument of type iterator. This leads to STL's iterator debugging book-keeping code to later assert.

This happens only for x86 and does not repro in the desktop framework where the copy ctor is correctly called.

Expected behavior

The copy ctor is correctly called and no assert fails in STL iterator debugging code.

Actual behavior

STL iterator debugging assert popup.

Regression?

Yes. From desktop framework behavior. Customer is blocked from adopting Net 8.0

Known Workarounds

Disabling iterator debugging. But this still leaves open the question of silent bad codegen happening for similar code that is not protected by asserts.

Configuration

.NET 8 x86

Other information

No response

[jkoritzinsky]

For reference, this behavior has been since .NET Core 3.1 since the introduction of C++/CLI support: dotnet/coreclr#22805

[jkoritzinsky]

As we now know, there is a breaking change in dotnet/coreclr#22805. This can be observed with a type that escapes the this pointer (or a pointer relative to the this pointer) in a constructor/copy constructor and expects to find it again in the destructor. Without escaping this or storing a pointer that is relative to this in a C++ struct passed between managed and unmanaged by value, the breaking change is not observable.

The change was caused by a removal of machinery that injected additional copy-constructor/destructor calls in locations unrepresentable in IL (ie in the middle of a call instruction and in the unmanaged->managed stub).

This is only observable in x86 (other platforms have a different calling convention), and the most likely scenario where this pops up (STL iterators) is only observable with _ITERATOR_DEBUG_LEVEL=2 when passed an iterator (or a struct that has an iterator field).

Since removing this logic, we have removed a significant amount of the machinery that supported these features, to the point that restoring the prior machinery (especially in the unmanaged->managed scenario) is not possible. We would need to implement a new approach integrated into the JIT to add support for this scenario.

Below is a minimal repro of the code-patterns that can observe the breaking change:

#pragma unmanaged
#include <vector>
#include <iostream>

struct Relative;

std::vector<Relative*> relatives;

struct Relative
{
    void* relative;
    Relative()
    {
        std::cout << "Registering " << std::hex << this << "\n";
        relatives.push_back(this); // Escaping this
        relative = this - 1; // Relative pointer stored in object
    }

    Relative(const Relative& other)
    {
        std::cout << "Registering copy of " << std::hex << &other << " at " << this << "\n";
        relatives.push_back(this); // Escaping this
        relative = this - 1; // Relative pointer stored in object
    }

    ~Relative()
    {
        auto location = std::find(relatives.begin(), relatives.end(), this); // Find escaped this
        if (location != relatives.end())
        {
            std::cout << "Unregistering " << std::hex << this << "\n";
            relatives.erase(location);
        }
        else
        {
            std::cout << "Error: Relative object " << std::hex << this << " not registered\n";
        }

        if (relative != this - 1) // Checking relative pointer
        {
            std::cout << " Error: Relative object " << std::hex << this << " has invalid relative pointer " << std::hex << relative << "\n";
        }
    }
};

void UseRelative(Relative rel)
{
    std::cout << "Unmanaged: Using relative at address " << std::hex << &rel << "\n";
}

void UseRelativeManaged(Relative rel);

void CallRelative()
{
    Relative rel;
    UseRelativeManaged(rel);
}

#pragma managed

extern "C" void __declspec(dllexport) Run()
{
    // Managed to unmanaged
    {
        Relative rel;
        UseRelative(rel);
    }

    // Unmanaged to managed
    CallRelative();
}

void UseRelativeManaged(Relative rel)
{
    std::cout << "Managed: Using relative at address " << std::hex << &rel << "\n";
}

Output:

Running test
Registering 0031FCC8
Registering copy of 0031FCC8 at 0031FCCC
Registering copy of 0031FCCC at 0031FCA0
Unregistering 0031FCCC
Unmanaged: Using relative at address 0031FC70
Error: Relative object 0031FC70 not registered
 Error: Relative object 0031FC70 has invalid relative pointer 0031FC9C
Unregistering 0031FCC8
Registering 0031FC64
Registering copy of 0031FC64 at 0031FC54
Managed: Using relative at address 0031FC40
Error: Relative object 0031FC40 not registered
 Error: Relative object 0031FC40 has invalid relative pointer 0031FC50
Unregistering 0031FC64

I'm looking into costing what a new approach to fixing this would be.

[jkoritzinsky]

This is still fixed as per #100050