#expect is causing mismatch behavior on Linux + release configuration build

[Kyle-Ye]

Description

I notice some test case is failing on Linux platform + release configuration.

Test case pass on Linux+Debug & Darwin+Debug & Darwin+Release

Issue 1: #expect issue

Digging into it, I found if I comment all #expect code or change to the old XCTAssertTrue from XCTest, the test case will pass normally.

Issue 2: withKnownIssue issue

Also I can't record the such known issue with withKnownIssue. If I use it the failing case will magically pass again.

// A simplified version of the business code
struct PointerOffsetTests {
    @Test
    func demo() {
        var tuple = Tuple(first: 1, second: 2)
        typealias Base = Tuple<Int, Int>
        let firstOffset = PointerOffset<Base, Int>(byteOffset: 0)
        let secondOffset = PointerOffset<Base, Int>(byteOffset: 8)
        withUnsafePointer(to: tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 1)
            #expect(pointer[offset: secondOffset] == 2)
        }
        withUnsafeMutablePointer(to: &tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 1)
            #expect(pointer[offset: secondOffset] == 2)

            pointer[offset: firstOffset] = 3
            pointer[offset: secondOffset] = 4
                
            #expect(pointer[offset: firstOffset] == 3)
            #expect(pointer[offset: secondOffset] == 4)
        }
        withUnsafePointer(to: &tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 3)
            #expect(pointer[offset: secondOffset] == 4)
        }
        #if !canImport(Darwin) && !DEBUG
        // FIXME: The issue only occur on Linux + Release configuration (Swift 5.10)
        // Uncomment the following withKnownIssue code will make the result back to normal thus causing 5 new issues
//            withKnownIssue {
//                withUnsafePointer(to: tuple) { pointer in
//                    #expect(pointer[offset: firstOffset] == 3)
//                    #expect(pointer[offset: secondOffset] == 4)
//                }
//                #expect(tuple.first == 3)
//                #expect(tuple.second == 4)
//            }
        withUnsafePointer(to: tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 1)
            #expect(pointer[offset: secondOffset] == 2)
        }
        #expect(tuple.first == 1)
        #expect(tuple.second == 2)
        #else
        withUnsafePointer(to: tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 3)
            #expect(pointer[offset: secondOffset] == 4)
        }
        #expect(tuple.first == 3)
        #expect(tuple.second == 4)
        #endif
    }
}

Expected behavior

Test case works normal

Actual behavior

Test case is failing

Steps to reproduce

Unzip the following DemoKit.zip on Ubuntu 22.04 + Swift 5.10 env.

Run swift test -c release

DemoKit.zip

swift-testing version/commit hash

0.6.0

Swift & OS version (output of swift --version ; uname -a)

Swift 5.10 release + Ubuntu 22.04 + Arm64

Other info

My opinion is that the root cause of this issue is most likely not within swift-testing. It is rather caused by some optimization in the swift compiler, but the direct cause currently seems to be related to swift-testing.

Also reproducible with the following conditions

• swift-testing 0.6.0 + Swift 5.10
• swift-testing 0.8.0 + Swift 5.10
• swift-testing 0.8.0 + Swift 6.0-dev 2024-05-01-a snapshot

[Kyle-Ye]

I'm trying to upgrade to 0.8.0 to see if the issue still exists.

Update: Yes, the issue still exist for DemoKit + swift-testing 0.8.0

swift-syntax version:

• 600.0.0-prerelease-2024-05-02
• 3301d3362555b679097e82f93be0b524c5083e65)

[Kyle-Ye]

From @grynspan's reply on Slack channel
I see you're hard-coding the pointer offsets to 0 and 8. In Swift, there's no guarantee of the layout of a type in memory and the compiler in release mode is free to aggressively rearrange bits for any reason. At a guess, you're stomping on the stack here.
Have you tried using pointer(to:) to compute inner pointers instead?

1. The hard-coding 0 and 8 is just for testing here. We can replace them with runtime calculated value here. And I do not think it is the issue here.

let firstOffset = PointerOffset<Base, Int>.offset { .of(&$0.first) }
let secondOffset = PointerOffset<Base, Int>.offset { .of(&$0.second) }

The PointerOffset.offset and PointerOffset.of implementation is using runtime offset here.

2. If you suspect the pointer issue, then let's consider the following code below.

I'm not using any pointer operation after "MARK". But adding withKnownIssue will magically cause mismatch behavior within the scope and after the scope.

struct PointerOffsetTests {
    @Test
    func demo() {
        var tuple = Tuple(first: 1, second: 2)
        typealias Base = Tuple<Int, Int>
        let firstOffset = PointerOffset<Base, Int>.offset { .of(&$0.first) }
        let secondOffset = PointerOffset<Base, Int>.offset { .of(&$0.second) }
        withUnsafePointer(to: tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 1)
            #expect(pointer[offset: secondOffset] == 2)
        }
        withUnsafeMutablePointer(to: &tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 1)
            #expect(pointer[offset: secondOffset] == 2)

            pointer[offset: firstOffset] = 3
            pointer[offset: secondOffset] = 4
                
            #expect(pointer[offset: firstOffset] == 3)
            #expect(pointer[offset: secondOffset] == 4)
        }
        withUnsafePointer(to: &tuple) { pointer in
            #expect(pointer[offset: firstOffset] == 3)
            #expect(pointer[offset: secondOffset] == 4)
        }
        #if !canImport(Darwin) && !DEBUG
        // MARK
        print(tuple) // Tuple<Int, Int>(first: 1, second: 2)
        withKnownIssue {
            print(tuple) // Tuple<Int, Int>(first: 3, second: 4)
        }
        print(tuple) // Tuple<Int, Int>(first: 3, second: 4)
        #else
        print(tuple) // Tuple<Int, Int>(first: 3, second: 4)
        #endif
   }
}

[grynspan]

The hard-coding 0 and 8 is just for testing here.

Then it's a variable we need to eliminate in order to narrow down the root cause of the issue.

Looking at the code you shared in the zip file:

    public static func of(_ member: inout Member) -> PointerOffset {
        withUnsafePointer(to: &member) { memberPointer in
            let offset = UnsafeRawPointer(memberPointer) - UnsafeRawPointer(invalidScenePointer())
            return PointerOffset(byteOffset: offset)
        }
    }    

This code is invalid. memberPointer is a temporary pointer to a copy of member, and member itself is a temporary copy of the value outside the call to of(_:). The address you have here is almost certainly an arbitrary location on the stack that is unrelated to the original value (presumably tuple.) Mathing with it and invalidScenePointer() is going to give you an arbitrary value as your offset that has no relation to the address of tuple. Under different optimization regimes (release vs. debug) it's entirely unsurprising that you'd get different results.

As I suggested earlier, use UnsafePointer.pointer(to:) to get an inner pointer inside an existing value. Be aware that withUnsafePointer(to:_:) gives you a pointer to a copy of the original value, not the address of the original value: in Swift, most values are not guaranteed to have unique addresses and the compiler must synthesize one when needed. So the pointer returned from pointer(to:) is only valid for the lifetime of the enclosing call to withUnsafePointer(to:_:).

[Kyle-Ye]

Change to use pointer(to:) do seem to solve the issue. Thanks.