Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

<vector>, <string>: Avoid dereferencing null (fancy) pointers when calling _Seek_to/_Unwrapped on or comparing vector and basic_string's iterators #4275

Merged
merged 7 commits into from
Jan 11, 2024

Conversation

frederick-vs-ja
Copy link
Contributor

@frederick-vs-ja frederick-vs-ja commented Dec 17, 2023

Per CWG-2823 (recently voted into the Standard), dereferencing a null pointer is UB in the involved case. Currently, MSVC correctly diagnoses this while Clang 17 doesn't.

Perhaps other iterator's _Seek_to functions (edit: other than basic_string's) are fine, since

  • basic_string's iterators obtained from containers are always non-null, and
  • other _Seek_to functions don't seem to dereference pointers themselves.

Edit: basic_string's iterator may also fail in std::find(I{}, I{}, T{}) == I{} when the iterator contains a null fancy pointer.

Unblocks libcxx tests:

  • std/containers/sequences/vector/vector.erasure/erase.pass.cpp
  • std/containers/sequences/vector/vector.erasure/erase_if.pass.cpp

@frederick-vs-ja frederick-vs-ja requested a review from a team as a code owner December 17, 2023 17:10
@StephanTLavavej StephanTLavavej added the bug Something isn't working label Dec 18, 2023
@StephanTLavavej StephanTLavavej self-assigned this Dec 18, 2023
stl/inc/vector Outdated Show resolved Hide resolved
@frederick-vs-ja frederick-vs-ja changed the title <vector>: Avoid dereferencing null pointers when _Seek_to-ing vector's iterators <vector>, <string>: Avoid dereferencing null pointers when _Seek_to-ing vector and basic_string's iterators Jan 4, 2024
@CaseyCarter
Copy link
Member

CaseyCarter commented Jan 4, 2024

The test failure is EDG complaining about comparing null pointers with <= in a constant expression, which bug (DevCom-1670927, also VSO-1161663) was supposedly fixed AGES ago (17.6 preview 3). I'm confused.

I've approved the change in principle, but we'll need to avoid the test failure and file a new EDG bug before merging.

EDIT: constexpr bugs are always so much fun. So far reduced to:

template <class I>
constexpr void meow(I first, const I last) {
    _STD _Adl_verify_range(first, last);
    first._Seek_to(first._Unwrapped());
}

constexpr bool test_seeking_vector_iterators() {
    using VIt = vector<int, min_allocator<int>>::iterator;
    meow(VIt{}, VIt{});
    return true;
}
static_assert(test_seeking_vector_iterators());

The _Adl_verify_range call triggers the error unless the following _Seek_to line is commented out.

@frederick-vs-ja
Copy link
Contributor Author

frederick-vs-ja commented Jan 6, 2024

The _Adl_verify_range call triggers the error unless the following _Seek_to line is commented out.

Wow... it seems that the real error comes from _Unwrapped. The current implementation possibly calls _Unfancy on a null fancy pointer, which is core UB.

I think we should use _Unfancy_maybe_null in these functions:

STL/stl/inc/xstring

Lines 2032 to 2034 in b84bf09

_NODISCARD _CONSTEXPR20 const value_type* _Unwrapped() const noexcept {
return _Unfancy(_Ptr);
}

STL/stl/inc/xstring

Lines 2156 to 2158 in b84bf09

_NODISCARD _CONSTEXPR20 value_type* _Unwrapped() const noexcept {
return const_cast<value_type*>(_Unfancy(this->_Ptr));
}

STL/stl/inc/vector

Lines 216 to 218 in b84bf09

_NODISCARD _CONSTEXPR20 const value_type* _Unwrapped() const noexcept {
return _Unfancy(_Ptr);
}

STL/stl/inc/vector

Lines 344 to 346 in b84bf09

_NODISCARD _CONSTEXPR20 value_type* _Unwrapped() const noexcept {
return _Unfancy(this->_Ptr);
}

(Not sure whether _Unfancy in operator<=> is OK. I'll open another issue.) operator<=> overloads were also bad. I'll fix them.

Slightly reduced example

constexpr bool test_seeking() {
    using VIt  = std::vector<int, min_allocator<int>>::iterator;
    auto first = VIt{};
    _Verify_range(first, first);
    (void) first._Unwrapped();
    return true;
}
static_assert(test_seeking());

IIUC, in constant evaluation:

  • Binding the result of dereference a null pointer to a reference function parameter is now diagnosed by MSVC, but not Clang and EDG. LLVM-77186 is reported but may be closed in favor of CWG-2823.
  • Binding the result of dereference a null pointer to the return value of a function is now diagnosed by Clang (but the behavior in SFINAE is not yet conforming), and sometime but not always by MSVC. DevCom-10554350 is reported for this.
    • EDG seems able to cause constant evaluation failure due to such UB by accident, which is encountered in this PR.

@frederick-vs-ja frederick-vs-ja changed the title <vector>, <string>: Avoid dereferencing null pointers when _Seek_to-ing vector and basic_string's iterators <vector>, <string>: Avoid dereferencing null (fancy) pointers when _Seek_to/_Unwrapped-ing vector and basic_string's iterators Jan 6, 2024
@frederick-vs-ja frederick-vs-ja changed the title <vector>, <string>: Avoid dereferencing null (fancy) pointers when _Seek_to/_Unwrapped-ing vector and basic_string's iterators <vector>, <string>: Avoid dereferencing null (fancy) pointers when calling _Seek_to/_Unwrapped on or comparing vector and basic_string's iterators Jan 6, 2024
@CaseyCarter
Copy link
Member

Wow... it seems that the real error comes from _Unwrapped. The current implementation possibly calls _Unfancy on a null fancy pointer, which is core UB.

Good catch!

Comment on lines +12 to +13
// some portions of this file are derived from libc++'s test files:
// * support/min_allocator.h
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No change requested: Note that tests/std can include headers from libcxx's support directory. (We've arranged for this to work in both the GitHub and MSVC-internal test harnesses.) It looks like <min_allocator.h> defines a bunch of other stuff that we don't need here, so distilling it down to what we need is reasonable, I just wanted to mention this if it's ever useful in the future.

Comment on lines +37 to +59
template <class ID>
class min_pointer<const void, ID> {
const void* ptr_;

public:
min_pointer() = default;
min_pointer(nullptr_t) noexcept : ptr_(nullptr) {}
template <class T>
min_pointer(min_pointer<T, ID> p) noexcept : ptr_(p.ptr_) {}

explicit operator bool() const {
return ptr_ != nullptr;
}

friend bool operator==(min_pointer x, min_pointer y) {
return x.ptr_ == y.ptr_;
}
friend bool operator!=(min_pointer x, min_pointer y) {
return !(x == y);
}
template <class U, class XID>
friend class min_pointer;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No change requested: I observe that min_pointer<const void, ID> lacks constexpr when the other specializations have it. This appears to be the case in libc++'s original code, so maybe they were trying to do something special there.

@StephanTLavavej
Copy link
Member

Thanks! I pushed a conflict-free merge with main, and included <cstdlib> for abort() (it was slightly above my "don't want to reset testing" threshold 😹).

@StephanTLavavej StephanTLavavej removed their assignment Jan 11, 2024
@StephanTLavavej StephanTLavavej self-assigned this Jan 11, 2024
@StephanTLavavej
Copy link
Member

I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed.

@StephanTLavavej StephanTLavavej merged commit 9ab77b9 into microsoft:main Jan 11, 2024
35 checks passed
@StephanTLavavej
Copy link
Member

Thanks for investigating and fixing these libcxx failures - I'm so happy to see this! 🎩 😻 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants