Skip to content

Commit

Permalink
git merge smart_holder (#30066)
Browse files Browse the repository at this point in the history
* fix: Use lowercase builtin collection names (#4833)

* Update render for buffer sequence and handle  (#4831)

* fix: Add capitalize render name of `py::buffer` and `py::sequence`

* fix: Render `py::handle` same way as `py::object`

* tests: Fix tests `handle` -> `object`

* tests: Test capitaliation of `py::sequence` and `py::buffer`

* style: pre-commit fixes

* fix: Render `py::object` as `Any`

* Revert "fix: Render `py::object` as `Any`"

This reverts commit 7861dcfabb78ac210b4c67c35a0d47fb67525a96.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>

* fix: Missing typed variants of `iterator` and `iterable` (#4832)

* Fix small bug introduced with PR #4735 (#4845)

* Bug fix: `result[0]` called if `result.empty()`

* Add unit test that fails without the fix.

* fix(cmake): correctly detect FindPython policy and better warning (#4806)

* fix(cmake): support DEBUG_POSTFIX correctly (#4761)

* cmake: split extension

Into suffix and debug postfix. Pybind11 is currently treating both as
suffix, which is problematic when something else defines the
DEBUG_POSTFIX because they will be concatenated.

pybind11_extension sets SUFFIX to _d.something and if DEBUG_POSTFIX is
set to _d.

    _d + _d.something = _d_d.something

The issue has been reported at:

pybind/pybind11#4699

* style: pre-commit fixes

* fix(cmake): support postfix for old FindPythonInterp mode too

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>

* Avoid copy in iteration by using const auto & (#4861)

This change is fixing a Coverity AUTO_CAUSES_COPY issues.

* Add 2 missing `throw error_already_set();` (#4863)

Fixes oversights in PR #4570.

* MAINT: Include `numpy._core` imports (#4857)

* MAINT: Include numpy._core imports

* style: pre-commit fixes

* Apply review comments

* style: pre-commit fixes

* Add no-inline attribute

* Select submodule name based on numpy version

* style: pre-commit fixes

* Update pre-commit check

* Add error_already_set and simplify if statement

* Update .pre-commit-config.yaml

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>

* MAINT: Remove np.int_ (#4867)

* chore(deps): update pre-commit hooks (#4868)

* chore(deps): update pre-commit hooks

updates:
- [github.com/psf/black-pre-commit-mirror: 23.7.0 → 23.9.1](psf/black-pre-commit-mirror@23.7.0...23.9.1)
- [github.com/astral-sh/ruff-pre-commit: v0.0.287 → v0.0.292](astral-sh/ruff-pre-commit@v0.0.287...v0.0.292)
- [github.com/codespell-project/codespell: v2.2.5 → v2.2.6](codespell-project/codespell@v2.2.5...v2.2.6)
- [github.com/shellcheck-py/shellcheck-py: v0.9.0.5 → v0.9.0.6](shellcheck-py/shellcheck-py@v0.9.0.5...v0.9.0.6)
- [github.com/PyCQA/pylint: v3.0.0a7 → v3.0.0](pylint-dev/pylint@v3.0.0a7...v3.0.0)

* Update .pre-commit-config.yaml

* style: pre-commit fixes

* Update .pre-commit-config.yaml

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Sergei Izmailov <sergei.a.izmailov@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: László Papp <108480845+lpapp-foundry@users.noreply.github.com>
Co-authored-by: Oleksandr Pavlyk <oleksandr.pavlyk@intel.com>
Co-authored-by: Mateusz Sokół <8431159+mtsokol@users.noreply.github.com>
  • Loading branch information
7 people authored Oct 4, 2023
1 parent f0fd911 commit 1121f25
Show file tree
Hide file tree
Showing 25 changed files with 194 additions and 57 deletions.
2 changes: 1 addition & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ The valid options are:
* Use `-G` and the name of a generator to use something different. `cmake
--help` lists the generators available.
- On Unix, setting `CMAKE_GENERATER=Ninja` in your environment will give
you automatic mulithreading on all your CMake projects!
you automatic multithreading on all your CMake projects!
* Open the `CMakeLists.txt` with QtCreator to generate for that IDE.
* You can use `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` to generate the `.json` file
that some tools expect.
Expand Down
12 changes: 6 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ repos:

# Black, the code formatter, natively supports pre-commit
- repo: https://github.com/psf/black-pre-commit-mirror
rev: "23.7.0" # Keep in sync with blacken-docs
rev: "23.9.1" # Keep in sync with blacken-docs
hooks:
- id: black

# Ruff, the Python auto-correcting linter written in Rust
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.0.287
rev: v0.0.292
hooks:
- id: ruff
args: ["--fix", "--show-fixes"]
Expand Down Expand Up @@ -126,15 +126,15 @@ repos:
# Use tools/codespell_ignore_lines_from_errors.py
# to rebuild .codespell-ignore-lines
- repo: https://github.com/codespell-project/codespell
rev: "v2.2.5"
rev: "v2.2.6"
hooks:
- id: codespell
exclude: ".supp$"
args: ["-x.codespell-ignore-lines", "-Lccompiler"]

# Check for common shell mistakes
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: "v0.9.0.5"
rev: "v0.9.0.6"
hooks:
- id: shellcheck

Expand All @@ -144,12 +144,12 @@ repos:
- id: disallow-caps
name: Disallow improper capitalization
language: pygrep
entry: PyBind|Numpy|Cmake|CCache|PyTest
entry: PyBind|\bNumpy\b|Cmake|CCache|PyTest
exclude: ^\.pre-commit-config.yaml$

# PyLint has native support - not always usable, but works for us
- repo: https://github.com/PyCQA/pylint
rev: "v3.0.0a7"
rev: "v3.0.0"
hooks:
- id: pylint
files: ^pybind11
10 changes: 10 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@
# All rights reserved. Use of this source code is governed by a
# BSD-style license that can be found in the LICENSE file.

# Propagate this policy (FindPythonInterp removal) so it can be detected later
if(NOT CMAKE_VERSION VERSION_LESS "3.27")
cmake_policy(GET CMP0148 _pybind11_cmp0148)
endif()

cmake_minimum_required(VERSION 3.5)

# The `cmake_minimum_required(VERSION 3.5...3.26)` syntax does not work with
Expand All @@ -16,6 +21,11 @@ else()
cmake_policy(VERSION 3.26)
endif()

if(_pybind11_cmp0148)
cmake_policy(SET CMP0148 ${_pybind11_cmp0148})
unset(_pybind11_cmp0148)
endif()

# Avoid infinite recursion if tests include this as a subdirectory
if(DEFINED PYBIND11_MASTER_PROJECT)
return()
Expand Down
14 changes: 13 additions & 1 deletion include/pybind11/cast.h
Original file line number Diff line number Diff line change
Expand Up @@ -817,7 +817,7 @@ class tuple_caster {
}

static constexpr auto name
= const_name("Tuple[") + concat(make_caster<Ts>::name...) + const_name("]");
= const_name("tuple[") + concat(make_caster<Ts>::name...) + const_name("]");

template <typename T>
using cast_op_type = type;
Expand Down Expand Up @@ -1044,6 +1044,10 @@ struct handle_type_name<bytes> {
static constexpr auto name = const_name(PYBIND11_BYTES_NAME);
};
template <>
struct handle_type_name<buffer> {
static constexpr auto name = const_name("Buffer");
};
template <>
struct handle_type_name<int_> {
static constexpr auto name = const_name("int");
};
Expand All @@ -1064,10 +1068,18 @@ struct handle_type_name<function> {
static constexpr auto name = const_name("Callable");
};
template <>
struct handle_type_name<handle> {
static constexpr auto name = handle_type_name<object>::name;
};
template <>
struct handle_type_name<none> {
static constexpr auto name = const_name("None");
};
template <>
struct handle_type_name<sequence> {
static constexpr auto name = const_name("Sequence");
};
template <>
struct handle_type_name<args> {
static constexpr auto name = const_name("*args");
};
Expand Down
2 changes: 2 additions & 0 deletions include/pybind11/detail/cross_extension_shared_state.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ inline object get_python_state_dict() {
#endif
if (!state_dict) {
raise_from(PyExc_SystemError, "pybind11::detail::get_python_state_dict() FAILED");
throw error_already_set();
}
return state_dict;
}
Expand Down Expand Up @@ -95,6 +96,7 @@ struct cross_extension_shared_state {
" Retrieve payload_type** from capsule FAILED for ABI ID \""
+ std::string(AdapterType::abi_id()) + "\"")
.c_str());
throw error_already_set();
}
payload_pp() = static_cast<payload_type **>(raw_ptr);
return *payload_pp();
Expand Down
2 changes: 1 addition & 1 deletion include/pybind11/detail/init.h
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ constexpr bool is_alias(void *) {
}

// Constructs and returns a new object; if the given arguments don't map to a constructor, we fall
// back to brace aggregate initiailization so that for aggregate initialization can be used with
// back to brace aggregate initialization so that for aggregate initialization can be used with
// py::init, e.g. `py::init<int, int>` to initialize a `struct T { int a; int b; }`. For
// non-aggregate types, we need to use an ordinary T(...) constructor (invoking as `T{...}` usually
// works, but will not do the expected thing when `T` has an `initializer_list<T>` constructor).
Expand Down
1 change: 1 addition & 0 deletions include/pybind11/detail/internals.h
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,7 @@ inline internals **get_internals_pp_from_capsule(handle obj) {
void *raw_ptr = PyCapsule_GetPointer(obj.ptr(), /*name=*/nullptr);
if (raw_ptr == nullptr) {
raise_from(PyExc_SystemError, "pybind11::detail::get_internals_pp_from_capsule() FAILED");
throw error_already_set();
}
return static_cast<internals **>(raw_ptr);
}
Expand Down
1 change: 1 addition & 0 deletions include/pybind11/detail/native_enum_data.h
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ inline void native_enum_add_to_parent(object parent, const detail::native_enum_d
if (!enum_module) {
raise_from(PyExc_SystemError,
"`import enum` FAILED at " __FILE__ ":" PYBIND11_TOSTRING(__LINE__));
throw error_already_set();
}
auto py_enum_type = enum_module.attr(data.use_int_enum ? "IntEnum" : "Enum");
auto py_enum = py_enum_type(data.enum_name, data.members);
Expand Down
27 changes: 21 additions & 6 deletions include/pybind11/numpy.h
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,20 @@ inline numpy_internals &get_numpy_internals() {
return *ptr;
}

PYBIND11_NOINLINE module_ import_numpy_core_submodule(const char *submodule_name) {
module_ numpy = module_::import("numpy");
str version_string = numpy.attr("__version__");

module_ numpy_lib = module_::import("numpy.lib");
object numpy_version = numpy_lib.attr("NumpyVersion")(version_string);
int major_version = numpy_version.attr("major").cast<int>();

/* `numpy.core` was renamed to `numpy._core` in NumPy 2.0 as it officially
became a private module. */
std::string numpy_core_path = major_version >= 2 ? "numpy._core" : "numpy.core";
return module_::import((numpy_core_path + "." + submodule_name).c_str());
}

template <typename T>
struct same_size {
template <typename U>
Expand Down Expand Up @@ -263,9 +277,13 @@ struct npy_api {
};

static npy_api lookup() {
module_ m = module_::import("numpy.core.multiarray");
module_ m = detail::import_numpy_core_submodule("multiarray");
auto c = m.attr("_ARRAY_API");
void **api_ptr = (void **) PyCapsule_GetPointer(c.ptr(), nullptr);
if (api_ptr == nullptr) {
raise_from(PyExc_SystemError, "FAILURE obtaining numpy _ARRAY_API pointer.");
throw error_already_set();
}
npy_api api;
#define DECL_NPY_API(Func) api.Func##_ = (decltype(api.Func##_)) api_ptr[API_##Func];
DECL_NPY_API(PyArray_GetNDArrayCFeatureVersion);
Expand Down Expand Up @@ -626,11 +644,8 @@ class dtype : public object {

private:
static object _dtype_from_pep3118() {
static PyObject *obj = module_::import("numpy.core._internal")
.attr("_dtype_from_pep3118")
.cast<object>()
.release()
.ptr();
module_ m = detail::import_numpy_core_submodule("_internal");
static PyObject *obj = m.attr("_dtype_from_pep3118").cast<object>().release().ptr();
return reinterpret_borrow<object>(obj);
}

Expand Down
14 changes: 8 additions & 6 deletions include/pybind11/pybind11.h
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,13 @@ inline std::string replace_newlines_and_squash(const char *text) {
std::string result(text);
bool previous_is_whitespace = false;

// Do not modify string representations
char first_char = result[0];
char last_char = result[result.size() - 1];
if (first_char == last_char && first_char == '\'') {
return result;
if (result.size() >= 2) {
// Do not modify string representations
char first_char = result[0];
char last_char = result[result.size() - 1];
if (first_char == last_char && first_char == '\'') {
return result;
}
}
result.clear();

Expand Down Expand Up @@ -1146,7 +1148,7 @@ class cpp_function : public function {
}
msg += "kwargs: ";
bool first = true;
for (auto kwarg : kwargs) {
for (const auto &kwarg : kwargs) {
if (first) {
first = false;
} else {
Expand Down
10 changes: 5 additions & 5 deletions include/pybind11/stl.h
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ while pybind11 was very far on the strict side. Originally PyCLIF accepted any
Python iterable as input for a C++ `vector`/`set`/`map` argument, as long as
the elements were convertible. The obvious (in hindsight) problem was that
any empty Python iterable could be passed to any of these C++ types, e.g. `{}`
was accpeted for C++ `vector`/`set` arguments, or `[]` for C++ `map` arguments.
was accepted for C++ `vector`/`set` arguments, or `[]` for C++ `map` arguments.
The functions below strike a practical permissive-vs-strict compromise,
informed by tens of thousands of use cases in the wild. A main objective is
Expand Down Expand Up @@ -202,7 +202,7 @@ struct set_caster {
return s.release();
}

PYBIND11_TYPE_CASTER_RVPP(type, const_name("Set[") + key_conv::name + const_name("]"));
PYBIND11_TYPE_CASTER_RVPP(type, const_name("set[") + key_conv::name + const_name("]"));
};

template <typename Type, typename Key, typename Value>
Expand Down Expand Up @@ -273,7 +273,7 @@ struct map_caster {
}

PYBIND11_TYPE_CASTER_RVPP(Type,
const_name("Dict[") + key_conv::name + const_name(", ")
const_name("dict[") + key_conv::name + const_name(", ")
+ value_conv::name + const_name("]"));
};

Expand Down Expand Up @@ -340,7 +340,7 @@ struct list_caster {
return l.release();
}

PYBIND11_TYPE_CASTER_RVPP(Type, const_name("List[") + value_conv::name + const_name("]"));
PYBIND11_TYPE_CASTER_RVPP(Type, const_name("list[") + value_conv::name + const_name("]"));
};

template <typename Type, typename Alloc>
Expand Down Expand Up @@ -422,7 +422,7 @@ struct array_caster {

PYBIND11_TYPE_CASTER_RVPP(ArrayType,
const_name<Resizable>(const_name(""), const_name("Annotated["))
+ const_name("List[") + value_conv::name + const_name("]")
+ const_name("list[") + value_conv::name + const_name("]")
+ const_name<Resizable>(const_name(""),
const_name(", FixedSize(")
+ const_name<Size>()
Expand Down
20 changes: 20 additions & 0 deletions include/pybind11/typing.h
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ class Set : public set {
using set::set;
};

template <typename T>
class Iterable : public iterable {
using iterable::iterable;
};

template <typename T>
class Iterator : public iterator {
using iterator::iterator;
};

template <typename Signature>
class Callable;

Expand Down Expand Up @@ -85,6 +95,16 @@ struct handle_type_name<typing::Set<T>> {
static constexpr auto name = const_name("set[") + make_caster<T>::name + const_name("]");
};

template <typename T>
struct handle_type_name<typing::Iterable<T>> {
static constexpr auto name = const_name("Iterable[") + make_caster<T>::name + const_name("]");
};

template <typename T>
struct handle_type_name<typing::Iterator<T>> {
static constexpr auto name = const_name("Iterator[") + make_caster<T>::name + const_name("]");
};

template <typename Return, typename... Args>
struct handle_type_name<typing::Callable<Return(Args...)>> {
using retval_type = conditional_t<std::is_same<Return, void>::value, void_type, Return>;
Expand Down
7 changes: 7 additions & 0 deletions tests/test_buffers.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,3 +219,10 @@ def test_ctypes_from_buffer():
assert cinfo.shape == pyinfo.shape
assert cinfo.strides == pyinfo.strides
assert not cinfo.readonly


def test_buffer_docstring():
assert (
m.get_buffer_info.__doc__.strip()
== "get_buffer_info(arg0: Buffer) -> pybind11_tests.buffers.buffer_info"
)
4 changes: 2 additions & 2 deletions tests/test_builtin_casters.py
Original file line number Diff line number Diff line change
Expand Up @@ -352,15 +352,15 @@ def test_tuple(doc):
assert (
doc(m.pair_passthrough)
== """
pair_passthrough(arg0: Tuple[bool, str]) -> Tuple[str, bool]
pair_passthrough(arg0: tuple[bool, str]) -> tuple[str, bool]
Return a pair in reversed order
"""
)
assert (
doc(m.tuple_passthrough)
== """
tuple_passthrough(arg0: Tuple[bool, str, int]) -> Tuple[int, str, bool]
tuple_passthrough(arg0: tuple[bool, str, int]) -> tuple[int, str, bool]
Return a triple in reversed order
"""
Expand Down
4 changes: 2 additions & 2 deletions tests/test_factory_constructors.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ def test_init_factory_signature(msg):
1. m.factory_constructors.TestFactory1(arg0: m.factory_constructors.tag.unique_ptr_tag, arg1: int)
2. m.factory_constructors.TestFactory1(arg0: str)
3. m.factory_constructors.TestFactory1(arg0: m.factory_constructors.tag.pointer_tag)
4. m.factory_constructors.TestFactory1(arg0: handle, arg1: int, arg2: handle)
4. m.factory_constructors.TestFactory1(arg0: object, arg1: int, arg2: object)
Invoked with: 'invalid', 'constructor', 'arguments'
"""
Expand All @@ -95,7 +95,7 @@ def test_init_factory_signature(msg):
3. __init__(self: m.factory_constructors.TestFactory1, arg0: m.factory_constructors.tag.pointer_tag) -> None
4. __init__(self: m.factory_constructors.TestFactory1, arg0: handle, arg1: int, arg2: handle) -> None
4. __init__(self: m.factory_constructors.TestFactory1, arg0: object, arg1: int, arg2: object) -> None
"""
)

Expand Down
2 changes: 2 additions & 0 deletions tests/test_kwargs_and_defaults.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ TEST_SUBMODULE(kwargs_and_defaults, m) {
"kw_lb_func7",
[](const std::string &) {},
py::arg("str_arg") = "First line.\n Second line.");
m.def(
"kw_lb_func8", [](const CustomRepr &) {}, py::arg("custom") = CustomRepr(""));

// test_args_and_kwargs
m.def("args_function", [](py::args args) -> py::tuple {
Expand Down
6 changes: 5 additions & 1 deletion tests/test_kwargs_and_defaults.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ def test_function_signatures(doc):
assert doc(m.kw_func1) == "kw_func1(x: int, y: int) -> str"
assert doc(m.kw_func2) == "kw_func2(x: int = 100, y: int = 200) -> str"
assert doc(m.kw_func3) == "kw_func3(data: str = 'Hello world!') -> None"
assert doc(m.kw_func4) == "kw_func4(myList: List[int] = [13, 17]) -> str"
assert doc(m.kw_func4) == "kw_func4(myList: list[int] = [13, 17]) -> str"
assert doc(m.kw_func_udl) == "kw_func_udl(x: int, y: int = 300) -> str"
assert doc(m.kw_func_udl_z) == "kw_func_udl_z(x: int, y: int = 0) -> str"
assert doc(m.args_function) == "args_function(*args) -> tuple"
Expand Down Expand Up @@ -55,6 +55,10 @@ def test_function_signatures(doc):
doc(m.kw_lb_func7)
== "kw_lb_func7(str_arg: str = 'First line.\\n Second line.') -> None"
)
assert (
doc(m.kw_lb_func8)
== "kw_lb_func8(custom: m.kwargs_and_defaults.CustomRepr = ) -> None"
)


def test_named_arguments():
Expand Down
Loading

0 comments on commit 1121f25

Please sign in to comment.