Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move PyTypeInfo::AsRefTarget onto new trait HasPyGilBoundRef #2956

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion guide/src/class.md
Original file line number Diff line number Diff line change
Expand Up @@ -974,7 +974,6 @@ struct MyClass {
num: i32,
}
unsafe impl pyo3::type_object::PyTypeInfo for MyClass {
type AsRefTarget = pyo3::PyCell<Self>;
const NAME: &'static str = "MyClass";
const MODULE: ::std::option::Option<&'static str> = ::std::option::Option::None;
#[inline]
Expand All @@ -985,6 +984,10 @@ unsafe impl pyo3::type_object::PyTypeInfo for MyClass {
}
}

unsafe impl pyo3::HasPyGilBoundRef for MyClass {
type AsRefTarget = pyo3::PyCell<Self>;
}

impl pyo3::PyClass for MyClass {
type Frozen = pyo3::pyclass::boolean_struct::False;
}
Expand Down
1 change: 1 addition & 0 deletions newsfragments/2956.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Move `PyTypeInfo::AsRefTarget` onto new trait `HasPyGilBoundRef`.
5 changes: 3 additions & 2 deletions pyo3-macros-backend/src/pyclass.rs
Original file line number Diff line number Diff line change
Expand Up @@ -747,8 +747,6 @@ fn impl_pytypeinfo(

quote! {
unsafe impl _pyo3::type_object::PyTypeInfo for #cls {
type AsRefTarget = _pyo3::PyCell<Self>;

const NAME: &'static str = #cls_name;
const MODULE: ::std::option::Option<&'static str> = #module;

Expand All @@ -761,6 +759,9 @@ fn impl_pytypeinfo(
TYPE_OBJECT.get_or_init::<Self>(py)
}
}
unsafe impl _pyo3::HasPyGilBoundRef for #cls {
type AsRefTarget = _pyo3::PyCell<Self>;
}
}
}

Expand Down
20 changes: 15 additions & 5 deletions src/instance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use crate::pycell::{PyBorrowError, PyBorrowMutError, PyCell};
use crate::types::{PyDict, PyString, PyTuple};
use crate::{
ffi, AsPyPointer, FromPyObject, IntoPy, IntoPyPointer, PyAny, PyClass, PyClassInitializer,
PyRef, PyRefMut, PyTypeInfo, Python, ToPyObject,
PyRef, PyRefMut, Python, ToPyObject,
};
use std::marker::PhantomData;
use std::mem;
Expand Down Expand Up @@ -39,6 +39,16 @@ pub unsafe trait PyNativeType: Sized {
}
}

/// Trait to denote types which have a GIL-bound reference, e.g. PyAny has &'a PyAny.
///
/// # Safety
/// `Py<T>` will hand out references to `Self::AsRefTarget`. This _must_ have the
/// same layout as `*mut ffi::PyObject`.
Comment on lines +45 to +46
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the layout part guaranteed by PyNativeType? Unless "this" refers to Self rather than the target.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so. Probably still worth repeating this constraint here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure 👍

pub unsafe trait HasPyGilBoundRef {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the name could use some work. I don't have a name that is obviously good, but what about something like AsGilBorrow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing. I think we can sit on this and think for a bit. I'm a bit lost for a good name too. StorableInPy?

Copy link
Member

@adamreichold adamreichold Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to add completely different colour for the bikeshed: PyHeapType meaning that can be stored on the Python heap (i.e. in Py<T>) and thereby shared references must be bound to the GIL (as it protects access to the heap).

/// Type resulting from Py::as_ref or Py::into_ref
type AsRefTarget: PyNativeType;
}

/// A GIL-independent reference to an object allocated on the Python heap.
///
/// This type does not auto-dereference to the inner object because you must prove you hold the GIL to access it.
Expand Down Expand Up @@ -262,7 +272,7 @@ where

impl<T> Py<T>
where
T: PyTypeInfo,
T: HasPyGilBoundRef,
{
/// Borrows a GIL-bound reference to the contained `T`.
///
Expand Down Expand Up @@ -949,7 +959,7 @@ impl<T> Drop for Py<T> {

impl<'a, T> FromPyObject<'a> for Py<T>
where
T: PyTypeInfo,
T: HasPyGilBoundRef,
&'a T::AsRefTarget: FromPyObject<'a>,
T::AsRefTarget: 'a + AsPyPointer,
{
Expand All @@ -968,14 +978,14 @@ where
/// Use .as_ref() to get the GIL-scoped error if you need to inspect the cause.
impl<T> std::error::Error for Py<T>
where
T: std::error::Error + PyTypeInfo,
T: std::error::Error + HasPyGilBoundRef,
T::AsRefTarget: std::fmt::Display,
{
}

impl<T> std::fmt::Display for Py<T>
where
T: PyTypeInfo,
T: HasPyGilBoundRef,
T::AsRefTarget: std::fmt::Display,
{
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
Expand Down
2 changes: 1 addition & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,7 @@ pub use crate::err::{PyDowncastError, PyErr, PyErrArguments, PyResult};
#[cfg(not(PyPy))]
pub use crate::gil::{prepare_freethreaded_python, with_embedded_python_interpreter};
pub use crate::gil::{GILGuard, GILPool};
pub use crate::instance::{Py, PyNativeType, PyObject};
pub use crate::instance::{HasPyGilBoundRef, Py, PyNativeType, PyObject};
pub use crate::marker::Python;
pub use crate::pycell::{PyCell, PyRef, PyRefMut};
pub use crate::pyclass::PyClass;
Expand Down
6 changes: 3 additions & 3 deletions src/pyclass.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! `PyClass` and related traits.
use crate::{
callback::IntoPyCallbackOutput, ffi, impl_::pyclass::PyClassImpl, IntoPy, IntoPyPointer,
PyCell, PyObject, PyResult, PyTypeInfo, Python,
callback::IntoPyCallbackOutput, ffi, impl_::pyclass::PyClassImpl, HasPyGilBoundRef, IntoPy,
IntoPyPointer, PyCell, PyObject, PyResult, PyTypeInfo, Python,
};
use std::{cmp::Ordering, os::raw::c_int};

Expand All @@ -16,7 +16,7 @@ pub use self::gc::{PyTraverseError, PyVisit};
/// The `#[pyclass]` attribute implements this trait for your Rust struct -
/// you shouldn't implement this trait directly.
pub trait PyClass:
PyTypeInfo<AsRefTarget = PyCell<Self>> + PyClassImpl<Layout = PyCell<Self>>
PyTypeInfo + PyClassImpl<Layout = PyCell<Self>> + HasPyGilBoundRef<AsRefTarget = PyCell<Self>>
{
/// Whether the pyclass is frozen.
///
Expand Down
5 changes: 1 addition & 4 deletions src/type_object.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
//! Python type object information

use crate::types::{PyAny, PyType};
use crate::{ffi, AsPyPointer, PyNativeType, Python};
use crate::{ffi, AsPyPointer, Python};

/// `T: PyLayout<U>` represents that `T` is a concrete representation of `U` in the Python heap.
/// E.g., `PyCell` is a concrete representation of all `pyclass`es, and `ffi::PyObject`
Expand Down Expand Up @@ -40,9 +40,6 @@ pub unsafe trait PyTypeInfo: Sized {
/// Module name, if any.
const MODULE: Option<&'static str>;

/// Utility type to make Py::as_ref work.
type AsRefTarget: PyNativeType;

/// Returns the PyTypeObject instance for this type.
fn type_object_raw(py: Python<'_>) -> *mut ffi::PyTypeObject;

Expand Down
18 changes: 1 addition & 17 deletions src/types/iterator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
//
// based on Daniel Grunwald's https://github.com/dgrunwald/rust-cpython

use crate::{ffi, AsPyPointer, IntoPyPointer, Py, PyAny, PyErr, PyNativeType, PyResult, Python};
use crate::{ffi, AsPyPointer, PyAny, PyErr, PyResult, Python};
use crate::{PyDowncastError, PyTryFrom};

/// A Python iterator object.
Expand Down Expand Up @@ -85,22 +85,6 @@ impl<'v> PyTryFrom<'v> for PyIterator {
}
}

impl Py<PyIterator> {
/// Borrows a GIL-bound reference to the PyIterator. By binding to the GIL lifetime, this
/// allows the GIL-bound reference to not require `Python` for any of its methods.
pub fn as_ref<'py>(&'py self, _py: Python<'py>) -> &'py PyIterator {
let any = self.as_ptr() as *const PyAny;
unsafe { PyNativeType::unchecked_downcast(&*any) }
}

/// Similar to [`as_ref`](#method.as_ref), and also consumes this `Py` and registers the
/// Python object reference in PyO3's object storage. The reference count for the Python
/// object will not be decreased until the GIL lifetime ends.
pub fn into_ref(self, py: Python<'_>) -> &PyIterator {
unsafe { py.from_owned_ptr(self.into_ptr()) }
}
}

#[cfg(test)]
mod tests {
use super::PyIterator;
Expand Down
20 changes: 1 addition & 19 deletions src/types/mapping.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,7 @@ use crate::err::{PyDowncastError, PyErr, PyResult};
use crate::once_cell::GILOnceCell;
use crate::type_object::PyTypeInfo;
use crate::types::{PyAny, PySequence, PyType};
use crate::{
ffi, AsPyPointer, IntoPy, IntoPyPointer, Py, PyNativeType, PyTryFrom, Python, ToPyObject,
};
use crate::{ffi, AsPyPointer, IntoPy, Py, PyNativeType, PyTryFrom, Python, ToPyObject};

static MAPPING_ABC: GILOnceCell<PyResult<Py<PyType>>> = GILOnceCell::new();

Expand Down Expand Up @@ -161,22 +159,6 @@ impl<'v> PyTryFrom<'v> for PyMapping {
}
}

impl Py<PyMapping> {
/// Borrows a GIL-bound reference to the PyMapping. By binding to the GIL lifetime, this
/// allows the GIL-bound reference to not require `Python` for any of its methods.
pub fn as_ref<'py>(&'py self, _py: Python<'py>) -> &'py PyMapping {
let any = self.as_ptr() as *const PyAny;
unsafe { PyNativeType::unchecked_downcast(&*any) }
}

/// Similar to [`as_ref`](#method.as_ref), and also consumes this `Py` and registers the
/// Python object reference in PyO3's object storage. The reference count for the Python
/// object will not be decreased until the GIL lifetime ends.
pub fn into_ref(self, py: Python<'_>) -> &PyMapping {
unsafe { py.from_owned_ptr(self.into_ptr()) }
}
}

#[cfg(test)]
mod tests {
use std::collections::HashMap;
Expand Down
6 changes: 4 additions & 2 deletions src/types/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,10 @@ macro_rules! pyobject_native_type_base(
($name:ty $(;$generics:ident)* ) => {
unsafe impl<$($generics,)*> $crate::PyNativeType for $name {}

unsafe impl $crate::HasPyGilBoundRef for $name {
type AsRefTarget = Self;
}

impl<$($generics,)*> ::std::fmt::Debug for $name {
fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>)
-> ::std::result::Result<(), ::std::fmt::Error>
Expand Down Expand Up @@ -177,8 +181,6 @@ macro_rules! pyobject_native_type_named (
macro_rules! pyobject_native_type_info(
($name:ty, $typeobject:expr, $module:expr $(, #checkfunction=$checkfunction:path)? $(;$generics:ident)*) => {
unsafe impl<$($generics,)*> $crate::type_object::PyTypeInfo for $name {
type AsRefTarget = Self;

const NAME: &'static str = stringify!($name);
const MODULE: ::std::option::Option<&'static str> = $module;

Expand Down
28 changes: 1 addition & 27 deletions src/types/sequence.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ use crate::once_cell::GILOnceCell;
use crate::type_object::PyTypeInfo;
use crate::types::{PyAny, PyList, PyString, PyTuple, PyType};
use crate::{ffi, PyNativeType, ToPyObject};
use crate::{AsPyPointer, IntoPy, IntoPyPointer, Py, Python};
use crate::{AsPyPointer, IntoPy, Py, Python};
use crate::{FromPyObject, PyTryFrom};

static SEQUENCE_ABC: GILOnceCell<PyResult<Py<PyType>>> = GILOnceCell::new();
Expand Down Expand Up @@ -359,32 +359,6 @@ impl<'v> PyTryFrom<'v> for PySequence {
}
}

impl Py<PySequence> {
/// Borrows a GIL-bound reference to the PySequence. By binding to the GIL lifetime, this
/// allows the GIL-bound reference to not require `Python` for any of its methods.
///
/// ```
/// # use pyo3::prelude::*;
/// # use pyo3::types::{PyList, PySequence};
/// # Python::with_gil(|py| {
/// let seq: Py<PySequence> = PyList::empty(py).as_sequence().into();
/// let seq: &PySequence = seq.as_ref(py);
/// assert_eq!(seq.len().unwrap(), 0);
/// # });
/// ```
pub fn as_ref<'py>(&'py self, _py: Python<'py>) -> &'py PySequence {
let any = self.as_ptr() as *const PyAny;
unsafe { PyNativeType::unchecked_downcast(&*any) }
}

/// Similar to [`as_ref`](#method.as_ref), and also consumes this `Py` and registers the
/// Python object reference in PyO3's object storage. The reference count for the Python
/// object will not be decreased until the GIL lifetime ends.
pub fn into_ref(self, py: Python<'_>) -> &PySequence {
unsafe { py.from_owned_ptr(self.into_ptr()) }
}
}

#[cfg(test)]
mod tests {
use crate::types::{PyList, PySequence};
Expand Down