-
Notifications
You must be signed in to change notification settings - Fork 784
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Calculate offsets for weakreflist and dict in PyCell. #1060
Conversation
An alternative implementation that makes maintenance easier without pulling in unsoundness: #[repr(C)]
pub struct PyCell<T: PyClass> {
inner: PyCellInner<T>,
thread_checker: T::ThreadChecker,
// DO NOT CHANGE THE ORDER OF THESE FIELDS WITHOUT CHANGING PyCell::dict_offset()
// AND PyCell::weakref_offset()
dict: T::Dict,
weakref: T::WeakRef,
}
impl<T: PyClass> PyCell<T> {
/// Get the offset of the dictionary from the start of the struct in bytes.
pub(crate) fn dict_offset() -> usize {
mem::size_of::<Self>() - mem::size_of::<T::Dict>() - mem::size_of::<T::WeakRef>()
}
/// Get the offset of the weakref list from the start of the struct in bytes.
pub(crate) fn weakref_offset() -> usize {
mem::size_of::<Self>() - mem::size_of::<T::WeakRef>()
}
} |
src/pyclass.rs
Outdated
if let Some(dict_offset) = T::Dict::OFFSET { | ||
offset += dict_offset as ffi::Py_ssize_t; | ||
type_object.tp_dictoffset = offset; | ||
if !T::Dict::IS_DUMMY { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about moving this if statement to the PyCell
?
I don't like that dict_offset
can return a meaningless value even if it doesn't have a dict.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, I PyCell::*_offset
now returns Option<usize>
Thanks! |
Do you want to add a commit with the test case you wrote for #1061? |
Yeah, could you please copy-paste it to this branch? |
Should be good to go if CI passes |
Thanks! |
As demonstrated by #1022, depending on the order of structmembers to calculate offsets in bytes is fragile.
This PR adds a dependency tomemoffset
which provides an interface to find offsets into structs in a more robust manner.Caveat: This macro is technically unsound, see Gilnaa/memoffset#24, so I'm marking this as a draft for now.This PR moves the calcualtion of class dict and weakreflist offsets to
PyCell
rather thaninitialize_type_object
.Thank you for contributing to pyo3!
Here are some things you should check for submitting your pull request:
cargo fmt
(This is checked by travis ci)cargo clippy
and check there are no hard errors (There are a bunch of existing warnings; This is also checked by travis)black .
. You can install black withpip install black
)You might want to run
tox
(pip install tox
) locally to check compatibility with all supported python versions. If you're using linux or mac you might find the Makefile helpful for testing.