Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculate offsets for weakreflist and dict in PyCell. #1060

Merged
merged 2 commits into from
Jul 21, 2020

Conversation

sebpuetz
Copy link
Contributor

@sebpuetz sebpuetz commented Jul 21, 2020

As demonstrated by #1022, depending on the order of structmembers to calculate offsets in bytes is fragile. This PR adds a dependency to memoffset which provides an interface to find offsets into structs in a more robust manner.

Caveat: This macro is technically unsound, see Gilnaa/memoffset#24, so I'm marking this as a draft for now.

This PR moves the calcualtion of class dict and weakreflist offsets to PyCell rather than initialize_type_object.


Thank you for contributing to pyo3!

Here are some things you should check for submitting your pull request:

  • Run cargo fmt (This is checked by travis ci)
  • Run cargo clippy and check there are no hard errors (There are a bunch of existing warnings; This is also checked by travis)
  • If applicable, add an entry in the changelog.
  • If applicable, add documentation to all new items and extend the guide.
  • If applicable, add tests for all new or fixed functions
  • If you changed any python code, run black .. You can install black with pip install black)

You might want to run tox (pip install tox) locally to check compatibility with all supported python versions. If you're using linux or mac you might find the Makefile helpful for testing.

@sebpuetz
Copy link
Contributor Author

An alternative implementation that makes maintenance easier without pulling in unsoundness:

#[repr(C)]
pub struct PyCell<T: PyClass> {
    inner: PyCellInner<T>,
    thread_checker: T::ThreadChecker,
    // DO NOT CHANGE THE ORDER OF THESE FIELDS WITHOUT CHANGING PyCell::dict_offset()
    // AND PyCell::weakref_offset()
    dict: T::Dict,
    weakref: T::WeakRef,
}

impl<T: PyClass> PyCell<T> {
    /// Get the offset of the dictionary from the start of the struct in bytes.
    pub(crate) fn dict_offset() -> usize {
        mem::size_of::<Self>() - mem::size_of::<T::Dict>() - mem::size_of::<T::WeakRef>()
    }

    /// Get the offset of the weakref list from the start of the struct in bytes.
    pub(crate) fn weakref_offset() -> usize {
        mem::size_of::<Self>() - mem::size_of::<T::WeakRef>()
    }
}

@sebpuetz
Copy link
Contributor Author

@kngwyu

@sebpuetz sebpuetz changed the title Use memoffset to compute the offset of dict and weakref list. Calculate offsets for weakreflist and dict in PyCell. Jul 21, 2020
src/pyclass.rs Outdated
if let Some(dict_offset) = T::Dict::OFFSET {
offset += dict_offset as ffi::Py_ssize_t;
type_object.tp_dictoffset = offset;
if !T::Dict::IS_DUMMY {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about moving this if statement to the PyCell?
I don't like that dict_offset can return a meaningless value even if it doesn't have a dict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I PyCell::*_offset now returns Option<usize>

@kngwyu
Copy link
Member

kngwyu commented Jul 21, 2020

Thanks!

@sebpuetz
Copy link
Contributor Author

Do you want to add a commit with the test case you wrote for #1061?

@sebpuetz sebpuetz marked this pull request as ready for review July 21, 2020 08:38
@kngwyu
Copy link
Member

kngwyu commented Jul 21, 2020

Do you want to add a commit with the test case you wrote for #1061?

Yeah, could you please copy-paste it to this branch?

@sebpuetz
Copy link
Contributor Author

Should be good to go if CI passes

@kngwyu
Copy link
Member

kngwyu commented Jul 21, 2020

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants