ffi:PyFrameObject doesn't match the structure in CPython

[alex]

Bug Description

https://github.com/PyO3/pyo3/blob/main/pyo3-ffi/src/cpython/frameobject.rs#L20 is the ffi's definition PyFrameObject, however this doesn't match CPython's (3.8): https://github.com/python/cpython/blob/3.8/Include/frameobject.h#L16

Notably the f_exc_type, f_exc_value, f_exc_traceback fields were removed in 3.7 (I believe), and the f_trace_lines and f_trace_opcodes fields were added at some point.

Steps to Reproduce

1. Look at the links I shared :-)

Any attempt to actually use these fields would result in UB/a segfault.

Backtrace

No response

Your operating system and version

all

Your Python version (python --version)

3.8

Your Rust version (rustc --version)

any

Your PyO3 version

HEAD

How did you install python? Did you use a virtualenv?

Not relevant

Additional Info

I suspect there are other places where the struct definitions do not match upstream. It may make sense to add some sort of test to CI to verify they match on all the configurations tested in CI. The simplest version might be simply verifying that struct sizes match.

[messense]

Tried ctest2 but running into issue with its old libsyntax dependency that can not parse pyo3-ffi code.

[davidhewitt]

It might be nice to use ctest2 in the future, for now I threw together https://github.com/davidhewitt/pyo3-ffi-check which crudely just checks the size of the generated structs, which will be much better having nothing. @messense if you think it makes sense, I'll transfer it to the PyO3 organisation.

Unfortunately it looks like we might have quite a few incorrect definitions! https://github.com/davidhewitt/pyo3-ffi-check/runs/6710130891?check_suite_focus=true

I'll get working on a patch to get some of those runs green :)

[alex]

Ooof, well, glad I suggested it :-/ Thanks for diving in.

…

On Thu, Jun 2, 2022 at 10:37 AM David Hewitt ***@***.***> wrote: It might be nice to use ctest2 in the future, for now I threw together https://github.com/davidhewitt/pyo3-ffi-check which crudely just checks the size of the generated structs, which will be much better having nothing. @messense if you think it makes sense, I'll transfer it to the PyO3 organisation. Unfortunately it looks like we might have quite a few incorrect definitions! https://github.com/davidhewitt/pyo3-ffi-check/runs/6710130891?check_suite_focus=true I'll get working on a patch to get some of those runs green :) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

[messense]

Great work on pyo3-ffi-check, the ctest2 issue looks like won't be fixed soon, so it makes sense to take advantage of pyo3-ffi-check now.

[davidhewitt]

👍 I'll move pyo3-ffi-check into the PyO3 org now!