Enum with struct variants can only have 12 fields

[MartinJepsen]

Bug Description

I was refactoring some code from

#[pyclass]
enum TupleEnum {
   Variant1(InnerType1),
   Variant2(InnerType2),
}

#[pyclass]
struct InnerType1 {
    field_1: u32,
    field_2: u32,
    field_3: String,
    field_4: f32,
}

...

to this

#[pyclass]
enum StructEnum {
    Variant1 {
        field_1: u32,
        field_2: u32,
        field_3: String,
        field_4: f32,
    },
    Variant2 {
        etc...
    }
}

Effectively getting rid of the wrapped types and representing them in the enum's variants directly.

However, it looks like there is some kind of limit on the number of fields that you can have in an enum member. The example in the "Steps to reproduce" produces this error:

error[E0277]: `(&str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str)` cannot be converted to a Python object
   --> pyanif/src/blocks/beam.rs:373:1
    |
373 | #[pyclass]
    | ^^^^^^^^^^ the trait `pyo3::IntoPyObject<'_>` is not implemented for `(&str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str)`
    |
    = note: `IntoPyObject` is automatically implemented by the `#[pyclass]` macro
    = note: if you do not wish to have a corresponding Python type, implement it manually
    = note: if you do not own `(&str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str)` you can perform a manual conversion to one of the types in `pyo3::types::*`
    = help: the following other types implement trait `pyo3::IntoPyObject<'py>`:
              &(T0, T1)
              &(T0, T1, T2)
              &(T0, T1, T2, T3)
              &(T0, T1, T2, T3, T4)
              &(T0, T1, T2, T3, T4, T5)
              &(T0, T1, T2, T3, T4, T5, T6)
              &(T0, T1, T2, T3, T4, T5, T6, T7)
              &(T0, T1, T2, T3, T4, T5, T6, T7, T8)
            and 17 others
note: required by a bound in `pyo3::IntoPyObjectExt::into_py_any`
   --> /home/majp/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.23.3/src/conversion.rs:367:33
    |
367 | pub trait IntoPyObjectExt<'py>: IntoPyObject<'py> + into_pyobject_ext::Sealed {
    |                                 ^^^^^^^^^^^^^^^^^ required by this bound in `IntoPyObjectExt::into_py_any`
...
380 |     fn into_py_any(self, py: Python<'py>) -> PyResult<Py<PyAny>> {
    |        ----------- required by a bound in this associated function
    = note: this error originates in the attribute macro `pyclass` (in Nightly builds, run with -Z macro-backtrace for more info)

Steps to Reproduce

// my_module.rs

use pyo3::prelude::*

enum StructEnum {
    Variant1 {
        field_1: u32,
        field_2: f32,
        field_3: String,
        field_4: f32,
        field_5: f32,
        field_6: u32,
        field_7: u32,
        field_8: u32,
        field_9: u32,
        field_10: u32,
        field_11: u32,
        field_12: u32,
        field_13: u32,
        field_14: u32,
        field_15: u32,
    }
}

won't compile. However, with 12 fields, there is no error:

#[pyclass]
enum StructEnum {
    Variant1 {
        field_1: u32,
        field_2: f32,
        field_3: String,
        field_4: f32,
        field_5: f32,
        field_6: u32,
        field_7: u32,
        field_8: u32,
        field_9: u32,
        field_10: u32,
        field_11: u32,
        field_12: u32,
        // field_13: u32,
        // field_14: u32,
        // field_15: u32,
    }
}

Backtrace

error[E0277]: `(&str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str)` cannot be converted to a Python object
   --> pyanif/src/blocks/beam.rs:373:1
    |
373 | #[pyclass]
    | ^^^^^^^^^^ the trait `pyo3::IntoPyObject<'_>` is not implemented for `(&str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str)`
    |
    = note: `IntoPyObject` is automatically implemented by the `#[pyclass]` macro
    = note: if you do not wish to have a corresponding Python type, implement it manually
    = note: if you do not own `(&str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str, &str)` you can perform a manual conversion to one of the types in `pyo3::types::*`
    = help: the following other types implement trait `pyo3::IntoPyObject<'py>`:
              &(T0, T1)
              &(T0, T1, T2)
              &(T0, T1, T2, T3)
              &(T0, T1, T2, T3, T4)
              &(T0, T1, T2, T3, T4, T5)
              &(T0, T1, T2, T3, T4, T5, T6)
              &(T0, T1, T2, T3, T4, T5, T6, T7)
              &(T0, T1, T2, T3, T4, T5, T6, T7, T8)
            and 17 others
note: required by a bound in `pyo3::IntoPyObjectExt::into_py_any`
   --> /home/majp/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.23.3/src/conversion.rs:367:33
    |
367 | pub trait IntoPyObjectExt<'py>: IntoPyObject<'py> + into_pyobject_ext::Sealed {
    |                                 ^^^^^^^^^^^^^^^^^ required by this bound in `IntoPyObjectExt::into_py_any`
...
380 |     fn into_py_any(self, py: Python<'py>) -> PyResult<Py<PyAny>> {
    |        ----------- required by a bound in this associated function
    = note: this error originates in the attribute macro `pyclass` (in Nightly builds, run with -Z macro-backtrace for more info)

Your operating system and version

Ubuntu 24.04

Your Python version (python --version)

Python 3.12.3

Your Rust version (rustc --version)

rustc 1.83.0 (90b35a623 2024-11-26)

Your PyO3 version

0.23.3

How did you install python? Did you use a virtualenv?

No venv, just the python3 that comes with Ubuntu 24.04.

Additional Info

Thanks for this awesome project. I imagine creating this is an insane task. So lots of appreciation from here :)

[LilyFoote]

Hi @MartinJepsen!

Unfortunately, this is caused by a limitation of Rust itself. Specifically, Rust does not support Variadic Generics, so we follow the Rust standard library pattern of supporting up to length 12 tuples. See also https://gist.github.com/PoignardAzur/aea33f28e2c58ffe1a93b8f8d3c58667.

Can I ask why you want to refactor away from the wrapped types? That seems to me a better pattern than expanding the support in PyO3 to longer tuples.

[Icxolu]

I agree that adding support for larger tuples is not a helpful. However the fundamental problem seems to be caused merely by our expansion for __match__args__ support here:

pyo3/pyo3-macros-backend/src/pyclass.rs

Lines 1198 to 1201 in 3965f5f

	#[allow(non_upper_case_globals)]
	const #ident: ( #(#args_tp,)* ) = (
	#(stringify!(#field_names),)*
	);

which expands to something like

#[classattr]
const __match_args__: (&str, ..., &str) = ("field_1", ..., "field_15");

in this case, which relies on the IntoPyObject impl for tuples. We could construct the attribute manually without going through Rust tuples like so:

#[classattr]
fn __match_args__(py: Python<'_>) -> PyResult<Bound<'_, PyTuple>> {
    PyTuple::new(py, ["field_1", ..., "field_15"])
}

I prototyped this in #4832

[MartinJepsen]

Can I ask why you want to refactor away from the wrapped types? That seems to me a better pattern than expanding the support in PyO3 to longer tuples.

I just thought it looked a bit cleaner. I'm basically deserializing the structs from binary data, so instead of implementing a From for each enum variant, I can generalize it to the enum itself. Total noob at Rust, so maybe I should change it back :)

I don't expect support for longer tuples. If it's a limitation of Rust, then that's completely fine and not something pyo3 should solve. I was mostly wondering about the reason for this behaviour. Thank you for the explanation!