Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test on Python 3.11 #2423

Closed
jorgensd opened this issue Oct 31, 2022 · 7 comments · Fixed by #2500
Closed

Test on Python 3.11 #2423

jorgensd opened this issue Oct 31, 2022 · 7 comments · Fixed by #2500
Labels
ci Continuous Integration packaging proposal Suggested change or addition

Comments

@jorgensd
Copy link
Member

Python 3.11 has been released: https://docs.python.org/3.11/whatsnew/3.11.html
We should probably add some test suite running over different versions of Python, and add it to the conda CI

@jorgensd jorgensd added proposal Suggested change or addition packaging ci Continuous Integration labels Oct 31, 2022
@tiagovla
Copy link
Contributor

tiagovla commented Nov 8, 2022

The current conda CI fails to install on python 3.11 with:

Run conda create -n env3-11 -c conda-forge python=3.11 fenics-dolfinx mpich
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done

Building graph of deps:   0%|          | 0/7 [00:00<?, ?it/s]
Examining @/linux-6[4](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:5)::__linux==[5](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:6).15.0=0:   0%|          | 0/7 [00:00<?, ?it/s]
Examining @/linux-[6](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:7)4::__glibc==2.35=0:  14%|█▍        | 1/[7](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:8) [00:00<00:00, 4275.54it/s]
Examining mpich:  29%|██▊       | 2/7 [00:00<00:00, 4647.43it/s]                      
Examining python=3.11:  43%|████▎     | 3/7 [00:00<00:00, 12.[8](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:9)7it/s]
Examining python=3.11:  57%|█████▋    | 4/7 [00:00<00:00, 17.14it/s]
Examining @/linux-64::__unix==0=0:  57%|█████▋    | 4/7 [00:00<00:00, 17.14it/s]
Examining @/linux-64::__archspec==1=x86_64:  71%|███████▏  | 5/7 [00:00<00:00, 17.14it/s]
Examining fenics-dolfinx:  86%|████████▌ | 6/7 [00:00<00:00, 17.14it/s]                  
Examining fenics-dolfinx: 100%|██████████| 7/7 [00:20<00:00, 17.14it/s]
                                                                       

Determining conflicts:   0%|          | 0/7 [00:00<?, ?it/s]
Examining conflict for __glibc mpich:   0%|          | 0/7 [00:00<?, ?it/s]
Examining conflict for fenics-dolfinx mpich:  14%|█▍        | 1/7 [00:00<00:00, 35.15it/s]
Examining conflict for fenics-dolfinx mpich python:  2[9](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:10)%|██▊       | 2/7 [00:04<00:11,  2.36s/it]
Examining conflict for fenics-dolfinx mpich python:  43%|████▎     | 3/7 [00:04<00:06,  1.57s/it]
Examining conflict for fenics-dolfinx python:  43%|████▎     | 3/7 [00:23<00:06,  1.57s/it]      
Examining conflict for fenics-dolfinx python:  57%|█████▋    | 4/7 [00:23<00:21,  7.16s/it]
Solving environment: ...working... 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed
                                                                                           
UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package _libgcc_mutex conflicts for:
mpich -> libgcc-ng[version='>=[10](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:11).3.0'] -> _libgcc_mutex[version='*|0.1',build='main|main|conda_forge']
python=3.[11](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:12) -> libgcc-ng[version='>=[12](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:13)'] -> _libgcc_mutex==0.1=conda_forge
fenics-dolfinx -> libgcc-ng[version='>=12'] -> _libgcc_mutex==0.1=conda_forge

Package libstdcxx-ng conflicts for:
fenics-dolfinx -> libstdcxx-ng[version='>=12']
fenics-dolfinx -> mpich[version='>=4.0.2,<5.0a0'] -> libstdcxx-ng[version='>=10.3.0|>=9.4.0|>=7.3.0|>=4.9|>=11.2.0|>=7.5.0|>=9.3.0|>=7.2.0']

Package mpich conflicts for:
fenics-dolfinx -> hdf5[version='>=1.12.2,<1.12.3.0a0',build=mpi_mpich_*] -> mpich[version='3.2.*|>=3.2,<3.3.0a0|>=3.3,<5.0.0a0|>=3.4,<5.0.0a0|>=4.0.1,<5.0a0|>=4.0.2,<4.1.0a0|>=3.3,<3.4.0a0|>=3.4.2,<5.0a0|>=3.4.1,<5.0a0|>=3.3.2,<5.0.0a0|>=3.3.1,<5.0.0a0|>=3.2.1,<3.3.0a0|>=4.0,<4.1.0a0']
mpich
fenics-dolfinx -> mpich[version='>=4.0.2,<5.0a0']

Package libgcc-ng conflicts for:
fenics-dolfinx -> libgcc-ng[version='>=12']
fenics-dolfinx -> cffi -> libgcc-ng[version='>=10.3.0|>=11.2.0|>=4.9|>=7.3.0|>=7.5.0|>=9.3.0|>=9.4.0|>=7.2.0']

Package python conflicts for:
python=3.11
fenics-dolfinx -> cffi -> python[version='2.7.*|3.5.*|3.6.*|3.6.12|3.6.12|3.7.10|3.7.10|3.7.12|3.8.12|3.9.10|>=3.11,<3.12.0a0|>=3.6,<3.7.0a0|3.7.9|3.6.9|3.6.9|3.6.9|>=2.7,<2.8.0a0|3.6.9|>=3.5,<3.6.0a0|3.4.*|>=3.7|3.9.*|3.10.*|3.8.*|3.7.*',build='1_73_pypy|3_73_pypy|4_73_pypy|5_73_pypy|5_73_pypy|1_73_pypy|0_73_pypy|0_73_pypy|2_73_pypy|0_73_pypy']
fenics-dolfinx -> python[version='>=3.10,<3.11.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0']The following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.35=0
  - mpich -> libgcc-ng[version='>=10.3.0'] -> __glibc[version='>=2.[17](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:18)']

Your installed version is: 2.[35](https://github.com/tiagovla/dolfinx/actions/runs/3415902752/jobs/5685509973#step:3:36)


Error: Process completed with exit code 1.```

@minrk
Copy link
Contributor

minrk commented Nov 28, 2022

FWIW, the builds on conda-forge are succeeding, but simple tests are failing with:

Traceback (most recent call last):
  File "/Users/runner/miniforge3/conda-bld/fenics-dolfinx-split_1667515964271/test_tmp/python/demo/demo_poisson.py", line 87, in <module>
    msh = mesh.create_rectangle(comm=MPI.COMM_WORLD,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/miniforge3/conda-bld/fenics-dolfinx-split_1667515964271/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.11/site-packages/dolfinx/mesh.py", line 373, in create_rectangle
    return Mesh.from_cpp(mesh, domain)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/miniforge3/conda-bld/fenics-dolfinx-split_1667515964271/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.11/site-packages/dolfinx/mesh.py", line 56, in from_cpp
    obj.__class__ = Mesh
    ^^^^^^^^^^^^^
TypeError: __class__ assignment: 'Mesh' object layout differs from 'dolfinx.cpp.mesh.Mesh'

Apparently this assignment is not legal in 3.11. I'm not sure how to fix that.

@drew-parsons
Copy link
Contributor

TypeError: class assignment: 'Mesh' object layout differs from 'dolfinx.cpp.mesh.Mesh'

I get the same error with python3.11 on debian.

@francesco-ballarin
Copy link
Member

You may also want to track this issue numba/numba#8304

@drew-parsons
Copy link
Contributor

drew-parsons commented Jan 4, 2023

Printing the class and type of both obj and Mesh at the failure point (mesh.py l.56) sees

from_cpp: 
obj:  obj.__class__= <class 'dolfinx.cpp.mesh.Mesh'>  type(obj)= <class 'dolfinx.cpp.mesh.Mesh'>
Mesh:  Mesh.__class__= <class 'pybind11_builtins.pybind11_type'>  type(Mesh)= <class 'pybind11_builtins.pybind11_type'>

So Mesh certainly is a "different" class to cpp.mesh.Mesh. It's a subclass that introduces ufl_cell() and ufl_domain(), with the addition of obj._ufl_domain changing the data structure.

python3.10 returns the same values. Looks like python3.11 has introduced stricter type checking for the __class__ attribute.

This pybind11 issue might be relevant: pybind/pybind11#1640

@drew-parsons
Copy link
Contributor

A simple workaround is to instantiate a new Mesh using obj properties, rather than changing obj.__class__

return Mesh(obj.comm, obj.topology, obj.geometry, domain)

instead of return obj

There might be smarter pybind11 approach though.

@rmsc
Copy link

rmsc commented Jun 4, 2023

Python 3.11 is now the default on Archlinux, and possibly also on others.

The fix unfortunately didn't make it to the latest release (0.6.0), so mesh handling is now broken for me.

Given that the last release was several months ago, would it make sense to release a 0.6.1 version? Thanks!

matt-graham added a commit to UCL/dxh that referenced this issue Jul 12, 2023
Current v0.6.0 release of DOLFINx appears not support Python 3.11
FEniCS/dolfinx#2423
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Continuous Integration packaging proposal Suggested change or addition
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants