Python 3.10 waiting list/status

[ZLLentz]

I'm going to periodically check the travis cron to keep up with which packages are preventing us from getting to python 3.10, updating here.

11/8/2021 status:

• ipython has no python 3.10 builds
• mamba stops early before checking later dependencies (alphabetically) and sub-dependencies

[ZLLentz]

Reminder to self: by the time 3.10 is ready, it'll also be time to drop 3.7 support (scheduled for end of 2021 in NEP 29)
This will require a PR to pcds-ci-helpers such that:

• the latest-only build will run 3.10 only
• the full build will run 3.8, 3.9, and 3.10

[ZLLentz]

11/18/2021 status:

• pyca is locked to a specific 3.9 build due to the epicscorelibs versioning issues
• I think that might be the only thing but I may be misinterpreting the mamba output

[klauer]

That's pretty great... maybe we should get back to pyca, then.
It'd be really cool - but wholly unnecessary at this point - to have a pcds-3.10 env to poke around with.

[ZLLentz]

I've been considering if it's worth specifying alternate env builds for testing/poking around with that remove the packages with low compatibility. Some examples might be:

• Python versions (including bleeding edge/nightly)
• Alternate platforms (Windows, OSX)
• The cron dev build

I'm not sure the cleanest way to do these

[ZLLentz]

6/6/2022 status is this perplexing message:

Encountered problems while solving:
  - package docutils-0.16-py27h8c360ce_1 requires python_abi 2.7.* *_cp27mu, but none of the providers can be installed

I think all previously known incompatibilities have been resolved

[ZLLentz]

After removing the docutils pin:

Encountered problems while solving:
  - package bluesky-queueserver-0.0.10-py37h89c1867_0 requires python_abi 3.7.* *_cp37m, but none of the providers can be installed

[ZLLentz]

I guess we still have dependencies that haven't made any py3.10 builds oh well

[ZLLentz]

py310 now fails on ipython which doesn't make any sense to me since it has a py310 build

[ZLLentz]

The build errors are still confusing. The latest try gives:

Encountered problems while solving:
  - package epicscorelibs-7.0.6.99.2.0-py37hb62d391_0 requires python_abi 3.7.* *_cp37m, but none of the providers can be installed

Like the previous failure, this is a package that has an available python 3.10 version. I suspect something that depends on epicscorelibs must have some strict pinning that excludes python 3.10.

[ZLLentz]

After quick investigation, this is not the case. All the packages that depend on epicscorelibs are installable in 3.10. I'm going to detour myself and create a script to figure out what's going on because I want to be allowed to do py3.10 at some point.

[ZLLentz]

The following packages on our conda packages list can't be installed individually along with py310:

• hxrsnd
• scikit-image=0.17.0

For everything else, conda lets you do the install.

[klauer]

hxrsnd is noarch - what's happening there?

• https://anaconda.org/pcds-tag/hxrsnd/files
• https://github.com/pcdshub/hxrsnd/blob/master/conda-recipe/meta.yaml#L12

[ZLLentz]

Ok actually my script was really crude and hxrsnd passes on retry

It's just the precise tag on scikit-image that holds us to py39. That means we can remove the pin and deploy a beta py10 env to try out whenever we want to.

[ZLLentz]

Ok there is actually something to the hxrsnd one, you can install it on 3.10 but not the latest tag on 3.10 because the recipe explicitly forbids it 🤦
https://github.com/pcdshub/hxrsnd/blob/5e1201bbb43d535d8c30a4fbf967c253b9d7dd00/conda-recipe/meta.yaml#L17

[ZLLentz]

It looks like a misguided past me decided that it was better to set the recipe upper pin than to engage with the issues. Mistake.

[ZLLentz]

Resulting rabbit hole leads to moved imports in python's collections module pcdshub/hxrsnd#122 (comment)

[ZLLentz]

Is it now possible to create a python 3.10 environment! Let's run some tests on it and potentially deploy a py310 folder with pcds-6.0.0 for beta testing.

[ZLLentz]

First pass at the tests fails on some technicalities without making much headway: /cds/home/z/zlentz/github/pcds-envs/scripts/test_py310_001.txt
But it's a start

[klauer]

From the logs, it looks like pytest isn't even running? At least, I don't see any pytest output and it exits with code 1, hmm

[ZLLentz]

Yeah it's a silent version conflict between pytest-qt and pytest. I thought this was only in an old version so I'm surprised to see it back. We probably have one or both of these pinned and it's causing an issue.

The other break at the bottom of the log is the pyca/epicscorelibs fiasco

[ZLLentz]

The latest pyca and p4p modules currently require different incompatible versions of epicscorelibs. First, mamba installs the one that pyca needs, then pip removes it in favor of the one p4p needs.

[ZLLentz]

The following packages failed all retries:
tests/lucid
tests/pcdswidgets
tests/pyca
tests/typhos
The following packages had race conditions, but passed at least once:
tests/lightpath
Skipping extra-tests.sh, did not input an environment name as argument 1
pyca 3.2.0 has requirement epicscorelibs<7.0.6.99.3,>=7.0.6.99.2.0, but you have epicscorelibs 7.0.7.99.0.0.

• lucid has test failures that look like they must be a qt api change, but as far as I can tell it is running the same version as the py3.9 version
• pcdswidgets passes a float into an int field and qt is no longer being a bro and converting it for us (???)
• typhos segfaults every time during test_positioner_widget_with_signal_limits

py3.10: conda list | grep qt

pyqt                      5.12.3          py310hff52083_8    conda-forge
pyqt-impl                 5.12.3          py310h1f8e252_8    conda-forge
pyqt5-sip                 4.19.18         py310h122e73d_8    conda-forge
pyqtads                   3.8.2           py310hd8f1fbe_0    conda-forge
pyqtchart                 5.12            py310hfcd6d55_8    conda-forge
pyqtgraph                 0.12.4             pyhd8ed1ab_0    conda-forge
pyqtwebengine             5.12.1          py310hfcd6d55_8    conda-forge
pytest-qt                 4.2.0              pyhd8ed1ab_0    conda-forge
qt                        5.12.9               h1304e3e_6    conda-forge
qtawesome                 1.2.1              pyhd8ed1ab_0    conda-forge
qtconsole                 5.4.0              pyhd8ed1ab_0    conda-forge
qtconsole-base            5.4.0              pyha770c72_0    conda-forge
qtpy                      2.1.0              pyhd8ed1ab_0    conda-forge
qtpyinheritance           0.0.2              pyhd8ed1ab_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge

py3.9: conda list | grep qt

conda list | grep qt
pyqt                      5.12.3           py39hf3d152e_8    conda-forge
pyqt-impl                 5.12.3           py39hde8b62d_8    conda-forge
pyqt5-sip                 4.19.18          py39he80948d_8    conda-forge
pyqtads                   3.8.2            py39h5a03fae_0    conda-forge
pyqtchart                 5.12             py39h0fcd23e_8    conda-forge
pyqtgraph                 0.12.4             pyhd8ed1ab_0    conda-forge
pyqtwebengine             5.12.1           py39h0fcd23e_8    conda-forge
pytest-qt                 3.3.0                      py_0    conda-forge
qt                        5.12.9               h1304e3e_6    conda-forge
qtawesome                 1.1.1              pyhd8ed1ab_0    conda-forge
qtconsole                 5.3.1              pyhd8ed1ab_0    conda-forge
qtconsole-base            5.3.1              pyha770c72_0    conda-forge
qtpy                      2.1.0              pyhd8ed1ab_0    conda-forge
qtpyinheritance           0.0.2              pyhd8ed1ab_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge

[ZLLentz]

Based on the above, it's probably safe to expand the standard test suites to check py3.10, but it can also wait until we resolve these items.

[ZLLentz]

There is a pattern for python 3.10-related failures in the qt modules: whenever the qt docs says that something must be an int, they now must be an int and not a float, or else bad things happen. Previously it looks like something was converting these to ints for us.

This covers all the test failures and the interactive failures I've observed so far, except for typhos's segfault which I haven't looked at yet.

Here's an example from clicking around in lucid:

[2022-11-17 09:28:20,394] [ERROR   ] - An uncaught exception happened: arguments did not match any overloaded call:
  QPoint(): too many arguments
  QPoint(int, int): argument 1 has unexpected type 'float'
  QPoint(QPoint): argument 1 has unexpected type 'float'
Traceback (most recent call last):
  File "/cds/home/z/zlentz/miniconda3/envs/pcds-6.0.0/bin/lucid", line 10, in <module>
    sys.exit(main())
  File "/cds/home/z/zlentz/github/lucid/lucid/launcher.py", line 255, in main
    launch(**kwargs)
  File "/cds/home/z/zlentz/github/lucid/lucid/launcher.py", line 243, in launch
    app.exec_()
  File "/cds/home/z/zlentz/miniconda3/envs/pcds-6.0.0/lib/python3.10/site-packages/pydm/widgets/byte.py", line 46, in paintEvent
    self._painter.drawEllipse(QPoint(w / 2.0, h / 2.0), r, r)
TypeError: arguments did not match any overloaded call:
  QPoint(): too many arguments
  QPoint(int, int): argument 1 has unexpected type 'float'
  QPoint(QPoint): argument 1 has unexpected type 'float'
[2022-11-17 09:28:20,560] [INFO    ] - Saved screenshot: /tmp/43a651da.1.png
[2022-11-17 09:28:22,740] [INFO    ] - Saved screenshot: /tmp/43a651da.2.LUCID_-_TMO.png
QPainter::begin: Painter already active
Segmentation fault (core dumped)

I don't know the best way to go about tracking these down or if there's some other way to address this. I'll put patches in for lucid and pcdswidgets since I identified it already but after that I want to slow down and consider systematic ways to either fix this everywhere or if there's some better way to approach the problem.

[ZLLentz]

We can't update to 3.10 until we either fix all the places this happens in pydm, typhos, etc. or find a different way to address it (different libraries? versions? builds?)

[ZLLentz]

I have not yet identified precisely which change is causing the new behavior.

[klauer]

Seems likely that it'd be in sip (the C++ wrapping tool which backs pyqt5). This will be a difficult one to fix generally and not in a whack-a-mole style - perhaps our best tool will be static analysis with pyright/mypy/etc.

[ZLLentz]

I agree that whack-a-mole isn't great here. I'm whacking a few obvious ones today under the principle that I can't unsee them but I'm not going to charge forward further without a tool.

[ZLLentz]

Linking the root cause we found in slack for completeness:
https://docs.python.org/3/whatsnew/3.10.html#other-language-changes

Builtin and extension functions that take integer arguments no longer accept Decimals, Fractions and other objects that can be converted to integers only with a loss (e.g. that have the int() method but do not have the index() method). (Contributed by Serhiy Storchaka in bpo-37999.)

I've also now noticed https://docs.python.org/3/whatsnew/3.10.html#porting-to-python-3-10 which doesn't mention the above but is worth a quick skim

[ZLLentz]

The only blockage for this now (pending test suite failures) is the previous comment.
A python 3.10 build now can be created with no environment-level build issues.