Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix building for Python 3.8 #765

Merged
merged 2 commits into from
Dec 10, 2019
Merged

Conversation

bryanwweber
Copy link
Member

Thanks for contributing code! Please include a description of your change and
check your PR against the list below (for further questions, refer to the
contributing guide).

  • There is a clear use-case for this code change
  • The commit message has a short title & references relevant issues
  • Build passes (scons build & scons test) and unit tests address code coverage

Changes proposed in this pull request

  • Bump the minimum Cython version when building for Python 3.8
  • Disable deprecation warnings on macOS when compiling _cantera.cpp that are caused by a deprecated feature that Cython handles

@bryanwweber bryanwweber changed the title Fix buliding for Python 3.8 Fix building for Python 3.8 Dec 5, 2019
@speth
Copy link
Member

speth commented Dec 6, 2019

Do the compiler warnings go away with Cython 0.29.14? If so, can we just set that to the minimum version and not have to worry about disabling deprecation warnings?

@bryanwweber
Copy link
Member Author

For reference, I'm using Python 3.8.0 on macOS and Linux built from the source tarball by pyenv, and on Windows Python 3.8.0 from the Anaconda distribution.

The warnings are present only on macOS and only with Cython 0.29.14. For reference, these are the warnings I get:

g++ -o build/temp-py/_cantera.os -c -std=c++0x -O3 -Wno-inline -g -fPIC -DNDEBUG -Iinclude -I/Users/bryan/.pyenv/versions/3.8.0/include/python3.8 -I/Users/bryan/.pyenv/versions/cantera-dev/lib/python3.8/site-packages/numpy/core/include -I/Users/bryan/miniconda3/envs/cantera-dev/include interfaces/cython/cantera/_cantera.cpp
... (NumPy warnings removed)
interfaces/cython/cantera/_cantera.cpp:105531:3: warning: 'tp_print' is deprecated [-Wdeprecated-declarations]
  0, /*tp_print*/
  ^
... (hundreds of messages differing only in the source line)

The relevant lines from _cantera.cpp are

  #if PY_VERSION_HEX >= 0x030800b4 && PY_VERSION_HEX < 0x03090000
  0, /*tp_print*/
  #endif

which is basically repeated for each PyTypeObject that is generated.

These warnings are not present on Windows or on Linux for Cython >=0.29.12. With Cython < 0.29.12, the compile of _cantera.cpp fails with (here from Windows, but the same basic reason about __Pyx_PyCode_New happens on at least macOS even if the specific message is different):

cl /Fobuild\temp-py\_cantera.obj /c interfaces\cython\cantera\_cantera.cpp /EHsc /MD /nologo /D_SCL_SECURE_NO_WARNINGS /D_CRT_SECURE_NO_WARNINGS /O2 /Zi /Fdbuild\temp-py\_cantera.obj.pdb /DNDEBUG /Iinclude /IC:\Users\bww09001\AppData\Local\Continuum\Miniconda3\envs\cantera-dev\include /IC:\Users\bww09001\AppData\Local\Continuum\Miniconda3\envs\cantera-dev\lib\site-packages\numpy\core\include /IC:\Users\bww09001\AppData\Local\Continuum\Miniconda3\envs\cantera-dev\Library\Include
_cantera.cpp
... (NumPy warnings removed)
interfaces\cython\cantera\_cantera.cpp(116964): warning C4003: not enough arguments for function-like macro invocation '__Pyx_PyCode_New'
interfaces\cython\cantera\_cantera.cpp(116964): error C2059: syntax error: ')'

Interestingly, Cython 0.29.12 on Windows and 0.29.13 on macOS do not generate the warnings that cython/cython#3171 and cython/cython#3201 were merged to fix. In fact, they don't generate any warnings at all (except the standard NumPy one). 🤷‍♂

Basically, I have no idea what's going on here. The changes in this PR allow building with Python 3.8.0 without spurious warnings. That's all I know 😅

Copy link
Member

@speth speth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of very minor suggestions. Feel free to address as you like.

SConstruct Outdated
Comment on lines 1297 to 1305
elif python_version >= LooseVersion("3.8"):
# Reset the minimum Cython version if the Python version is 3.8 or higher
# Due to internal changes in the CPython API, more recent versions of
# Cython are necessary to build for Python 3.8
cython_min_version = LooseVersion("0.29.12")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be helpful for the comments here to be slightly more clear that there's nothing Cantera-specific about any of this: Only Cython >= 0.29.12 is compatible with Python 3.8, and we're just setting this dependency so that users get a more useful error message than the compilation errors that would otherwise result.

# Only applies to Python 3.8 and Cython < 3.0. The source of the warnings
# will be removed after Python 3.8, so the minimum version of Cython will
# need to change for Python 3.9.
if LooseVersion(py_version) == LooseVersion("3.8"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the documentation for distutils.sysconfig.get_python_version(), this could just be:

Suggested change
if LooseVersion(py_version) == LooseVersion("3.8"):
if py_version == "3.8":

Due to internal C API changes in CPython, newer versions of Cython are
required to compile with Python 3.8 and higher.
The clang compiler on macOS prints a number of warnings about deprecated
fields in the CPython 3.8 API. These warnings are spurious because they
are caused by Cython and will be handled in a future release for Python
3.9.  They also appear to only occur on macOS.
@bryanwweber
Copy link
Member Author

For what it's worth, there were various other internal changes in CPython in the beta period for 3.8, such that the Cython maintainer states that Cython 0.29.12 is the first version compatible with the released CPython 3.8: https://bugs.python.org/issue37221#msg347468

Also related, with more information for future researchers: python/cpython#14193 and https://bugs.python.org/issue37250 and cython/cython#2976

@bryanwweber bryanwweber merged commit 484441a into Cantera:master Dec 10, 2019
@bryanwweber bryanwweber deleted the fix-py-38 branch December 10, 2019 20:02
@codecov
Copy link

codecov bot commented Dec 10, 2019

Codecov Report

Merging #765 into master will increase coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #765      +/-   ##
=========================================
+ Coverage    71.4%   71.4%   +<.01%     
=========================================
  Files         372     372              
  Lines       43859   43859              
=========================================
+ Hits        31316   31317       +1     
+ Misses      12543   12542       -1
Impacted Files Coverage Δ
src/thermo/HMWSoln.cpp 71.58% <0%> (+0.04%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 750a80b...484441a. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants