Add support and CI testing for Python 3.8

[hugovk]

Released on 2019-10-14 🎉

• https://discuss.python.org/t/python-3-8-0-is-now-available/2478

Not yet available on Azure Pipelines:

• https://github.com/microsoft/azure-pipelines-image-generation/issues/1317

Nor Conda, so "Linux - Python 3.8" is left out from Travis CI for now:

• python v3.8.0 conda-forge/python-feedstock#274

Reminders

• Run make format and make check to make sure the code follows the style guide.
• [n/a] Add tests for new features or tests that would have caught the bug that you're fixing.
• [n/a] Add new public functions/methods/classes to doc/api/index.rst.
• [n/a] Write detailed docstrings for all functions/methods.
• [n/a] If adding new functionality, add an example to the docstring, gallery, and/or tutorials.

[leouieda]

Thanks @hugovk! It's probably best to wait until we can get conda-forge binaries for everything and test that it actually works :) Alternatively, we could add a configuration script to fatiando/continuous-integration to setup the dependencies with pip.

[leouieda]

Add the WIP flag to make sure we don't merge this in just yet accidentally.

[hugovk]

Yep, I think it's best to wait, they'll add them soon enough.

[leouieda]

@hugovk I enabled the 3.8 builds on CI since conda-forge already has all our dependencies on 3.8. Let's see if everything passes 🤞

[leouieda]

No yet, I guess. We can try rebuilding in a while as we wait for the conda-forge migration to finish.

[dopplershift]

That would imply that either a runtime or test dependency is missing on Mac/Windows. Would love to know what’s missing to see what we need to fix.

[leouieda]

@dopplershift the runtime dependencies are fine since the pooch feedstock was already rebuilt for 3.8 on conda-forge. The problem are the development dependencies. This is the output from the Travis Linux build:

UnsatisfiableError: The following specifications were found to be incompatible with a past

explicit spec that is not an explicit spec in this operation (pip):

  - appdirs -> python -> pip
  - appdirs -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - codecov -> coverage -> python[version='3.4.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - codecov -> python -> pip
  - codecov -> python[version='2.7.*|3.5.*|3.6.*']
  - codecov -> requests[version='>=2.7.9'] -> urllib3[version='>=1.21.1,<1.25'] -> ipaddress -> python[version='<=3.3']
  - coverage -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - coverage -> python[version='>=2.7,<2.8.0a0'] -> pip
  - numpydoc -> python -> pip
  - numpydoc -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - numpydoc -> sphinx -> docutils[version='>=0.11'] -> python[version='>=3.8,<3.9.0a0']
  - numpydoc -> sphinx -> python[version='>=3.5']
  - numpydoc -> sphinx -> requests[version='>=2.0.0'] -> urllib3[version='>=1.21.1,<1.25'] -> ipaddress -> python[version='<=3.3']
  - packaging -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - packaging -> python[version='>=3.7,<3.8.0a0'] -> pip
  - packaging -> six -> python[version='>=3.8,<3.9.0a0']
  - pytest -> more-itertools[version='>=4.0'] -> python[version='>=3.4|>=3.8.0a,<3.9.0a0']
  - pytest -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - pytest -> python[version='>=3.7,<3.8.0a0'] -> pip
  - pytest-cov -> coverage -> python[version='>=3.8,<3.9.0a0']
  - pytest-cov -> pytest -> more-itertools[version='>=4.0'] -> python[version='>=3.4|>=3.8.0a,<3.9.0a0']
  - pytest-cov -> python=3.5 -> pip
  - pytest-cov -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - python=3.8 -> pip
  - requests -> chardet[version='>=3.0.2,<3.1.0'] -> python
  - requests -> python=3.4 -> pip
  - requests -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - requests -> urllib3[version='>=1.21.1,<1.25'] -> ipaddress -> python[version='<=3.3']
  - sphinx==1.8.5 -> alabaster[version='>=0.7,<0.8'] -> python[version='2.7.*|3.6.*']
  - sphinx==1.8.5 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - sphinx==1.8.5 -> python[version='>=3.6,<3.7.0a0'] -> pip
  - sphinx==1.8.5 -> requests[version='>=2.0.0'] -> certifi[version='>=2017.4.17'] -> python[version='3.4.*|3.5.*|>=3.5,<3.6.0a0|>=3.8,<3.9.0a0']
  - sphinx==1.8.5 -> requests[version='>=2.0.0'] -> urllib3[version='>=1.21.1,<1.24'] -> ipaddress -> python[version='<=3.3']
  - sphinx_rtd_theme -> python=2.7 -> pip
  - sphinx_rtd_theme -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

Azure Mac:

UnsatisfiableError: The following specifications were found to be incompatible with a past
explicit spec that is not an explicit spec in this operation (python):

  - appdirs -> python -> pip
  - codecov -> coverage -> python[version='3.4.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - codecov -> python=3.6 -> pip
  - codecov -> python[version='2.7.*|3.5.*|3.6.*']
  - codecov -> requests[version='>=2.7.9'] -> urllib3[version='>=1.21.1,<1.22'] -> ipaddress -> python[version='<=3.3']
  - coverage -> python=3.4 -> pip
  - coverage -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - numpydoc -> python=2.7 -> pip
  - numpydoc -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - numpydoc -> sphinx -> docutils -> python[version='>=3.8,<3.9.0a0']
  - numpydoc -> sphinx -> python[version='>=3.5']
  - numpydoc -> sphinx -> requests -> urllib3[version='>=1.21.1,<1.22'] -> ipaddress -> python[version='<=3.3']
  - packaging -> python=3.5 -> pip
  - packaging -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - packaging -> six -> python[version='>=3.8,<3.9.0a0']
  - pytest -> more-itertools[version='>=4.0'] -> python[version='>=3.4|>=3.8.0a,<3.9.0a0']
  - pytest -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - pytest -> python[version='>=3.6,<3.7.0a0'] -> pip
  - pytest-cov -> coverage -> python[version='>=3.8,<3.9.0a0']
  - pytest-cov -> pytest -> more-itertools[version='>=4.0'] -> python[version='>=3.4|>=3.8.0a,<3.9.0a0']
  - pytest-cov -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - pytest-cov -> python[version='>=3.7,<3.8.0a0'] -> pip
  - python=3.8 -> pip
  - requests -> chardet[version='>=3.0.2,<3.1.0'] -> python
  - requests -> python=3.6 -> pip
  - requests -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
  - requests -> urllib3[version='>=1.21.1,<1.22'] -> ipaddress -> python[version='<=3.3']
  - sphinx==1.8.5 -> alabaster[version='>=0.7,<0.8'] -> python[version='2.7.*|3.6.*']
  - sphinx==1.8.5 -> python[version='>=2.7,<2.8.0a0'] -> pip
  - sphinx==1.8.5 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - sphinx==1.8.5 -> requests[version='>=2.0.0'] -> certifi[version='>=2017.4.17'] -> python[version='3.4.*|3.5.*|>=3.5,<3.6.0a0|>=3.8,<3.9.0a0']
  - sphinx==1.8.5 -> requests[version='>=2.0.0'] -> urllib3[version='>=1.21.1,<1.22'] -> ipaddress -> python[version='<=3.3']
  - sphinx_rtd_theme -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - sphinx_rtd_theme -> python[version='>=3.5,<3.6.0a0'] -> pip

and Azure Windows (surprisingly the shortest one 😮):

UnsatisfiableError: The following specifications were found to be in conflict:
  - python=3.8
  - sphinx==1.8.5 -> python[version='>=2.7,<2.8.0a0']

So it seems that our use of Sphinx 1.8.5 is causing trouble. I tried updating but the RTD theme is still not rendering numpydoc arguments correctly with newer sphinx.

[dopplershift]

Yeah, unfortunately on conda-forge, only sphinx>=2 is a noarch package, meaning it doesn't need rebuilding for Python versions. 1.8.5 has only been built for up to 3.7. I've not noticed any problems with our doc builds and Sphinx 2. Do you have an example, or better yet, an upstream issue to point to?

[leouieda]

This was made with 1.8.5:

And this is the result of 2.2.1:

I remember chasing this down a while ago and it seemed like it was a weird compatibility issue between sphinx_rtd_theme, numpydoc, and sphinx 2. At the time, I just pinned sphinx and moved on. There seems to be some recent work towards fixing this in readthedocs/sphinx_rtd_theme#766 so it might get resolved soon. I have yet to try one of the workarounds in that issue, though.

[dopplershift]

Interesting. We don't have that problem with Sphinx 2 on MetPy, but we have pinned to a custom version of sphinx_rtd_theme in order to support the version chooser. So I guess you may be able to get something reasonable with a new sphinx and an older sphinx_rtd_theme? Just food for thought.

Ugh, what a mess.

[leouieda]

We don't have that problem with Sphinx 2 on MetPy

Yeah, I noticed that. I prefer the old look but it's no good to keep using an old sphinx. I'll try pinning the theme and see what happens. Thanks, Ryan!

[leouieda]

Ha of course it wasn't that simple. sphinx_rtd_theme isn't available on conda-forge for 3.7 in those versions. Buuuut, it turns out that using sphinx-ext-napoleon instead of numpydoc kind of fixes the problem!

It looks different but not necessarily worse. Bonus that this works with the most recent sphinx and rtd theme. I'd be fine with this if it means we can stop pinning sphinx to 1.8.5.

Any contradictory opinions?

cc @santisoler

[santisoler]

I think it even looks better with napoleon. Congratulations for solving this! 🎉

[leouieda]

🎉 all builds passing! Merging this in. Thanks @hugovk and everyone for all the help!