Add jupyter-book

[chrisjsewell]

Checklist

• Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml"
• License file is packaged (see here for an example)
• Source is from official source
• Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged)
• If static libraries are linked in, the license of the static library is packaged.
• Build number is 0
• A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details)
• GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there

[chrisjsewell]

just in case someone else does it first

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/jupyter-book) and found it was in an excellent condition.

[bollwyvl]

It might also be nice to add outputs to support capturing the pdf output target... e.g. jupyter-book-pdf...

[chrisjsewell]

Can you explain this further thanks, I'm not quite sure what added benefit this brings?

[bollwyvl]

Conda doesn't support the [extras] section that pip does (which ends up being kinda weird, anyway, as it isn't respected by pip check, nor will extra pins be taking into account for later solves, but anyway...) so if folk want the pdf output option (which is great) they'll get the nastygram about installing pypetteer, etc. If a lot of people want pdf, this might be a sensible default, but that toolchain has problems of its own (downloading chromium at runtime), so an easy out would be useful

concretely, there could be a jupyter-book-core, which contains all the hard dependencies, and a jupyter-book-webpdf which depends on -core and pypetteer (with the appropriate version), while jupyter-book could depend on -core and -pdf.

outputs:
- name: jupyter-book
  requirements:
    run: 
      - jupyter-book-core ={{ version }}
      - jupyter-book-webpdf ={{ version }}
- name: jupyter-book-core
  requirements:
    run:  # all the reqs
 - name: jupyter-book-webpdf
  requirements:
    run: 
      - jupyter-book-core ={{ version }}
      - pypeteer

Of course, this would be better if there was an nbformat-webpdf we could depend on... but the run_constrained will at least take care of the version, so the pin wouldn't have to track here.

[chrisjsewell]

Oh yeh ta I forgot that was under outputs. I do something similar on another project https://github.com/conda-forge/aiida-core-feedstock/blob/master/recipe/meta.yaml

[bollwyvl]

yeah, outputs are kinda gross, but the best we have today...

[henryiii]

This looks good to me, and would add some nice flexibility.

[bollwyvl]

Might this need a rerender now that sphinx-panels is good to go? Not my PR, so might not work, but:

@conda-forge-admin please rerender

[bollwyvl]

ha, wait this is staged-recipes... perhaps

@conda-forge-admin please restart ci

failing that, just closing and re-opening it would probably work

[bollwyvl]

Thanks @conda-forge-admin 😆

[bollwyvl]

Looks like there's just a touch of -_ impedance...

conda.exceptions.ResolvePackageNotFound: 
  - sphinx_book_theme[version='>=0.0.34'] # sphinx-book-theme-feedstock
  - myst-nb[version='>=0.8.0,<0.9']       # myst-nb-feedstock up, but not this version
  - sphinx_togglebutton                   # sphinx-togglebutton-feedstock

[bollwyvl]

Looks like upstream add yet another dependency on the 0.9 line...

https://github.com/russellballestrini/nested-lookup

It's public domain, but doesn't have any description of its intent in-repo:

russellballestrini/nested-lookup#41

[choldgraf]

Oof, I didn’t realize that it didn’t have a license. If that is a big dealbreaker we can try to disentangle the dependency from it, it’s not a huge dep in terms of our use, but it’d take a bit of work

[chrisjsewell]

Looks like upstream add yet another dependenc

out of interest, what top-level dependency does this dependency come from?

[choldgraf]

It’s added in Jupyter book, I believe by this pr jupyter-book/jupyter-book#1123

not sure it was the right decision to include it as a dep 🤔

[bollwyvl]

If that is a big dealbreaker

Well, kinda... there are well-known examples of stuff that is public domain (e.g. sqlite), but usually there's something that clarifies what that means to the authors.

it’s not a huge dep in terms of our use, but it’d take a bit of work

Yeah, turns out it's kind of an odd duck to solve... something like jmespath doesn't support recursive queries like that, unlike xpath, some sql dialects, etc. maybe something with functools?

[ericdill]

noting here that @bollwyvl (🎉 )has opened an issue on the nested-lookup repo asking them to add a license but there's been no response thus far

[choldgraf]

Note - it sounds like the author does not want to license it as anything other than "public domain", so I think these are our options if they won't add a license:

1. Remove the dependency and just write the code ourselves
2. Create a soft fork of that repository that simply pulls in the latest changes from main, and adds an MIT license. Depend on that.
3. Decide that it's NBD that this is "public domain" and just add it into the recipe 🤷

[bollwyvl]

1. it's likely a one-to-five liner of functools, depending on what's being used
2. this won't change the underlying issue, as it would become a derived work which would be encumbered with the same uncertainty, but worse
3. unfortunately, now that we've done the due diligence, this wouldn't satisfy someone who wanted to use jupyter-book as part of a product.

if jupyter-book doesn't want to be part of products, that's fine, but then the interest from folks who do like to ship products (such as myself, or day job alter ego) just dropped down to close to zero in helping maintain/improve it.

Also: a number of the dependencies are lagging behind significantly on the conda-forge feedstocks. Once this feedstock is in, it would be possible to have a number of maintainers in that group (e.g. conda-forge/jupyter-book), and have the group itself become the maintainer of all of these dependencies, reducing bottlenecks.

It's also possible to add automerge, which will... automatically merge PRs created by the bot for new versions. I am very leery of this unless the full test suite of such a feedstock is being run, with a coverage threshold, pip check, etc. etc, but this also means having all of the test dependencies available on conda-forge... usually not a bad thing, either, as it will reveal issues like this. Some packages with automerge like setuptools have broken half the condaverse for a couple hours/days by either merging straight up bad versions, or missing seemingly-minor dependency changes.

[chrisjsewell]

Remove the dependency and just write the code ourselves

I'd say do this; there is a very small number of code lines anyway, so not worth dealing with an author that does not want to add a simple license (also no other issues have been addressed recently.)

[choldgraf]

I should have noted my preferences too - I think i prefer option 1, I suspect the amount of time we've all spent on figuring out this license issue is more than the time itd take to just write a recursive lookup function with just the functionality we need.

Thanks for clarifying the other downsides of points 2 and 3, that sounds reasonable to me!

And for the group of maintainers ,that sounds great to me.

[henryiii]

setuptools pushed the bad versions to PyPI too, I don't think this is necessarily a problem with automerge :)

[chrisjsewell]

We're not using automerge

[bollwyvl]

setuptools pushed the bad versions to PyPI too, I don't think this is necessarily a problem with automerge :)

Right, but my point is: running more robust tests as a downstream distribution occasionally discovers issues that upstreams won't find, as their test setups either pin down to point releases, don't run on exotic architectures, etc. This helps lead to upstream PRs, which sometimes starts changing peoples' minds about conda-forge as somehow "coming for your ecosystem." We're just a bunch of nerds that want all our software to work well together for SCIENCE, with as little manual intervention (or redistribution woes) as possible.

[bollwyvl]

this is what i came up with. It's very well insufficient, but a starting point:

import typing

def find_values(key: str, obj: typing.Any) -> typing.Iterator[typing.Any]:
    """find all  of `key` in a nested JSON-compatible object
    """
    if obj is None or isinstance(obj, (str, int, bool, float)):
        return
    elif isinstance(obj, dict):
        if key in obj:
            yield obj[key]
        for child in obj.values():
            yield from find_values(key, child)
    elif isinstance(obj, list):
        for child in obj:
            yield from find_values(key, child)

If we want to get pedantic: i make the above available under the terms of the BSD-3-Clause license, blah, blah.

[chrisjsewell]

Yeh looks good, just make a PR to jupyter-book 👍

[moorepants]

I came across this when trying a conda install jupyter-book today. I think you should simply add any license you want in the nested-lookup-feedstock. The author states in numerous ways that he giving his copyright of the source code to the public domain. If conda-forge needs a license, anything can be applied to something in the public domain. Just drop a license file in the feedstock. There are numerous other license compatibly issues present in conda-forge that no one really worries at this level of detail about. This issues seems much more benign than some other GPL related issues.

[moorepants]

Here is an attempt: #13918 Maybe it will get merged as is.

[moorepants]

The nested-lookup feedstock is now available under an MIT license.

[chrisjsewell]

Thanks

[choldgraf]

I'm happy with whatever y'all think is a reasonable-enough practice, I don't know much about licensing :-)

[moorepants]

I've opened this PR: #13989 which builds on this one and builds jupyter-book 0.8.2. FYI

[moorepants]

We merged in a feedstock for 0.8.2. Once it is live, feel free to iterate on that to add newer versions or the outputs as @bollwyvl suggested. But hopefully that version can at least get folks started using this with conda. Thanks!

[bollwyvl]

Huzzah! The other thing mentioned was PRing @conda-forge/jupyter-book as a maintainer on all the otherwise-novel-to-cf deps, which should help keep the machine running.

[choldgraf]

hooray!

@bollwyvl - is that actionable on our end? I don't understand most of those words 😅