Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump pandoc upper bound to 2.5 #933

Closed
wants to merge 1 commit into from
Closed

Conversation

tk3369
Copy link

@tk3369 tk3369 commented Dec 29, 2018

@mgeier
Copy link
Contributor

mgeier commented Dec 29, 2018

Related: #928

@choldgraf
Copy link
Contributor

the error in travis on 3.4 ends with this:

  s.bind("tcp://%s:%i" % (self.ip, port))
  File "zmq/backend/cython/socket.pyx", line 547, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Address already in use
==================== 1 failed, 253 passed in 150.85 seconds ====================
The command "py.test --cov nbconvert -v --pyargs nbconvert" exited with 1.

perhaps that's unrelated to the pandoc bump? In which case somebody could try restarting to see if that fixes it. I think a good way to answer the question "should newer versions of pandoc be supported" is to see if the tests all pass w/ newer pandoc

@choldgraf
Copy link
Contributor

I know that @ivanov has been looking into pandoc/notebook/etc stuff, maybe he has experience w/ pandoc 2.X and knows if there would be compatibility issues

@tk3369
Copy link
Author

tk3369 commented Dec 29, 2018

The ZMQ error feels like a system issue rather than a code problem... Can anyone trigger the build again?

The nightly also failed but it seems to have a more tangible reason:

/home/travis/virtualenv/python3.7-dev/lib/python3.7/site-packages/nbconvert/preprocessors/tests/test_execute.py:83: in assert_notebooks_equal
    self.assertEqual(normalized_expected_outputs, normalized_actual_outputs)
E   AssertionError: Lists differ: [{'data': {'text/plain': "'Code 1'"}, 'exe[46 chars]lt'}] != []
E   
E   First list contains 1 additional elements.
E   First extra element 0:
E   {'data': {'text/plain': "'Code 1'"}, 'execution_count': 1, 'output_type': 'execute_result'}
E   
E   + []
E   - [{'data': {'text/plain': "'Code 1'"},
E   -   'execution_count': 1,
E   -   'output_type': 'execute_result'}]

@takluyver
Copy link
Member

takluyver commented Jan 8, 2019

FWIW, you can usually rerun tests by closing and reopening the PR. At the moment, though, there's an unrelated problem with incompatible testing packages, so it will fail if we re-run it now. I'm trying to fix that in #941.

The most fragile bit for pandoc versions is this filter which converts links within a notebook to Latex references. It's used here:

((( cell.source | citation2latex | strip_files_prefix | convert_pandoc('markdown+tex_math_double_backslash', 'json',extra_args=[]) | resolve_references | convert_pandoc('json','latex', extra_args=["--top-level-division=chapter"]) )))

That converts the markdown to pandoc's JSON document format, applies the link converter to it, and converts the updated JSON to Latex. So it depends on the structure of pandoc's document format.

So before we expand the constraint, please double check that the latest version of Latex pandoc you want to be supported still performs that conversion correctly.

@chrisjsewell
Copy link

chrisjsewell commented Mar 2, 2019

Yes restricting the pandoc version is particularly a pain if you are using conda, which currently prohibits installing the latest versions of nbconvert and pandoc together.

Personally I would suggest you swap out pandocfilters for panflute, which is more 'pythonic' and will deal with changes in the pandoc AST.

Somemthing along the lines of:

import io
import panflute as pf

def resolve_references(source):
   input_stream = io.StringIO(source)
   with io.StringIO() as output_stream:
       pf.run_filter(resolve_one_reference, 
                          input_stream=input_stream, 
                          output_stream=outut_stream)
       return output_stream..getvalue()

def resolve_one_reference(link, doc):
    # type: (Link, Doc) -> Element
    """extract links that point to internal items, e.g. [text](#label)"""
    if not isinstance(link, pf.Link):
        return None
    match = re.match(r'#(.+)$', link.url)
    if not match:
        return None
    # pandoc automatically makes labels for headings.
    label = m.group(1).lower()
    label = re.sub(r'[^\w-]+', '', label) # Strip HTML entities
    return pf.RawInline(r'Section \ref{%s}' % label, format="tex")

You could also basically replace nbconvert/nbconvert/utils/pandoc.py with panflute functions

@choldgraf
Copy link
Contributor

Just a note that another reason to bump pandoc at some point might be to support 2.6, which supports ipynb files and might make it possible to remove some code in nbconvert after a few depreciation cycles

@minrk minrk mentioned this pull request Mar 6, 2019
@MSeal MSeal added this to the no action milestone Sep 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants