upgrade to pulldown-cmark 0.4.1

[euclio]

No description provided.

[euclio]

Hm, looks like pulldown-cmark 0.4.1 doesn't preserve leading and trailing whitespace in table cells anymore. Not sure if that's a regression or a fix.

[carols10cents]

I tried this upgrade myself, and ran into the same issue :)

It looks like the whitespace stripping started in this PR, according to this comment.

I did some investigating in this commit and it seems like the spaces in the markdown that pulldown-cmark-to-cmark generates don't affect the parsing that pulldown-cmark does at all. That is, the test in that commit does the following:

• Takes a markdown table
• Parses it into pulldown-cmark Events
• Generates markdown with pulldown-cmark-to-cmark
• Parses the markdown generated by pulldown-cmark-to-cmark into pulldown-cmark Events again
• Tests that the two lists of Events are identical

The generated markdown tables aren't as pretty as the original, but given that the purpose of pulldown-cmark-to-cmark is to make markdown for further programs to consume rather than people, this might be fine? I'm interested to see what Byron thinks :)

[ehuss]

It looks like 0.5 was released today. The main change is that inline code blocks are now Event::Code(str) instead of Event::Start…End. I looked at updating this, but there seems to be some bugs in how tables are handled, so I wasn't sure if that should be fixed. The lines here assume the table header contents only appear as a single event, but there can be multiple events. I think it should accumulate the length of all content between Start(TableCell) and End(TableCell) when inside Start(TableHead), but that didn't seem terribly easy.

[euclio]

Found another bug while trying to update this PR to 0.5.1: pulldown-cmark/pulldown-cmark#327

@ehuss Are the bugs in the table handling you found covered by any existing tests? I didn't see any other failures.

[ehuss]

No, I don't think any of the existing tests caught the issues I saw. Here's an example:

foo f x	bar
1	2

In markdown:

`foo` f `x` | bar
------------|-----
1           | 2

The way these lines work probably needs to change. There can be multiple events within a single cell, so it should probably update the last table_alignments value for each event, then flush it when the cell closes. Unfortunately there are a lot of potential events (Text, Code, InlineHtml, etc.), and each one should be handled.