How to nest inline parsers?

[cideM]

I've stared at parser/link.go for hours now but I still can't figure out how it actually does nested inline parsing. There appears to be no general stack of nodes and also no way to change the parent node into which the next, parsed node will be appended. Based on some debugging sessions, my best guess so far is that when the link parser sees the opening [ it leaves a "marker" node in the AST by returning a link label state.

When it later on comes across the closing ], it will use that sneakily inserted marker node to loop over the siblings and move them into the actual link node and remove them from their parent. Nested emphasis and italic seems to work in an entirely different manner, I guess ProcessDelimiters somehow does this? I don't know.

It would be great to have an example of an inline parser that allows arbitrarily nesting stuff inside of it. Both nesting itself, and emphasis.

[%**foo[%bar%]**%] assuming [% %] is a new inline parser

[yuin]

goldmark is a CommonMark parser. I recommend reading the CommonMark spec.

OK. You have read the emphasis section.
Now you realize the emphasis in CommonMark has 17 rules as complicated as hell.

OK. you have read the parsing strategy section.
Now goldmark inline parsing source codes should be clear for you.

I must say "CommonMark is complicated as hell" (as I wrote in README). You seem underestimate complexity of the CommonMark.
This margin is too narrow to contain to explain CommonMark link and emphasis :)

[cideM]

Sorry I should have been more clear. This was more a question of how nesting inline parsers is done on the goldmark side of things, not really about the spec.

In the meantime I got it to work so here's what I've gathered.

In parser/link.go it seems that when the opening [ of a link is encountered, the link parser is triggered. It creates a linkLabelState struct which actually is a valid ast.Node, since it embeds one of the base structs. That's pretty unexpected since this struct is called state and isn't even exported, so it's a bit of a fake node. This node is also added to the parser context, so we can grab it later.

When we then come across a closing ']', we grab the original linkLabelState and iterate through its next siblings. Those siblings are added to the actual Link struct. Finally, the fake node linkLabelState is removed and the actual Link struct is returned and inserted into the AST. This is pretty unexpected and I guess it's a result of the way goldmark is set up internally, more than the CommonMark spec.

Essentially, parsing [**foo**](www.google.de) will first have a flat list of siblings so Document -> Paragraph -> linkLabelState, Emphasis, Text and then you hit the closing ] and it runs ProcessDelimiters and then it'll also do this little sibling iteration dance which turns it into Document -> Paragraph -> Link -> Emphasis -> Text (nested).

See https://github.com/yuin/goldmark/blob/v1.5.4/parser/link.go#L239-L244