Improve build time of documentation

[jennifer-shehane]

Many of our contributors are not happy with the build time of the documentation. We'd like it to build faster 🏃

Cause is inception of dependencies with checking internal links.

[lilaconlee]

Dug through docs build stuff, and it seems like the bulk of the slowness isn't related to validation but just to building the URL tags. Hexo even recommends not using tags if your build is slow.

Seems like the biggest gain would be to use markdown URLs and validate those on PR builds, but not sure it's worth it.

[jennifer-shehane]

If we decide to no longer validation for outside links, we could at least remove those url tags.

One of the benefits of the url tags also is just having to reference the defined name of the doc, instead of relative/absolute links. So linking to the cy.route() doc, I just have to write {% url "link" route %} Not sure if that would be possible with the markdown links.

[lilaconlee]

Yeah I'm really not sure why the Hexo tags are so slow compared to other templating options, I'll look around for faster alternative that still builds internal URLs.

[brian-mann]

Thats the magic stuff we had to edit hardcore. I even did PR's into the templating language to get it to do what we want. It's supposed to be async, but then hexo had problems with that so we ended up having to use synchronous rendering.

Per what we discussed on Monday - this is what I was talking about, the external URL validation is a red herring and it's not what is expensive. The page generation is what is expensive and it's because when you have 'X' page that's being built that references 'Y' page that's being built, it has to build all of 'Y' page to ensure that the link not only references an existing document, but that if it has a hash then it references a hashable content.

Moving to standard markdown tags would be an absolute nightmare, because we change the structure and organization of the pages all the time, and this system keeps you completely insulated from ever passing around document paths. However, I do see some ways this could be accomplished. We could split out the validation concerns away from the content generation concerns. Meaning - rather than building pages at runtime (in order to validate their contents / hash linking) we could just build the URL and then move the validation to another step, the same way we plan on doing the external URL validation.

That would likely give us all the benefit we get out of it now, without the slowdowns. URL and hash linking validation could then move to a separate build process or npm command.

[lilaconlee]

After pulling out validation completely and just returning dummy URLs from the URL tag, the build process still took essentially the same amount of time. I can give it another look now that I understand what y'all have worked on previously, but I don't think the build time would be very different than what it currently is after caching.

[brian-mann]

Right - that's likely because its all synchronous, which is forced due to what's happening inside of the URL helpers. If you switch the markdown rendering functions to use the async versions, try re-running it and see what kind of difference it makes.

[lilaconlee]

Oh totally did not connect that the synchronous rendering was only needed for the validation, that makes way more sense.

[lilaconlee]

It was the changelog's fault the whole time. We ran the hexo generate command with the --debug flag to determine where the bottleneck was, and it was processing the changelog file.

Going to move the changelog into several individual files and build with partials/tag helpers.

[bahmutov]

Changelog?! That file kept growing

…

Sent from my iPhone

On Sep 25, 2018, at 13:50, Lila Conlee ***@***.***> wrote: It was the changelog's fault the whole time. We ran the hexo generate command with the --debug flag to determine where the bottleneck was, and it was processing the changelog file. Going to move the changelog into several individual files and build with partials/tag helpers. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[jennifer-shehane]

omg....the changelog!!!!

[lilaconlee]

Addressed in #968