BAU: Check internal and external links

[mrwilson]

This PR introduces checking for internal links (within the GDS Way's own content) and external links (content on other sites).

We don't want to have broken links in the GDS Way as this is a bad and confusing user experience. This was highlighted in issue #700.

This PR builds on existing work in alphagov/tech-docs-template#187 and would aim to be folded into the tech docs template eventually.

This PR is also a place for discussion around:

• updated content and new external links prompted by the link checker
• how to handle rate-limiting by GitHub when checking links to repos, of which there are many in the project

[36degrees]

Thanks for picking this up, @mrwilson! A few things I'm not 100% sure about – might just be because I'm missing context.

My Ruby is rusty at best, so you might want a second set of eyes on some of this, depending on your confidence in the changes.

[domoscargin]

Edit: Seems like this is probably more to ignoring Github links 'cuz of the rate-limiting but I'll keep this comment here in case it's relevant once we solve that

How much do we care about hashes if the target page at least loads?

For instance, we're not catching the fact that this link:

gds-way/source/manuals/programming-languages/html.html.md.erb

Line 15 in db16b14

[stricter standards of browser support](https://github.com/alphagov/govuk-frontend#browser-support).

has updated to https://github.com/alphagov/govuk-frontend#browser-and-assistive-technology-support

Looks like we could enable check_external_hash (and check_internal_hash?) to catch these.

[mrwilson]

@domoscargin That's a very good shout.

According to the HTMLProofer docs, the default value for both of those flags is true, however I will add them to the configuration for the sake of explicitness and not relying on default behaviour that may change.

EDIT: Wow, it was definitely not on by default. Fix commit incoming.

EDIT 2: Internal hash checking is fine, but not adding external hash checking as that would mean our deployments could be broken by an external page changing its structure. "Does this page exist?" is a good enough heuristic for external links and will catch the majority of failures

[richardTowers]

Looks good to me!

Only thing that I think will be slightly annoying is the inevitable "I just want to fix this typo, but to get the build working I need to fix these unrelated broken links". But that's probably better than having broken links, and I don't think an alternative cadence (e.g. daily check with a slack message, doesn't fail the build) would work given the limited ownership of GDS way.