Docs: Use a link checker to prevent publishing broken links

[philknows]

Problem description

As seen in #6550 with PRs #6879 and #6864, there may be content that hyperlinks to things that no longer exist or was moved due to navigation changes. We want to avoid publishing docs with broken links and as we grow the developer content in our docs, it will become harder to constantly check and maintain working hyperlinks.

Solution description

The solution is to integrate a link checker in our CI using something like https://github.com/UmbrellaDocs/linkspector as part of our check docs workflow.

Additional context

No response

[nflaig]

It looks like docusaurus is already able to detect some broken links but requires to run a production build

docusaurus build

Running this on unstable branch

[WARNING] Docusaurus found broken links!

Please check the pages of your site in the list below, and make sure you don't reference any path that does not exist.
Note: it's possible to ignore broken links with the 'onBrokenLinks' Docusaurus configuration, and let the build pass.

Exhaustive list of all broken links found:
- Broken link on source page path = /lodestar/contribution/getting-started:
   -> linking to .devcontainer/devcontainer.json (resolved as: /lodestar/contribution/.devcontainer/devcontainer.json)
- Broken link on source page path = /lodestar/libraries/lightclient-prover/prover:
   -> linking to ./src/provider_types/ (resolved as: /lodestar/libraries/lightclient-prover/src/provider_types/)

@jeluard do you know why we only log a warning if links are broken? the default seems to be that docusaurus throws an error if links are broken which seems better to me

lodestar/docs/docusaurus.config.ts

Lines 23 to 24 in 79a008f

	onBrokenLinks: "warn",
	onBrokenMarkdownLinks: "warn",

See related docs

[nflaig]

The solution is to integrate a link checker in our CI using something like https://github.com/UmbrellaDocs/linkspector as part of our check docs workflow.

irrespective of enabling proper link checking in docusaurus, we will still need something like this for external links, and references from other files like README.md