[CI] Try to prevent npm/yarn problems from causing an agent to fail all future jobs

[brianseeders]

Most steps in our Buildkite pipelines use one-time use agents. But, the really short steps that every build runs (pipeline upload, pre-build, post-build) use long-running agents. There are a few things related to node modules that can cause agents to start failing all jobs they run.

Here's an example of what happens:

1. A job is run for a PR that needs a different version of yarn than what is currently installed on the agent
2. The job is cancelled after someone pushes a new commit to the PR, and it happens to be in the middle of npm install -g yarn
3. The yarn global node_modules directory is in a bad state
4. The next job and all future ones on this agent will fail when trying to install yarn

The long-term goal would be to use something like containers to provide more isolation between jobs while keeping fast start times, but this should help in the short-term.

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 5799527
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💔 Build #15941 failed 416241a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.15
✅	7.17
✅	7.16

The backport PRs will be merged automatically after passing CI.