[Fleet] caching package info with node-fetch-cache

[juliaElastic]

Summary

Related to #158361

Closes #160130

Alternative solution to building a packageInfoCache manually to use node-fetch-cache library that caches epr fetch package requests for 10 minutes.
Comparing used RSS with #161158

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[apmmachine]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 86ff623
• Interpreting CI Failures
• Cloud Deployment

Failed CI Steps

• Build API Docs
• Jest Tests #2
• Jest Tests #4
• Jest Tests #7
• Jest Tests #13
• Jest Tests #17
• Jest Integration Tests #1
• Jest Integration Tests #1
• Jest Integration Tests #1
• Jest Integration Tests #1
• Jest Integration Tests #2
• Jest Integration Tests #2
• Jest Integration Tests #2
• Jest Integration Tests #2
• Jest Integration Tests #4
• Jest Integration Tests #4
• Jest Integration Tests #4
• Jest Integration Tests #4
• Jest Integration Tests #5
• FTR Configs #24
• Check Types

Test Failures

• [job] [logs] FTR Configs #24 / discover/group3 discover request counts data view mode should send 2 requests (documents + chart) when adding a filter
• [job] [logs] Jest Integration Tests #5 / incompatible_cluster_routing_allocation retries the INIT action with a descriptive message when cluster settings are incompatible
• [job] [logs] Jest Integration Tests #5 / migrating from 7.3.0-xpack which used v1 migrations copies all the document of the previous index to the new one
• [job] [logs] Jest Integration Tests #5 / migrating from 7.3.0-xpack which used v1 migrations creates the new index and the correct aliases
• [job] [logs] Jest Integration Tests #5 / migrating from 7.3.0-xpack which used v1 migrations migrates the documents to the highest version
• [job] [logs] Jest Integration Tests #5 / migration v2 - read batch size does not reduce the read batchSize in half if no batches exceeded maxReadBatchSizeBytes
• [job] [logs] Jest Integration Tests #5 / migration v2 - read batch size reduces the read batchSize in half if a batch exceeds maxReadBatchSizeBytes
• [job] [logs] Jest Integration Tests #5 / SO default search fields make sure management types have the correct mappings for default search fields
• [job] [logs] Jest Integration Tests #5 / SO type registrations does not remove types from registrations without updating excludeOnUpgradeQuery
• [job] [logs] Jest Integration Tests #5 / split .kibana index into multiple system indices when migrating from a legacy version performs v1 migration and then relocates saved objects into different indices, depending on their types
• [job] [logs] Jest Integration Tests #5 / split .kibana index into multiple system indices when multiple Kibana migrators run in parallel correctly migrates 7.7.2_xpack_100k_obj.zip archive
• [job] [logs] Jest Integration Tests #5 / when splitting .kibana into multiple indices and one clone fails after resolving the problem and retrying the migration completes successfully

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @juliaElastic

[juliaElastic]

Did a test on cloud pr deployment by updating schema_version of 20 policies and triggering setup.
The RSS reaches 1gb, looks better than the other approach: #161158 (comment)
This is with the default batch size of 100.

[joshdover]

I think this is a good optimization regardless, but I still think we need to understand more about why downloading and unzipping 20 fairly small zips takes so much memory.