Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecation of returning all assets via the API #14140

Closed
MattIPv4 opened this issue Jun 15, 2022 · 9 comments
Closed

Deprecation of returning all assets via the API #14140

MattIPv4 opened this issue Jun 15, 2022 · 9 comments

Comments

@MattIPv4
Copy link
Member

MattIPv4 commented Jun 15, 2022

Hey folks!

Proposal

Moving forward, the cdnjs API endpoint /libraries/:library will no longer return assets for all versions of the library in the assets array. Instead, this array will only contain a single item with the assets for the latest version of the library.

The cdnjs API provides, and will continue to provide, the /libraries/:library/:version endpoint that will allow you to access assets and SRI hashes for a specific version of any library, and the versions property of the /libraries/:library endpoint response will continue to list all versions.

We hope this change will be non-breaking for most folks, as the type signature of the response from the endpoint will not be changing, only the number of items in the assets array being reduced to one, the latest version.

Justification

The reason behind us making this change and deprecating the full array in this response is simple -- it doesn't scale. As we look to move the cdnjs API to be a Cloudflare Worker instead of an Express server, we're more aware than ever of the resource usage behind each request hitting the cdnjs API and this array was proving to be a hard blocker for migrating to Workers.

For some libraries, like mathjax, this array contained 45 versions, with some containing over 25 thousand files (repeated essentially three times as we return both the files and rawFiles properties, plus SRI hashes), resulting in the response being upwards of 100MB of raw JSON data. This resulted in resource consumption that proved problematic for our Express server, and a complete blocker for a Worker.

Our work to move the cdnjs API to Cloudflare Workers is part of ongoing high-priority work to improve the stability of the cdnjs API, which has recently had issues resulting in incidents declared and bad responses returned to consumers.

Timeline

Date Delta Action Status
2022/06/15 - Proposal posted ✅ Posted
2022/06/18 +0.5 weeks Updated Worker with single-version assets array will be pushed live to a small % of requests. ✅ 10% rollout
2022/06/25 +1.5 weeks If no issues arise, Worker will be ramped up to 100% of traffic with the single-version assets array over a period of a week. ✅ 100% rollout completed (delayed, 2022/07/18 [+4.5 weeks])

Due to this deprecation not resulting in a type signature change to the endpoint, our belief that removing old versions from the response is unlikely to impact many (if any) API consumers, and it resolving an active technical blocker for improving API stability, we've introduced a shortened timeline for this deprecation.

@MattIPv4
Copy link
Member Author

Work has been prepared to enact this deprecation, in the new API worker, and in the docs:

@MattIPv4
Copy link
Member Author

API docs change is live in production: https://cdnjs.com/api#library

API change is live for 10% of production traffic.

@MattIPv4
Copy link
Member Author

MattIPv4 commented Jul 5, 2022

The rollout of this was paused at 10% due to the dependency discovered within LibMan on this field. Once they've shipped updates, we plan to continue this rollout. #14141 & aspnet/LibraryManager#685 can be used to track progress on LibMan.

@MattIPv4
Copy link
Member Author

This is now live for 25% of traffic: https://github.com/cdnjs/api-server/runs/7395641390

@MattIPv4
Copy link
Member Author

This is now live for 50% of traffic: https://github.com/cdnjs/api-server/runs/7416499906

@MattIPv4
Copy link
Member Author

This is now live for 75% of traffic: https://github.com/cdnjs/api-server/runs/7435971283

@MattIPv4
Copy link
Member Author

This is now live for 100% of traffic: https://github.com/cdnjs/api-server/runs/7456158760

@MattIPv4
Copy link
Member Author

With this API change now completed and live, as well as the docs being updated on the site, closing this issue out. Will leave it pinned until anything supersedes it.

axelhahn added a commit to axelhahn/cdnorlocal that referenced this issue Apr 28, 2023
v1.0.9 : update for changes in cdnjs api
see cdnjs/cdnjs#14140
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants
@MattIPv4 and others