docs: (de-scoping) first pass at updating the JS Core Roadmap #22
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,73 +1,72 @@ | ||
| # JS Core Dev Team 2019 Roadmap | ||
|
|
||
| > Oversees the development of the JS implementation of the IPFS protocol and related projects. | ||
|
|
||
| **Responsibilities include:** | ||
| - Implement and maintain the IPFS protocol and tools | ||
| - Enable IPFS usage in Node.js, browsers and other JavaScript runtimes | ||
| - Maintain interoperability with other IPFS implementations | ||
| - Incubate and grow projects that will encourage IPFS adoption | ||
|
|
||
| **Current Projects** | ||
| - [JS IPFS](https://github.com/ipfs/js-ipfs) | ||
| - [JS IPFS HTTP Client](https://github.com/ipfs/js-ipfs-http-client) | ||
| - [Go & JS IPFS Interop Test Suite](http://github.com/ipfs/interop) | ||
| - [NPM on IPFS](https://github.com/ipfs-shipyard/npm-on-ipfs) | ||
| - [IPFS Service Worker Gateway](https://github.com/ipfs-shipyard/service-worker-gateway) | ||
| - [IPFS Geo IP](https://github.com/ipfs/ipfs-geoip) | ||
| - [IPFS Performance Profiling](http://github.com/ipfs/ipfs-performance-profiling) | ||
| - https://benchmark-js.ipfs.io | ||
| - https://js.ipfs.io | ||
| - https://benchmarks.ipfs.team | ||
|
|
||
|
|
||
| ## 🚀 Major 2019 Goals to achieve | ||
|
|
||
| The major goals below and following milestones have been labeled with the emoji for the goal in the IPFS Project Roadmap that they contribute towards: | ||
|
|
||
| 📦 = Package Managers 🤝 = Partner Use Cases | ||
|
|
||
| - JS IPFS is ready for use in production. It is stable, performant, and interoperable with other IPFS implementations 📦 🤝 | ||
| - JS IPFS can ingest and move data with similar speeds to rsync or wget 📦 🤝 | ||
| - Data on JS IPFS is searchable both online and offline 📦 🤝 | ||
| - Developer experience is a joy 🤝 | ||
|
|
||
| ## 💎 Milestones | ||
|
|
||
| ### `M (P0)`: Production ready 📦 🤝 | ||
|
|
||
| JS IPFS has for a long time now been under heavy development with a focus on implementing features. We’re now at a stage where it has a good level of feature parity to Go IPFS. Implementing the last key missing features and focused attention on performance, stability and security issues will allow JS IPFS to be a viable alternative to Go IPFS. | ||
|
|
||
| - A JS IPFS daemon runs as part of the IPFS gateway cluster for at least 1 month without crashing | ||
| - Performance metrics for a JS IPFS node are measurable: | ||
| - Tools can process and graph performance data | ||
| - Data can be queried to track down bugs and performance bottlenecks | ||
| - Benchmarks exist to assert that: | ||
| - Memory footprint remains stable and should not consume more than 20% more memory than a Go IPFS node | ||
| - Network throughput is within an acceptable magnitude of Go IPFS | ||
| - Data can be fetched with comparable speeds to tools like bittorrent, wget or rsync when 5 peers are connected who already have the data | ||
| - Data can be exchanged with another IPFS node within 2x the speed a Go IPFS node would exchange it | ||
| - An interoperable DHT exists and scales to hundreds of thousands of nodes | ||
| - JS IPFS nodes can discover one another for specific topics via the rendezvous protocol | ||
| - \*-star servers are no longer in use | ||
| - JS IPFS nodes use, and are compliant with the MDNS spec | ||
| - A gossipsub implementation exists | ||
| - An IPFS log API is implemented and tests ensure log messages are part of the API | ||
| - A security audit is conducted on the code base and all issues resolved | ||
|
|
||
| ### `M (P1)`: Optimized for browser environments 📦 🤝 | ||
|
There was a problem hiding this comment. This Milestone will enable the Web Browsers Working Group to ship an amazing DWebCDN using JS IPFS and given that the Consider dropping priority for Electron, Mobile and/or IoT |
||
|
|
||
| One of the biggest challenges that faces JS IPFS is that it will be used in many different environments where the tools and resources available to it differ significantly. JS IPFS can be run in a browser (Chrome, Firefox etc.), in a web extension, in Node.js on a laptop or on a server, in Electron, on mobile or even on IoT devices. Not only does the runtime differ - Node.js, Blink, SpiderMonkey etc. but the APIs available (no filesystem or low level network in a browser) as well as the hardware resources available e.g. mobile or IoT. | ||
|
|
||
| - Default performance profiles exist and can be enabled to adapt IPFS behaviour appropriately to its environment. In addition to the profiles [described here](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#profiles), there will be a browser specific performance profile. | ||
| - The JS IPFS build can be customised to exclude functionality or include different modules that are more appropriate for a particular runtime environment | ||
| - At least 2 recommended builds for common environments will be provided in addition to the full build | ||
| - Developers can tailor an IPFS build to their specific needs | ||
| - Full build bundle size is monitored and does not exceed 2MB (minified) | ||
| - Examples for bundling JS IPFS with common tools such as webpack/parcel exist | ||
|
|
||
|
|
||
| ### `M (P1)`: Installable npm modules via JS IPFS 📦 | ||
|
|
||
| The npm registry is one of the biggest data sets JS developers interact with daily. There’s a myriad of benefits to it being accessible on the IPFS network. We’ll enable this at the same time as proving out the ability for JS IPFS to deal with large data sets. | ||
|
|
||
|
|
@@ -77,16 +76,16 @@ The npm registry is one of the biggest data sets JS developers interact with dai | |
| - Installing npm modules over LAN never fails (provided the module is available on another IPFS node connected to the LAN) | ||
| - Installing any module is faster than using npm when 5 peers on the local network already have the package | ||
| Node.js binaries are distributed via IPFS (making the whole Node.js/NPM installation process happen over IPFS) | ||
|
|
||
| ### `M (P2)`: Discoverable npm modules via JS IPFS 📦 | ||
|
|
||
| One of the big features that IPFS lacks right now is content discoverability. A mechanism of searching for content on the IPFS network would greatly improve the situation. Considering our milestone of installing npm modules via JS IPFS, it makes sense to augment it with a search component to make it more attractive and useful as well as allowing us to achieve our major goal of adding search to JS IPFS. | ||
|
|
||
| - Users can search for modules in the npm registry via IPFS | ||
| - Users can learn about the latest registry index (module updates) via IPFS | ||
| - Unixfs v2 is finalized and is the default format for new content added to IPFS | ||
|
|
||
| ### `M (P2)`: Improved documentation, specs and examples 🤝 | ||
|
|
||
| Improving the docs, specs and having great tutorials and examples goes a long way to realising the distributed web for the simple reason that if people can’t drop by and pick up IPFS easily they’re less likely to use it. This milestone addresses that problem and also helps people already invested in IPFS to continue to build on top of it. | ||
|
|
||
|
|
@@ -97,34 +96,92 @@ Improving the docs, specs and having great tutorials and examples goes a long wa | |
| - Return values are documented, including object properties and types | ||
| - Onboarding documentation exists to explain domain concepts and provide tutorials/examples for achieving common tasks | ||
|
|
||
| ### `M (P0)`: Revamped APIs 📦 🤝 | ||
|
|
||
| Revamping the APIs will make them easier to use and simpler to understand. Using modern syntax will make JS IPFS appealing to a wider segment of the JS community, aiding adoption and contributions. It also gives us scope to make performance optimisations and stability/debuggability improvements. | ||
|
|
||
| - Async/await is used throughout the codebase and callback APIs are dropped | ||
| - ArrayBuffer is supported as an alternative to Buffer input argument | ||
|
There was a problem hiding this comment. It'd be good to expand this to support more types so we can use bl to avoid Buffer creation and also some sort of Buffer pool one day. Would we also not want to support typed arrays (Uint8Array and friends)? |
||
| - Streaming APIs switch to using async iterators/iterables | ||
| - The files API is revamped: | ||
| - Files functions moved to the root namespace | ||
| - Non-MFS and MFS functions are merged | ||
| - A revamped RESTful HTTP API exists | ||
|
There was a problem hiding this comment. It needs some love, but see: https://github.com/ipfs-shipyard/ipfs-http |
||
| - Old APIs are deprecated and removed (e.g. object, block) | ||
|
There was a problem hiding this comment. Maybe add 'Support the DAG API over http', otherwise we can't really remove the object API. There was a problem hiding this comment. I'd like to see the object API being removed, but I think the block API still makes sense even if we have a proper DAG API. There was a problem hiding this comment. I’d like to see the Block API stay unless we have another way for developers to grab blocks by some other means and add to IPFS. |
||
|
There was a problem hiding this comment. This is a nice to have - lets get the APIs we have already in line before we branch out to others. |
||
|
There was a problem hiding this comment. Dropped this - it's not critical to using IPFS with package managers or being production ready and dropping it gives us scope to improve in the future. |
||
| - High priority features requested from other working groups that have experienced pain points with the JS IPFS API are implemented e.g. Pagination and “Live” streams | ||
|
There was a problem hiding this comment. Moved from deleted milestone "Support for specific use cases" |
||
|
There was a problem hiding this comment. Dropped this last point - libdweb is now very unlikely to be available to all web extensions (potentially will be available to "blessed" extensions). From the sounds of it bluetooth might be difficult or involve complicated user flows (@vasco-santos might be able to confirm) but either way, exotic discovery methods are a nice to have because they'll reach a much smaller audience than the discovery methods we have already. |
||
|
|
||
| ## Timeline | ||
|
There was a problem hiding this comment. @achingbrain @daviddias I've had a go at organising the roadmap milestones and goals into a timeline (below). Please take a look and let me know if it looks sensible - I've tried to leave Q4 with slightly fewer tasks than the preceeding quarters so we have a bit of a buffer for overflow. Q1 is essentially stuff I know is in progress or done. cc-ing @vasco-santos because I believe gossipsub is on your libp2p roadmap - does that align with when you were hoping to look at it? There was a problem hiding this comment. It looks good but there's quite a lot in Q2. |
||
|
|
||
| N.B. Some big goals span multiple quarters and so are listed multiple times. | ||
|
|
||
| ### Q1 | ||
| * `M (P0)`: Revamped APIs (Part 1) | ||
| - Async/await is used throughout the codebase and callback APIs are dropped | ||
| - Streaming APIs switch to using async iterators/iterables | ||
| * `M (P0)`: Production ready (Part 1) | ||
| - A JS IPFS daemon runs as part of the IPFS gateway cluster for at least 1 month without crashing | ||
| - Performance metrics for a JS IPFS node are measurable: | ||
| - Tools can process and graph performance data | ||
| - Data can be queried to track down bugs and performance bottlenecks | ||
| - An interoperable DHT exists and scales to hundreds of thousands of nodes | ||
| * `M (P1)`: Installable npm modules via JS IPFS (Part 1) | ||
| - There is an option to select IPFS as a transport on npm/yarn | ||
| - A CLI tool exists to install packages directly from IPFS | ||
| - JS IPFS can ingest the entire npm registry | ||
|
|
||
| ### Q2 | ||
| * `M (P0)`: Production ready (Part 2) | ||
| - A JS IPFS daemon runs as part of the IPFS gateway cluster for at least 1 month without crashing | ||
| - JS IPFS nodes can discover one another for specific topics via the rendezvous protocol | ||
| - \*-star servers are no longer in use | ||
| - A gossipsub implementation exists | ||
| - Benchmarks exist to assert that: | ||
| - Memory footprint remains stable and should not consume more than 20% more memory than a Go IPFS node | ||
| - Network throughput is within an acceptable magnitude of Go IPFS | ||
| - Data can be fetched with comparable speeds to tools like bittorrent, wget or rsync when 5 peers are connected who already have the data | ||
| - Data can be exchanged with another IPFS node within 2x the speed a Go IPFS node would exchange it | ||
| * `M (P1)`: Optimized for browser environments (Part 1) | ||
| - Default performance profiles exist and can be enabled to adapt IPFS behaviour appropriately to its environment. In addition to the profiles [described here](https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#profiles), there will be a browser specific performance profile. | ||
| - Full build bundle size is monitored and does not exceed 2MB (minified) | ||
| * `M (P0)`: Revamped APIs (Part 2) | ||
| - Async/await is used throughout the codebase and callback APIs are dropped | ||
| - Streaming APIs switch to using async iterators/iterables | ||
| * `M (P1)`: Installable npm modules via JS IPFS (Part 2) | ||
| - Installing npm modules over LAN never fails (provided the module is available on another IPFS node connected to the LAN) | ||
| - Installing any module is faster than using npm when 5 peers on the local network already have the package | ||
| Node.js binaries are distributed via IPFS (making the whole Node.js/NPM installation process happen over IPFS) | ||
|
|
||
| ### Q3 | ||
| * `M (P0)`: Revamped APIs (Part 3) | ||
| - The files API is revamped: | ||
| - Files functions moved to the root namespace | ||
| - Non-MFS and MFS functions are merged | ||
| - A revamped RESTful HTTP API exists | ||
| * `M (P0)`: Production ready (Part 3) | ||
| - JS IPFS nodes use, and are compliant with the MDNS spec | ||
| - An IPFS log API is implemented and tests ensure log messages are part of the API | ||
| * `M (P1)`: Optimized for browser environments (Part 2) | ||
| - The JS IPFS build can be customised to exclude functionality or include different modules that are more appropriate for a particular runtime environment | ||
| - At least 2 recommended builds for common environments will be provided in addition to the full build | ||
| - Developers can tailor an IPFS build to their specific needs | ||
| - Examples for bundling JS IPFS with common tools such as webpack/parcel exist | ||
| * `M (P2)`: Discoverable npm modules via JS IPFS (Part 1) | ||
| - Unixfs v2 is finalized and is the default format for new content added to IPFS | ||
|
|
||
| ### Q4 | ||
| * `M (P0)`: Production ready (Part 4) | ||
| - A security audit is conducted on the code base and all issues resolved | ||
| * `M (P0)`: Revamped APIs (Part 4) | ||
| - Old APIs are deprecated and removed (e.g. object, block) | ||
| - ArrayBuffer is supported as an alternative to Buffer input argument | ||
| - High priority features requested from other working groups that have experienced pain points with the JS IPFS API are implemented e.g. Pagination and “Live” streams | ||
| * `M (P2)`: Improved documentation, specs and examples | ||
| - Code level: | ||
| - Functions/methods all have meaningful descriptions | ||
| - Functions/methods all have inline examples | ||
| - Parameters are all documented, including types and defaults | ||
| - Return values are documented, including object properties and types | ||
| - Onboarding documentation exists to explain domain concepts and provide tutorials/examples for achieving common tasks | ||
| * `M (P2)`: Discoverable npm modules via JS IPFS (Part 2) | ||
| - Users can search for modules in the npm registry via IPFS | ||
| - Users can learn about the latest registry index (module updates) via IPFS | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved from deleted milestone "Support for specific use cases"