From 7801931f4d0b4704d3468714214c7132363e191c Mon Sep 17 00:00:00 2001 From: Sean McArthur Date: Tue, 12 Apr 2022 11:37:19 -0700 Subject: [PATCH] docs(lib): define the VISION and TENETS --- docs/TENETS.md | 100 +++++++++++++++++++ docs/VISION.md | 230 +++++++++++++++++++++++++++++++++++++++++++ docs/vision-arch.svg | 4 + 3 files changed, 334 insertions(+) create mode 100644 docs/TENETS.md create mode 100644 docs/VISION.md create mode 100644 docs/vision-arch.svg diff --git a/docs/TENETS.md b/docs/TENETS.md new file mode 100644 index 0000000000..91e41492e3 --- /dev/null +++ b/docs/TENETS.md @@ -0,0 +1,100 @@ +# Charter + +> hyper is a protective and efficient HTTP library for all. + +# Tenets + +Tenets are guiding principles. They guide how decisions are made for the whole +project. Ideally, we do all of them all the time. In some cases, though, we may +be forced to decide between slightly penalizing one goal or another. In that +case, we tend to support those goals that come earlier in the list over those +that come later (but every case is different). + +## 0. Open + +hyper is open source, always. The success of hyper depends on the health of the +community building and using it. All contributions are in the open. We don't +maintain private versions, and don't include features that aren't useful to +others. + +[We prioritize kindness][CONDUCT], compassion and empathy towards all +contributors. Technical skill is not a substitute for human decency. + +[CONDUCT]: https://github.com/hyperium/hyper/blob/master/docs/CODE_OF_CONDUCT.md + +### Examples + +It's not usually hard for an open source library to stay open and also meet its +other priorities. Here's some instances where being **Open** would be more +important than **Correct** or **Fast**: + +- Say an individual were to bring forward a contribution that makes hyper more + correct, or faster, perhaps fixing some serious bug. But in doing so, they + also insulted people, harrassed other contributors or users, or shamed + everyone for the previous code. They felt their contribution was "invaluable". + We would not accept such a contribution, instead banning the user and + rewriting the code amongst the kind collaborators of the project. + +- Say someone brings a contribution that adds a new feature useful for + performance or correctness, but their work accomplishes this by integrating + hyper with a proprietary library. We would not accept such a contribution, + because we don't want such a feature limited only to those users willing to + compromise openness, and we don't want to bifurcate the ecosystem between those + who make that compromise and those who don't. + +## 1. Correct + +hyper is a memory safe and precise implementation of the HTTP specification. +Memory safety is vital in a core Internet technology. Following the HTTP +specifications correctly protects users. It makes the software durable to the +“real world”. Where feasible, hyper enforces correct usage. + +This is more than just "don't write bugs". hyper actively protects the user. + +### Examples + +- Even though we follow the **HTTP/\*** specs, hyper doesn't blindly implement + everything without considering if it is safe to do so. + +## 2. Fast + +A fast experience delights users. A faster network library means a faster +application, resulting in delighting our users’ users. Whether with one request, +or millions. + +Being _fast_ means we improve throughput, drive down CPU usage, and improve +sustainability. + +Fast _enough_. We don't sacrifice sanity for speed. + +## 3. HTTP/* + +hyper is specifically focused on HTTP. Supporting new HTTP versions is in scope, +but supporting separate protocols is not. + +This also defines what the abstraction layer is: the API is designed around +sending and receiving HTTP messages. + +## 4. Flexible + +hyper enables as many usecases as possible. It has no opinion on application +structure, and makes few assumptions about its environment. This includes being +portable to different operating systems. + +### Examples + +- While we choose safer defaults to be **Correct**, hyper includes options to + _allow_ different behavior, when the user requires them. +- Providing choice usually makes things more complex, so being **Flexible** does + mean it's less _easy_. That can sometimes conflict with simplest way of making + hyper **Understandable**. + +## 5. Understandable + +hyper is [no more complicated than it has to +be](https://en.wikipedia.org/wiki/Occam%27s_razor). HTTP is not simple. It may +not be as "easy" as 1-line to do everything, but it shouldn't be "hard" to find +the answers. + +From logical and misuse-resistant APIs, to stellar documentation, to transparent +metrics. diff --git a/docs/VISION.md b/docs/VISION.md new file mode 100644 index 0000000000..4c9c3b917b --- /dev/null +++ b/docs/VISION.md @@ -0,0 +1,230 @@ +# hyper Vision + +## Purpose + +This is an overview of what the shape of hyper looks like, but also somewhat +zoomed out, so that the _vision_ can survive while the exact minute details +might shift and change over time. + +### Charter + +> hyper is a protective and efficient HTTP library for all. + +### Tenets + +Tenets are guiding principles. They guide how decisions are made for the whole +project. Ideally, we do all of them all the time. In some cases, though, we may +be forced to decide between slightly penalizing one goal or another. In that +case, we tend to support those goals that come earlier in the list over those +that come later (but every case is different). + +0. Open +1. Correct +2. Fast +3. HTTP/\* +4. Flexible +5. Understandable + +There's a lot more detail about each in [TENETS](./TENETS.md). + +## Use Cases + +Who are the *users* of hyper? How would they use hyper? + +### Low-Level Client Library (curl, reqwest, aws-sdk) + +These client libraries care that hyper is **Flexible**, since they are +expressing their own opinion on how a more-featured HTTP client should act. +This includes opinions on connection establishment, management, pooling, HTTP +version options, and even runtimes. + +curl's main reason for using hyper is that it is **Safe**. + +### Web Server Frameworks (deno, axum) + +These are using hyper's server feature to expose a different, higher-level API +to users. Besides the obvious requirements, these require that hyper is +**Fast**. Servers are costly, handling more requests faster is important to +them. + +That hyper is **Flexible** is also important, in that it needs to be flexible +enough for them to build a server framework, and allow them to express their +own opinions about API to their users. + +### Services and Proxies (linkerd, cloudflare, fastly) + +These are using hyper directly, likely both the client and server, in order to +build efficient and powerful services, applications, and tools for their end +users. They care greatly that hyper is **Correct**, since web traffic can +stretch the limits of what is valid HTTP, and exercise less-common parts of the +specifications. + +They also require hyper to be **Fast**, for similar reasons that the web server +frameworks do. + +### New Rust Web Developers + +These are developers who are either new to Rust, or new to web servers, and +have reached for hyper to start with. + +It's likely that these users don't have strong opinions about how an HTTP +server or client should work, just that it _should_ handle all the things they +normally assume it would. For these users, it would be best to quickly help +them compare their own expectations with hyper's capabitilities, and may +suggest reaching for higher-level, _easier_ libraries instead. + +Those that stick around after that recommendation are users that wish both to +learn at a lower level, and to pick and choose what batteries they plug in to +hyper as they move along. While they do care about the other tenets, that hyper +is **Understandable** is of extra importance to them. + +## The Library + +So with all that context in mind, what does hyper, the library, actually look +like? This doesn't highlight what _is_ and _isn't_ present. What currently +needs to change to reach this vision is left to individual version roadmaps. + +### Layers + +In all cases, a user brings their own runtime and IO to work with hyper. The IO +is provided to hyper, and hyper acts on top of it. hyper returns `Future`s that +the user then decides how to poll, likely involving their runtime options. + +![architecture diagram](./vision-arch.svg) + + +#### Protocol Codecs + +hyper has dedicated codecs for the major HTTP versions. Each is internally +designed to be **Correct** and **Fast** when it comes to encoding and decoding. + +The individual codecs may be implemented as sub-crates, with a less-stable +promise, to support the **Flexible** needs of some users who wish to build +their own connection management, or customize encoding and decoding beyond what +is officially supported. + +#### Connection State Management + +A **Correct** implementation includes more than just enforcing certain +characters when encoding and decoding. Order of frames, and flags in certain +frames can affect the state of the connection. Some examples of things enforced +at this layer: + +- If a message has a `content-length`, enforce only that many bytes are read or + written. +- Reading a `Response` before a `Request` is even written implies a mismatched + reply that should be interpreted as an error. +- The presence of some headers, such as `Connection: close`, or the absence of + others, such as `content-length` and `transfer-encoding`, can mean that the + connection should terminate after the current message. +- HTTP/2 and HTTP/3 may send connection-level frames that don't pertain to any + specific transaction, and must be read and handled regardless of if a user is + currently checking for a message. + +#### HTTP Role and Version Abstraction + +This is the public API layer. Methods exposed are around sending and receiving +`http::Request`s and `http::Response`s, not around framing specifics of the +different versions. These are built around a client or server `Connection` +interface. + +By exposing this layer publicly, we take care of the **Correct** tenet, by not +forcing the user to send the specific frames themselves. The API should be +designed in a way that a user cannot easily (if at all) create an _incorrect_ +HTTP connection. + +Motivated by the **Flexible** tenet, there _are_ version-specific options that +can be configured at this level, and version-specific functionality can usually +be handled via `http::Extensions`. + +### Not quite stable, but utile (useful) + +Beyond what is directly in the hyper crate, there are useful (utile) parts that +may not meet hyper's stability promise. Developing, experimenting, and exposing +those parts is the purpose of the `hyper-util` crate. That crate does not have +the same stability level as hyper. However, the goal is that things that other +libraries might want to expose as a public dependency do not live in +`hyper-util` forever, but rather stabilize and get promoted into `hyper`. + +Exactly what gets put into `hyper-util` presently is kept in the roadmap +documents. + +### Stability Promise + +What even is hyper's stability promise? Does it mean we are "done"? No. Will we +ever make breaking changes again? Probably. We'll still follow the [semantic +versioning](https://semver.org). + +Prior to 1.0, hyper has already only done breaking changes once a year. So 1 +year isn't much of a promise. We'll have significant more use and understanding +after a few years, and that could prompt some redesign. + +As of this writing, we'll promise that _major_ versions of hyper are stable for +3 years. New features will come out in _minor_ versions frequently. If it is +determined necessary to make breaking changes to the API, we'll save them for +after the 3 years. + +hyper also establishes a Minimum Supported Rust Version (MSRV). hyper will +support Rust versions at least 6 months old. If a new Rust version is released +with a feature hyper wishes to use, we won't do so until at least 6 months +afterwards. hyper will only ever require a new Rust version as a _minor_ +release (1.x), not as a patch (1.x.y). + +## Security + +The security of hyper is a large part of what makes hyper _protective_. We make +hyper secure via the combined efforts of being **Correct**, focusing on +**HTTP/\***, and making it all **Understandable**. + +### Memory Safety + +Being **Correct** requires that hyper be memory-safe. Using the Rust language +gets us most of the way there. But there is the ability to write `unsafe` +Rust. Does being **Correct** mean that we can _never_ write `unsafe` code +anywhere? Even if it helps make hyper **Fast**? We can, carefully. + +How do we balance the two, so that hyper is secure? + +hyper prefers not to have large modules of intertwined `unsafe` code. hyper +does allow small `unsafe` blocks, no more than a few lines, where it's easier +to verify that the `unsafe` code was written **Correctly**. + +### Meticulous Testing + +hyper's test suite grows and grows. There's a lot that needs to be right. +Parsers, encoders, state machines. When easily isolated, those pieces have +internal unit tests. But hyper also keeps a large list of growing integration +tests that make sure all the parts are **Correct**. + +Making writing new tests easy is a high priority. Investing in the testing +infrastructure is a proven way to make sure hyper stays **Correct** and secure. + +### Constant Fuzzing + +One thing is to know specific cases to test for. But we can't know all the +inputs or states that *might* cause a bug. That's why hyper has rounds of +fuzzing built into its CI. It's also why hyper signs up for and uses resources +to provide *constant*, around-the-clock fuzzing, always looking for something +that hyper should be hardened against. + +### Security Process + +hyper has an outlined +[SECURITY](https://github.com/hyperium/hyper/blob/master/SECURITY.md) process, +so we can safely report and fix issues. + +## Non-goals + +After writing this up, it is easier to articulate what sorts of things many +might associate with an HTTP library, but which are explicitly *not* for hyper. +These are all things that definitely **out of scope**. + +- TLS: We learned early that bundling TLS directly in hyper [has + problems](https://github.com/hyperium/hyper/issues/985). People also have + very strong opinions about which TLS implementation to use. The design of + hyper allows users to bring their own TLS. +- Routing +- Cookies +- Not-HTTP: WebSockets, or other protocols that are built next to HTTP. It + should be possible to _use_ hyper to upgrade, but the actual next-protocol + should be handled by a different library. diff --git a/docs/vision-arch.svg b/docs/vision-arch.svg new file mode 100644 index 0000000000..c0a8f9b0e6 --- /dev/null +++ b/docs/vision-arch.svg @@ -0,0 +1,4 @@ + + + +
client
client
server
server
hyper::proto
hyper::proto
http1
http1
h2
h2
h3
h3
IO
IO
executor / runtime
executor / runtimeText is not SVG - cannot display