Fixes #10293 - Improve documentation on how to write a response body …

…in Jetty 12.

Updated documentation about:
* Content.Source
* Content.Sink
* Handler
* Request/Response

Updated few APIs to make easier to write applications.

Signed-off-by: Simone Bordet <simone.bordet@gmail.com>

documentation/jetty-documentation/src/main/asciidoc/programming-guide/arch-io.adoc

@@ -200,3 +200,134 @@ Asynchronous programming is hard.
 
 Rely on the Jetty classes to implement `Connection` to avoid mistakes that will be difficult to diagnose and reproduce.
 ====
+
+[[pg-arch-io-content-source]]
+==== `Content.Source`
+
+The high-level abstraction that Jetty offers to read bytes is `org.eclipse.jetty.io.Content.Source`.
+
+`Content.Source` offers a non-blocking demand/read model where a read returns a `Content.Chunk` (see also xref:pg-arch-io-content-source-chunk[this section]).
+
+A `Content.Chunk` groups the following information:
+
+* A `ByteBuffer` with the bytes that have been read; it may be empty.
+* Whether the read reached end-of-file.
+* A failure that might have happened during the read.
+
+The ``Content.Chunk``'s `ByteBuffer` is typically a slice of a different `ByteBuffer` that has been read by a lower layer.
+There may be multiple layers between the bottom layer (where the initial read typically happens) and the application layer.
+
+By slicing the `ByteBuffer` (rather than copying its bytes), there is no copy of the bytes between the layers.
+However, this comes with the cost that the `ByteBuffer`, and the associated `Content.Chunk`, have an intrinsic lifecycle: the final consumer of a `Content.Chunk` at the application layer must indicate when it has consumed the chunk, so that the bottom layer may reuse/recycle the `ByteBuffer`.
+
+Consuming the chunk means that the bytes in the `ByteBuffer` are read (or ignored), and that the application will not look at or reference that `ByteBuffer` ever again.
+
+`Content.Chunk` offers a retain/release model to deal with the `ByteBuffer` lifecycle, with a simple rule:
+
+IMPORTANT: A `Content.Chunk` returned by a call to `Content.Source.read()` **must** be released.
+
+The example below is the idiomatic way to read from a `Content.Source`:
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=idiomatic]
+----
+<1> The `read()` that must be paired with a `release()`.
+<2> The `release()` that pairs the `read()`.
+
+Note how the reads happens in a loop, consuming the `Content.Source` as soon as it has content available to be read, and therefore no backpressure is applied to the reads.
+
+An alternative way to read from a `Content.Source`, to use when the chunk is consumed asynchronously, and you don't want to read again until the `Content.Chunk` is consumed, is the following:
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=async]
+----
+<1> The `read()` that must be paired with a `release()`.
+<2> The `release()` that pairs the `read()`.
+
+Note how the reads do not happen in a loop, and therefore backpressure is applied to the reads, because there is not a next read until the chunk from the previous read has been consumed (and this may take time).
+
+You can use `Content.Source` static methods to conveniently read (in a blocking way or non-blocking way), for example via `static Content.Source.asStringAsync(Content.Source, Charset)`, or via an `InputStream` using `static Content.Source.asInputStream(Content.Source source)`.
+
+For
+
+Refer to the `Content.Source` link:{javadoc-url}/org/eclipse/jetty/io/Content.Source.html[`javadocs`] for further details.
+
+[[pg-arch-io-content-source-chunk]]
+===== `Content.Chunk`
+
+`Content.Chunk` offers a retain/release API to control the lifecycle of its `ByteBuffer`.
+
+When ``Content.Chunk``s are consumed synchronously, no additional retain/release API call is necessary, for example:
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=chunkSync]
+----
+
+On the other hand, if the `Content.Chunk` is not consumed immediately, then it must be retained, and you must arrange for the `Content.Chunk` to be released at a later time, thus pairing the retain.
+For example, you may accumulate the ``Content.Chunk``s in a `List` to convert them to a `String` when all the ``Content.Chunk``s have been read.
+
+Since reading from a `Content.Source` is asynchronous, the `String` result is produced via a `CompletableFuture`:
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=chunkAsync]
+----
+<1> The `read()` that must be paired with a `release()`.
+<2> The `release()` that pairs the `read()`.
+<3> The `retain()` that must be paired with a `release()`.
+<4> The `release()` that pairs the `retain()`.
+
+Note how method `consume(Content.Chunk)` retains the `Content.Chunk` because it does not consume it, but rather stores it away for later use.
+With this additional retain, the retain count is now `2`: one implicitly from the `read()` that returned the `Content.Chunk`, and one explicit in `consume(Content.Chunk)`.
+
+However, just after returning from `consume(Content.Chunk)` the `Content.Chunk` is released (pairing the implicit retain from `read()`), so that the retain count goes to `1`, and an additional release is still necessary.
+
+Method `getResult()` arranges to release all the ``Content.Chunk``s that have been accumulated, pairing the retains done in `consume(Content.Chunk)`, so that the retain count for the ``Content.Chunk``s goes finally to `0`.
+
+[[pg-arch-io-content-sink]]
+==== `Content.Sink`
+
+The high-level abstraction that Jetty offers to write bytes is `org.eclipse.jetty.io.Content.Sink`.
+
+The primary method to use is `Content.Sink.write(boolean, ByteBuffer, Callback)`, which performs a non-blocking write of the given `ByteBuffer`, with the indication of whether the write is the last.
+
+The `Callback` parameter is completed, successfully or with a failure, and possibly asynchronously by a different thread, when the write is complete.
+
+Your application can typically perform zero or more non-last writes, and one final last write.
+
+However, because the writes may be asynchronous, you cannot start a next write before the previous write is completed.
+
+This code is wrong:
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=sinkWrong]
+----
+
+You must initiate a second write only when the first is finished, for example:
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=sinkMany]
+----
+
+When you need to perform an unknown number of writes, you must use an `IteratingCallback`, explained in xref:pg-arch-io-echo[this section], to avoid ``StackOverFlowError``s.
+
+For example, to copy from a `Content.Source` to a `Content.Sink` you should use the convenience method `Content.copy(Content.Source, Content.Sink, Callback)`.
+For illustrative purposes, below you can find the implementation of `copy(Content.Source, Content.Sink, Callback)` that uses an `IteratingCallback`:
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=copy]
+----
+
+Non-blocking writes can be easily turned in blocking writes.
+This leads to perhaps code that is simpler to read, but that also comes with a price: greater resource usage that may lead to less scalability and less performance.
+
+[source,java,indent=0]
+----
+include::{doc_code}/org/eclipse/jetty/docs/programming/ContentDocs.java[tags=blocking]
+----

documentation/jetty-documentation/src/main/asciidoc/programming-guide/client/http/client-http-api.adoc

@@ -224,6 +224,8 @@ The listener that follows this model is `Response.ContentSourceListener`.
 After the response headers have been processed by the `HttpClient` implementation, `Response.ContentSourceListener.onContentSource(response, contentSource)` is invoked once and only once.
 This allows the application to control precisely the read/demand loop: when to read a chunk, how to process it and when to demand the next one.
 
+// TODO: move this section to the IO arch docs.
+
 You must provide a `ContentSourceListener` whose implementation reads a `Content.Chunk` from the provided `Content.Source`, as follows:
 Then the following cases may happen:
 

documentation/jetty-documentation/src/main/asciidoc/programming-guide/migration/migration-11-to-12.adoc

@@ -157,55 +157,4 @@ The old model was a "demand+push" model: the application was demanding content;
 
 The new model is a "demand+pull" model: when the content is available, the implementation calls once `Response.ContentSourceListener.onContentSource(Content.Source)`; the application can then pull the content chunks from the `Content.Source`.
 
-Old usage:
-
-[source, java]
-----
-request.onResponseContentDemanded(new Response.DemandedContentListener()
-{
-    @Override
-    public void onBeforeContent(Response response, LongConsumer demand)
-    {
-        // Demand for first content.
-        demand.accept(1);
-    }
-
-    @Override
-    public void onContent(Response response, LongConsumer demand, ByteBuffer content, Callback callback)
-    {
-        // Consume the content.
-        callback.succeeded();
-
-        // Demand for more content.
-        demand.accept(1);
-    }
-});
-----
-
-New usage:
-
-[source, java]
-----
-request.onResponseContentSource((response, source) ->
-{
-    read(source);
-}
-
-private void read(Content.Source source)
-{
-    while (true)
-    {
-        // Pull the content from the source.
-        Content.Chunk chunk = source.read();
-
-        if (chunk == null)
-        {
-            source.demand(() -> read(source));
-            return;
-        }
-
-        // Consume the content.
-        chunk.release();
-    }
-});
-----
+For more information about the new model, see xref:pg-arch-io-content-source[this section].

documentation/jetty-documentation/src/main/asciidoc/programming-guide/server/http/server-http-connector.adoc

@@ -20,8 +20,10 @@ The available implementations are:
 
 * `org.eclipse.jetty.server.ServerConnector`, for TCP/IP sockets.
 * `org.eclipse.jetty.unixdomain.server.UnixDomainServerConnector` for Unix-Domain sockets (requires Java 16 or later).
+* `org.eclipse.jetty.quic.server.QuicServerConnector`, for the low-level QUIC protocol.
+* `org.eclipse.jetty.http3.server.HTTP3ServerConnector` for the HTTP/3 protocol.
 
-Both use a `java.nio.channels.ServerSocketChannel` to listen to a socket address and to accept socket connections.
+The first two use a `java.nio.channels.ServerSocketChannel` to listen to a socket address and to accept socket connections, while last two use a `java.nio.channels.DatagramChannel`.
 
 Since `ServerConnector` wraps a `ServerSocketChannel`, it can be configured in a similar way, for example the IP port to listen to, the IP address to bind to, etc.:
 
@@ -30,7 +32,7 @@ Since `ServerConnector` wraps a `ServerSocketChannel`, it can be configured in a
 include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=configureConnector]
 ----
 
-Likewise, `UnixDomainServerConnector` also wraps a `ServerSocketChannel` and can be configured with the Unix-Domain path to listen to:
+`UnixDomainServerConnector` also wraps a `ServerSocketChannel` and can be configured with the Unix-Domain path to listen to:
 
 [source,java,indent=0]
 ----
@@ -42,6 +44,13 @@ include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPSer
 You can use Unix-Domain sockets support only when you run your server with Java 16 or later.
 ====
 
+`QuicServerConnector` and its extension `HTTP3ServerConnector` wrap a `DatagramChannel` and can be configured in a similar way:
+
+[source,java,indent=0]
+----
+include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=configureConnectorQuic]
+----
+
 The _acceptors_ are threads (typically only one) that compete to accept socket connections.
 When a connection is accepted, `ServerConnector` wraps the accepted `SocketChannel` and passes it to the xref:pg-arch-io-selector-manager[`SelectorManager`].
 Therefore, there is a little moment where the acceptor thread is not accepting new connections because it is busy wrapping the just accepted connection to pass it to the `SelectorManager`.
@@ -53,12 +62,17 @@ The _selectors_ are components that manage a set of connected sockets, implement
 Each selector requires one thread and uses the Java NIO mechanism to efficiently handle a set of connected sockets.
 As a rule of thumb, a single selector can easily manage up to 1000-5000 sockets, although the number may vary greatly depending on the application.
 
-For example, web site applications tend to use sockets for one or more HTTP requests to retrieve resources and then the socket is idle for most of the time.
+For example, web applications for websites tend to use sockets for one or more HTTP requests to retrieve resources and then the socket is idle for most of the time.
 In this case a single selector may be able to manage many sockets because chances are that they will be idle most of the time.
-On the contrary, web messaging applications tend to send many small messages at a very high frequency so that sockets are rarely idle.
-In this case a single selector may be able to manage less sockets because chances are that many of them will be active at the same time.
+On the contrary, web messaging applications or REST applications tend to send many small messages at a very high frequency so that sockets are rarely idle.
+In this case a single selector may be able to manage less sockets because chances are that many of them will be active at the same time, so you may need more than one.
+
+It is possible to configure more than one `Connector` per `Server`.
+Typical cases are a `ServerConnector` for clear-text HTTP, and another `ServerConnector` for secure HTTP.
+Another case could be a publicly exposed `ServerConnector` for secure HTTP, and an internally exposed `UnixDomainServerConnector` for clear-text HTTP.
+Yet another example could be a `ServerConnector` for clear-text HTTP, a `ServerConnector` for secure HTTP/2, and an `HTTP3ServerConnector` for QUIC+HTTP/3.
 
-It is possible to configure more than one `ServerConnector` (each listening on a different port), or more than one `UnixDomainServerConnector` (each listening on a different path), or ``ServerConnector``s and ``UnixDomainServerConnector``s, for example:
+For example:
 
 [source,java,indent=0]
 ----