- Makes it possible to decompress data granularly via push-based API.
 - Adds ability to flush output at an arbitrary point.
 - Removes manual `ended` check as zlib checks it for us.

Contents of kj-gzip are guarded by KJ_HAS_ZLIB, but the library itself should be still added unconditionally to non-CAPNP_LITE build for tests to link properly.

Reading an unbounded amount of data from a stream is a potential DoS vector. To manage this risk, readAllText() and readAllBytes() now accept a `limit` parameter. For source backwards-compatibility, this limit defaults to kj::maxValue.

The braced array literals seem to be constructed as temporaries. The test passes in debug builds, but in optimized builds, the arrays have been overwritten by the time they are used.

The gather-write and tryPumpFrom() member functions weren't encoding their chunk sizes in hex, and the gather-write case had an off-by-n error building the body data pieces.

Occasionally servers processing HTTP requests will see URLs in the wild which have unescaped hash signs in path and query components. This change allows kj::Url to parse such URLs in the Url::HTTP_REQUEST and Url::HTTP_PROXY_REQUEST contexts.

Similar methods already exist on AsyncInputStream.

It turns out I was testing it wrong, and it still doesn't work. :(

Theory is that maybe it did this intentionally, e.g. because it really just wanted to disconnect and it already dealt with any errors. The client should notice the stream ending prematurely so it should be possible for the developer to see the problem from the client side.

In theory KJ style should allow this to be caught (it doesn't throw if another exception was already thrown), but I've observed this producing an std::terminate() for some reason.

Some memory-based filesystems, namely tmpfs, select their underlying block size
equal to the configured page size of the platform/OS.

On systems with page size not equal to 4K (for example, arm64 can be configured
to have 16K or 64K page sizes -
https://www.kernel.org/doc/Documentation/arm64/memory.txt) "DiskFile holes"
test fails, when creating a file on a memory-based filesystem, because it
"punches" a 4096 byte hole in the file (using fallocate) and expects the number
of filesystem blocks used by the file to decrease. When the system page size is
greater than 4K, 4096 byte hole is not enough to make the OS release the
underlying blocks.

Need to punch a whole, which is equal to the underlying filesystem block size at
least.

It turns out wrapping an HttpService in an HttpClient is considerably more complicated than vice versa, due to the need for pipes. This commit adds a WebSocket pipe implementation very similar to the recent byte-stream pipe (though considerably simpler since there's no need to deal with mismatched buffer sizes).

MinGW 7.2 (x86_64) build failed

Yay for fake timers in tests.

This speeds up the test somewhat, but more importantly, it tests the pipe implementation across a variety of usage patterns.

This actually uncovered a bug in the HTTP implementation: An HttpClient could inadvertently issue overlapping reads in cases where multiple concurrent (pipelined) requests are made.

This lets you construct an AsyncInputStream / AsyncOutputStream pair that operates entirely within userspace, rather than pushing through a kernel-level pipe. This is far more efficient, avoiding system calls and reducing copies.

The pipe does not buffer at all. Instead, it waits for both a read() and a write() call to be active at the same time, and then it fulfills one with the other.

This implementation also optimizes pumps. Imagine the situation: you create a pipe; you call pumpTo() on the write end to pump it to some other; then you write to the write end of the pipe. In this case, the write will *directly* call the target stream to which the pipe is being pumped. Hence, adding daisy-chained pipes on top of a final output stream does not incur additional copies of the data. Similarly, tryPumpFrom() is optimized on the read end.

This is necessary when someone other than the promise owner might need to destroy the state a promise is operating on. It comes up when implementing pumpTo() for userland pipes -- cancelling the pump needs to cancel any writes currently passing through the pipe.

Previously, the app could control Content-Length by passing `expectedBodySize`. This is great for enabling code that "just works" by handling GET and HEAD requests identically.

However, in somewhat more-complicated situations -- especilaly in proxies -- you end up having to write special-case hacks for HEAD requests to deal with the fact that the body is actually empty, but has a non-zero "expected" size.

We can make life easier for proxies by allowing the application to directly set the Content-Length and Transfer-Encoding headers in the case of HEAD responses, much like we allow applications to set WebSocket-related headers on non-WebSocket requests/responses.

This change actually fixes a bug in Cloudflare Workers where Content-Length is not passed through correctly for HEAD responses. No changes are needed on the Workers side (except adding a test).

Our query string encoding function (encodeWwwForm()) was already doing the right thing.

I changed the comment in encodeUriPath() to clarify that it's intended to implement a URL class which stores its path in percent-decoded form, not either/or. I was wrong before.

We'll need more hostname-related code changes in the future, but for now this will solve a Cloudflare customer's existing bug report.

This has the effect of making the authority, path, and fragment components each slightly more permissive in what they don't escape.

According to the WHATWG URL spec, each different component of a URL gets its very own percent encode set, which we've been doing wrong this whole time.

In terms of reserved characters, the fragment set is a subset of the path set, which is a subset of the userinfo set, which is a subset of RFC 2396's reserved set.