Capability::Clients do not follow the usual KJ style with regards to lifetimes of returned promises. RPC methods in particular automatically take a reference on the capability until the method completes. This makes some intuitive sense as Capability::Client itself is a pointer-like type implementing reference counting on some inner object.

whenResolved() did not follow the pattern, and instead required that the caller explicitly take a reference. I screwed this up when using it, suggesting that it's pretty unintuitive. It's cheap and safe to automatically take a reference, so let's do that.

A pump does not propagate EOF. So a BlockedRead state should not complete when a pump happens that does not satisfy the read.

Add a little more detail to invalid response status line errors.

Fix http-test.c++ to avoid dubious assumptions about gather-writes.

Most (all?) implementations of `write(ArrayPtr<const ArrayPtr<const byte>>)`, if the outer array contains only one inner array, do not use the outer array again after the initial call returns (as opposed to the promise resolving). But, this is not a safe assumption and http-test.c++ should not be relying on it.

(I found this when I tried forcing all writes to complete asynchronously to check if it resulted in any bugs. This is all I found.)

Add `kj::evalLast()` for running a callback after all other events are done.

Return HTTP 501 instead of 400 on unrecognized method.

Fix some error propagation bugs in HTTP.

Report raw HTTP content when handling client protocol errors in kj-http

This information is typically necessary to debug such protocol errors. However, it requires special handling, since it could contain sensitive information.

The code which has the best view on protocol errors is HttpHeaders::tryParseRequest(), which previously returned an empty Maybe<Request> on failure. This commit changes that function to return a OneOf<Request, ProtocolError>. This required some surgery in various other parts of the code to deal with the OneOf.

Add HttpServerErrorHandler interface to provide visibility and customization of protocol errors

The HttpServerErrorHandler is separate from the HttpService interface, because error-handling is more in the domain of the HttpServer, which can export multiple HttpServices (via the factory constructor overload).

I broke this while iterating on the new error handler mechanism, so here's a test.

Maybe<T> has a non-trivial destructor, but there's no point making it constexpr, but Maybe<T&> can be.

I decided that such wrapping did not make senes.

 Extend KJ event loop to support cross-thread events.

They are more efficient, and self-contained enough not to create trouble.

Also, Cygwin's pthread_rwlock implementation appears buggy. I am seeing it allow double locks from time to time.

Also required making KJ_WIN32 macros available on Cygwin.

(Cygwin allows direct calls to Win32 functions.)

I observed the cygwin async-xthread-test getting deadlocked here, and noticed the bug. However, the predicate in question was not flappy, so this doesn't really fix async-xthread-test.

A long, long time ago, an early version of the event loop code was multithread-aware and used futexes. I ripped that all out later but apparently didn't remove these includes.

The delay() here isn't long enough on Cygwin when using pipes for wakeup. Apparently, Cygwin pipes are very slow. :/

See my latest report: https://cygwin.com/ml/cygwin/2019-07/msg00052.html

We can't use signals for cross-thread wakeups on Cygwin because the same signal cannot be pending on two different threads at the same time.

So I broke down and made an implementation that uses pipes. Ugh.

Apparently, `sigprocmask()` on macOS affects all threads. You must use `pthread_sigmask()` to affect only the current thread. (To be fair, POSIX says that `sigprocmask()` has unspecified behavior in the presence of threads. However, having it affect all threads is bizarre. On Linux, it affects only the calling thread.)

Moreover, `siglongjmp()` on macOS is implemented in terms of `sigprocmask()`. Hence, `siglongjmp()` (with `true` for the second parameter) is not safe to use in a multithreaded program.

Instead, we must decompose `siglongjmp()` into its component parts, so that we can correctly use `pthread_sigmask()`.

This fixes the deadlocks on macOS.

Cygwin seems to be having an issue with this...

I guess this is particularly likely to happen when the threads are sharing a single core, which is probably common in CI services, explaining why I had trouble reproducing on my own hardware.

I should have done this a long time ago. We don't get any benefit from the parallel test runner, but we get a massive disadvantage in CI from not being able to see the logs.