- 18 Jun, 2019 20 commits
-
-
Kenton Varda authored
I think I imagined once upon a time that this would be a convenient way to deal with external interfaces that like to return nullable pointers. However, in practice it is used nowhere in KJ or Cap'n Proto, and it recently hid a bug in my code where I had assigned a `Maybe<T>` from an `Own<T>`. We can introduce a `fromNullablePointer()` helper or something if that turns out to be useful.
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
Consider a capnp streaming type that wraps a kj::AsyncOutputStream. KJ streams require the caller to avoid doing multiple writes at once. Capnp streaming conveniently guarantees only one streaming call will be delivered at a time. This is great because it means the app does not have to do its own queuing of writes. However, the app may want to use a CapabilityServerSet to unwrap the capability and get at the underlying KJ stream to optimize by writing to it directly. However, before it can issue a direct write, it has to wait for all RPC writes to complete. These RPC writes were probably issued by the same caller, before it realized it was talking to a local cap. Unfortunately, it can't just wait for those calls it issued to complete, because streaming flow control may have made them appear to complete long ago, when they're actually still in the server's queue. How does the app make sure that the directly-issued writes don't overlap with RPC writes? We can solve this by making CapabilityServerSet::getLocalServer() delay until all in-flight stream calls are complete before unwrapping. Now, the app can simply make sure that any requests it issued over RPC in the past completed before it starts issuing direct requests.
-
Kenton Varda authored
-
Kenton Varda authored
Also, push harder on the code generator such that `StreamResult` doesn't show up in generated code at all. So now we have `StreamingRequest<Params>` which is like `Request<Params, Results>`, and we have `StreamingCallContext<Params>` which is like `CallContext<Params, Results>`.
-
Kenton Varda authored
-
Kenton Varda authored
There are two things that every capability server must implement: * When a streaming method is delivered, it blocks subsequent calls on the same capability. Although not strictly needed to achieve flow control, this simplifies the implementation of streaming servers -- many would otherwise need to implement such serialization manually. * When a streaming method throws, all subsequent calls also throw the same exception. This is important because exceptions thrown by a streaming call might not actually be delivered to a client, since the client doesn't necessarily wait for the results before making the next call. Again, a streaming server could implement this manually, but almost all streaming servers will likely need it, and this makes things easier.
-
Kenton Varda authored
Note: Apparently, json.capnp had not been added to the bootstrap test, and the checked-in bootstrap had drifted from the source file.
-
Kenton Varda authored
This can be used on a method to indicate that it is used for "streaming", like: write @0 (bytes :Data) -> stream; A "streaming" method is one which is expected to be called many times to transmit an ordered stream of items. For best throughput, it is often necessary to make multiple overlapping calls, so as not to wait for a round trip for every item. However, to avoid excess buffering, it may be necessary to apply backpressure by having the client limit the total number of overlapping calls. This logic is difficult to get right at the application level, so making it a language feature gives us the opportunity to implement it in the RPC layer. We can, however, do it in a way that is backwards-compatible with implementations that don't support it. The above declaration is equivalent to: write @0 (bytes :Data) -> import "/capnp/stream.capnp".StreamResult; RPC implementations that don't explicitly support streaming can thus instead leave it up to the application to handle.
-
Kenton Varda authored
I have this pattern: Maybe<Own<T>> foo; // ... foo = heap<T>(); KJ_ASSERT_NONNULL(foo)->doSomething(); The assertion feels non-type-safe. Now you can do: auto& ref = foo.emplace(heap<T>()); ref.doSomething();
-
Kenton Varda authored
`kj::Quantity<T>` already supported this. I copied from it.
-
Kenton Varda authored
This was failing to chain the promises, and so returning `Promise<Promise<T>>`. The idea here is you can create a PromiseAdapter which eventually produces another promise to chain to. The adapter is finished and should be destroyed at that point, but the final promise should then redirect to the new promise.
-
Kenton Varda authored
Apparently, Return messages with empty capability tables have been allocated one word too small all along, causing many Return messages to be split into two segments and allocate twice the memory they need. I never bothered to check whether this was happening...
-
Kenton Varda authored
-
- 16 Jun, 2019 5 commits
-
-
Kenton Varda authored
Way back in 538a767e I added `RpcSystem::setFlowLimit()`, a blunt mechanism by which an RPC node can arrange to stop reading new messages from the connection when too many incoming calls are in-flight. This was needed to deal with buggy Sandstorm apps that would stream multi-gigabyte files by doing a zillion writes without waiting, which would then all be queued in the HTTP gateway, causing it to run out of memory. In implementing that, I inadertently caused the RPC system to do a tree walk on every call message it received, in order to sum up the message size. This is silly, becaues it's much cheaper to sum up the segment sizes. In fact, in the case of a malicious peer, the tree walk is potentially insufficient, because it doesn't count holes in the segments. The tree walk also means that any invalid pointers in the message cause an exception to be thrown even if that pointer is never accessed by the app, which isn't the usual behavior. I seem to recall this issue coming up in discussion once in the past, but I couldn't find the thread. For the new streaming feature, we'll be paying attention to the size of outgoing messages. Again, here, it would be nice to compute this size by summing segments without doing a tree walk. So, this commit adds `sizeInWords()` methods that do this.
-
Kenton Varda authored
Add OpenBSD to posix_memalign ifdef
-
pwrdwnsys authored
-
Kenton Varda authored
Introduce kj::attachVal(), kj::attachRef(), and capnp::clone() utility functions
-
Kenton Varda authored
Add TwoPartyServer::drain().
-
- 15 Jun, 2019 4 commits
-
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
This is frequently needed. Sandstorm had `OwnCapnp` for this purpose: https://github.com/sandstorm-io/sandstorm/blob/4d86a8144cdb43120ea12845738d0fe4a6ffcda1/src/sandstorm/util.h#L495-L525 The Workers codebase has some ad-hoc copies of this logic too, and multiple people have requested something similar on the mailing list.
-
Kenton Varda authored
This is a lot like Own<T>::attach() but for the case where you don't have a Own pointer, you just have a reference or value that you want to attach stuff to.
-
- 14 Jun, 2019 1 commit
-
-
pwrdwnsys authored
-
- 10 Jun, 2019 3 commits
-
-
Harris Hancock authored
Align HTTP entity-body delimiting rules with RFC 7230.
-
Harris Hancock authored
Skip test that fails under qemu-user, probably due to a qemu bug.
-
Kenton Varda authored
The main case where the code was wrong is when neither Content-Length nor Transfer-Encoding was provided on a response. In this case the response is delimited by closing the connection, but KJ previously rejected it outright. AFAICT almost no one on the whole internet relies on this anymore... almost.
-
- 09 Jun, 2019 3 commits
-
-
Kenton Varda authored
Fix exclusiveJoin() bug when both branches complete simultaneously.
-
Kenton Varda authored
-
Kenton Varda authored
-
- 30 May, 2019 2 commits
-
-
Kenton Varda authored
Implement FD passing in Cap'n Proto.
-
Kenton Varda authored
-
- 27 May, 2019 2 commits
-
-
Kenton Varda authored
Fix uninitialized byte arrays in encoding-test.
-
Kenton Varda authored
Initializer lists are temporaries. The code apparently works in debug mode but fails when optimized.
-