- 18 Jun, 2019 13 commits
-
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
Consider a capnp streaming type that wraps a kj::AsyncOutputStream. KJ streams require the caller to avoid doing multiple writes at once. Capnp streaming conveniently guarantees only one streaming call will be delivered at a time. This is great because it means the app does not have to do its own queuing of writes. However, the app may want to use a CapabilityServerSet to unwrap the capability and get at the underlying KJ stream to optimize by writing to it directly. However, before it can issue a direct write, it has to wait for all RPC writes to complete. These RPC writes were probably issued by the same caller, before it realized it was talking to a local cap. Unfortunately, it can't just wait for those calls it issued to complete, because streaming flow control may have made them appear to complete long ago, when they're actually still in the server's queue. How does the app make sure that the directly-issued writes don't overlap with RPC writes? We can solve this by making CapabilityServerSet::getLocalServer() delay until all in-flight stream calls are complete before unwrapping. Now, the app can simply make sure that any requests it issued over RPC in the past completed before it starts issuing direct requests.
-
Kenton Varda authored
-
Kenton Varda authored
Also, push harder on the code generator such that `StreamResult` doesn't show up in generated code at all. So now we have `StreamingRequest<Params>` which is like `Request<Params, Results>`, and we have `StreamingCallContext<Params>` which is like `CallContext<Params, Results>`.
-
Kenton Varda authored
-
Kenton Varda authored
There are two things that every capability server must implement: * When a streaming method is delivered, it blocks subsequent calls on the same capability. Although not strictly needed to achieve flow control, this simplifies the implementation of streaming servers -- many would otherwise need to implement such serialization manually. * When a streaming method throws, all subsequent calls also throw the same exception. This is important because exceptions thrown by a streaming call might not actually be delivered to a client, since the client doesn't necessarily wait for the results before making the next call. Again, a streaming server could implement this manually, but almost all streaming servers will likely need it, and this makes things easier.
-
Kenton Varda authored
Note: Apparently, json.capnp had not been added to the bootstrap test, and the checked-in bootstrap had drifted from the source file.
-
Kenton Varda authored
This can be used on a method to indicate that it is used for "streaming", like: write @0 (bytes :Data) -> stream; A "streaming" method is one which is expected to be called many times to transmit an ordered stream of items. For best throughput, it is often necessary to make multiple overlapping calls, so as not to wait for a round trip for every item. However, to avoid excess buffering, it may be necessary to apply backpressure by having the client limit the total number of overlapping calls. This logic is difficult to get right at the application level, so making it a language feature gives us the opportunity to implement it in the RPC layer. We can, however, do it in a way that is backwards-compatible with implementations that don't support it. The above declaration is equivalent to: write @0 (bytes :Data) -> import "/capnp/stream.capnp".StreamResult; RPC implementations that don't explicitly support streaming can thus instead leave it up to the application to handle.
-
Kenton Varda authored
Apparently, Return messages with empty capability tables have been allocated one word too small all along, causing many Return messages to be split into two segments and allocate twice the memory they need. I never bothered to check whether this was happening...
-
- 16 Jun, 2019 1 commit
-
-
Kenton Varda authored
Way back in 538a767e I added `RpcSystem::setFlowLimit()`, a blunt mechanism by which an RPC node can arrange to stop reading new messages from the connection when too many incoming calls are in-flight. This was needed to deal with buggy Sandstorm apps that would stream multi-gigabyte files by doing a zillion writes without waiting, which would then all be queued in the HTTP gateway, causing it to run out of memory. In implementing that, I inadertently caused the RPC system to do a tree walk on every call message it received, in order to sum up the message size. This is silly, becaues it's much cheaper to sum up the segment sizes. In fact, in the case of a malicious peer, the tree walk is potentially insufficient, because it doesn't count holes in the segments. The tree walk also means that any invalid pointers in the message cause an exception to be thrown even if that pointer is never accessed by the app, which isn't the usual behavior. I seem to recall this issue coming up in discussion once in the past, but I couldn't find the thread. For the new streaming feature, we'll be paying attention to the size of outgoing messages. Again, here, it would be nice to compute this size by summing segments without doing a tree walk. So, this commit adds `sizeInWords()` methods that do this.
-
- 15 Jun, 2019 3 commits
-
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
This is frequently needed. Sandstorm had `OwnCapnp` for this purpose: https://github.com/sandstorm-io/sandstorm/blob/4d86a8144cdb43120ea12845738d0fe4a6ffcda1/src/sandstorm/util.h#L495-L525 The Workers codebase has some ad-hoc copies of this logic too, and multiple people have requested something similar on the mailing list.
-
- 30 May, 2019 1 commit
-
-
Kenton Varda authored
-
- 26 Apr, 2019 1 commit
-
-
Kenton Varda authored
Fix typos spotted by @a-robinson and update test failure messages to indicate that either testing failing could be due to OS SCM_RIGHTS truncation bug.
-
- 22 Apr, 2019 3 commits
-
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-
- 19 Apr, 2019 2 commits
-
-
Kenton Varda authored
-
Kenton Varda authored
An endpoint (subclass of `Capability::Server`) may override `kj::Maybe<int> getFd()` to expose an underlying file descriptor. A remote client may use `Capability::Client::getFd()` on the endpoint's capability to get that FD. The client and server must explicitly opt into FD passing by passing a max-FDs-per-message limit to TwoPartyVatNetwork and using Unix sockets as the transport. Nothing other than that is needed.
-
- 18 Apr, 2019 1 commit
-
-
Kenton Varda authored
This is just the protocol change, not implementation.
-
- 15 Apr, 2019 1 commit
-
-
AlexDenisov authored
-
- 10 Mar, 2019 1 commit
-
-
Thomas Arcila authored
On Windows executables needs to end with .exe. Fix this in CMakeLists.txt and update capnp detection code accordingly.
-
- 12 Feb, 2019 1 commit
-
-
0xflotus authored
-
- 23 Jan, 2019 1 commit
-
-
Max FERGER authored
-
- 07 Dec, 2018 1 commit
-
-
Simon Struk authored
-
- 26 Nov, 2018 1 commit
-
-
Navin Francis authored
-
- 17 Nov, 2018 1 commit
-
-
David Renshaw authored
-
- 26 Oct, 2018 1 commit
-
-
Tim authored
Forward slashes don't need to be escaped in JSON.
-
- 10 Oct, 2018 1 commit
-
-
Harris Hancock authored
-
- 07 Oct, 2018 2 commits
-
-
Kenton Varda authored
This isn't related to JSON-RPC but is a fix needed for the Ekam language server.
-
Kenton Varda authored
This implements JSON-RPC 2.0: https://www.jsonrpc.org/specification
-
- 16 Sep, 2018 1 commit
-
-
Kenton Varda authored
-
- 26 Aug, 2018 3 commits
-
-
Kenton Varda authored
-
Kenton Varda authored
-
Kenton Varda authored
-