• Kenton Varda's avatar
    Implement byte streams over Cap'n Proto. · 63c34d47
    Kenton Varda authored
    This implementation features path-shortening through pumps. That is, if an incoming Cap'n Proto stream wraps a KJ stream which ends up pumping to an outgoing Cap'n Proto stream, the incoming stream will be redirected directly to the outgoing stream, in such a way that the RPC system can recognize and reduce the number of network hops as appropriate.
    
    This proved tricky due to the features of KJ's `pumpTo()`, in particular:
    - The caller of `pumpTo()` expects eventually to be told how many bytes were pumped (e.g. before EOF was hit).
    - A pump may have a specified length limit. In this case the rest of the stream can be pumped somewhere else.
    - Multiple streams can be pumped to the same destination stream -- this implies that a pump does not propagate EOF.
    
    These requirements mean that path-shortening is not as simple as redirecting the incoming stream to the outgoing. Intsead, we must first ask the outgoing stream's server to create a "substream" object, and then redirect to that. The substream can have a length limit, swallows EOF, informs the original creator on completion, and can even redirect *back* to the original creator to allow the stream to now pump somewhere else.
    63c34d47
Name
Last commit
Last update
..
compat Loading commit data...
compiler Loading commit data...
testdata Loading commit data...
CMakeLists.txt Loading commit data...
afl-testcase.c++ Loading commit data...
any-test.c++ Loading commit data...
any.c++ Loading commit data...
any.h Loading commit data...
arena.c++ Loading commit data...
arena.h Loading commit data...
blob-test.c++ Loading commit data...
blob.c++ Loading commit data...
blob.h Loading commit data...
bootstrap-test.ekam-rule Loading commit data...
c++.capnp Loading commit data...
c++.capnp.c++ Loading commit data...
c++.capnp.h Loading commit data...
canonicalize-test.c++ Loading commit data...
capability-test.c++ Loading commit data...
capability.c++ Loading commit data...
capability.h Loading commit data...
capnpc.ekam-rule Loading commit data...
common-test.c++ Loading commit data...
common.h Loading commit data...
dynamic-capability.c++ Loading commit data...
dynamic-test.c++ Loading commit data...
dynamic.c++ Loading commit data...
dynamic.h Loading commit data...
encoding-test.c++ Loading commit data...
endian-fallback-test.c++ Loading commit data...
endian-reverse-test.c++ Loading commit data...
endian-test.c++ Loading commit data...
endian.h Loading commit data...
ez-rpc-test.c++ Loading commit data...
ez-rpc.c++ Loading commit data...
ez-rpc.h Loading commit data...
fuzz-test.c++ Loading commit data...
generated-header-support.h Loading commit data...
layout-test.c++ Loading commit data...
layout.c++ Loading commit data...
layout.h Loading commit data...
list.c++ Loading commit data...
list.h Loading commit data...
membrane-test.c++ Loading commit data...
membrane.c++ Loading commit data...
membrane.h Loading commit data...
message-test.c++ Loading commit data...
message.c++ Loading commit data...
message.h Loading commit data...
orphan-test.c++ Loading commit data...
orphan.h Loading commit data...
persistent.capnp Loading commit data...
persistent.capnp.c++ Loading commit data...
persistent.capnp.h Loading commit data...
pointer-helpers.h Loading commit data...
pretty-print.h Loading commit data...
raw-schema.h Loading commit data...
rpc-prelude.h Loading commit data...
rpc-test.c++ Loading commit data...
rpc-twoparty-test.c++ Loading commit data...
rpc-twoparty.c++ Loading commit data...
rpc-twoparty.capnp Loading commit data...
rpc-twoparty.capnp.c++ Loading commit data...
rpc-twoparty.capnp.h Loading commit data...
rpc-twoparty.h Loading commit data...
rpc.c++ Loading commit data...
rpc.capnp Loading commit data...
rpc.capnp.c++ Loading commit data...
rpc.capnp.h Loading commit data...
rpc.h Loading commit data...
schema-lite.h Loading commit data...
schema-loader-test.c++ Loading commit data...
schema-loader.c++ Loading commit data...
schema-loader.h Loading commit data...
schema-parser-test.c++ Loading commit data...
schema-parser.c++ Loading commit data...
schema-parser.h Loading commit data...
schema-test.c++ Loading commit data...
schema.c++ Loading commit data...
schema.capnp Loading commit data...
schema.capnp.c++ Loading commit data...
schema.capnp.h Loading commit data...
schema.h Loading commit data...
serialize-async-test.c++ Loading commit data...
serialize-async.c++ Loading commit data...
serialize-async.h Loading commit data...
serialize-packed-test.c++ Loading commit data...
serialize-packed.c++ Loading commit data...
serialize-packed.h Loading commit data...
serialize-test.c++ Loading commit data...
serialize-text-test.c++ Loading commit data...
serialize-text.c++ Loading commit data...
serialize-text.h Loading commit data...
serialize.c++ Loading commit data...
serialize.h Loading commit data...
stream.capnp Loading commit data...
stream.capnp.c++ Loading commit data...
stream.capnp.h Loading commit data...
stringify-test.c++ Loading commit data...
stringify.c++ Loading commit data...
test-import.capnp Loading commit data...
test-import2.capnp Loading commit data...
test-util.c++ Loading commit data...
test-util.h Loading commit data...
test.capnp Loading commit data...