1. 16 Jun, 2019 1 commit
    • Kenton Varda's avatar
      Add cheaper way to check size of RPC messages for flow control. · 76e35a7c
      Kenton Varda authored
      Way back in 538a767e I added `RpcSystem::setFlowLimit()`, a blunt mechanism by which an RPC node can arrange to stop reading new messages from the connection when too many incoming calls are in-flight. This was needed to deal with buggy Sandstorm apps that would stream multi-gigabyte files by doing a zillion writes without waiting, which would then all be queued in the HTTP gateway, causing it to run out of memory.
      
      In implementing that, I inadertently caused the RPC system to do a tree walk on every call message it received, in order to sum up the message size. This is silly, becaues it's much cheaper to sum up the segment sizes. In fact, in the case of a malicious peer, the tree walk is potentially insufficient, because it doesn't count holes in the segments. The tree walk also means that any invalid pointers in the message cause an exception to be thrown even if that pointer is never accessed by the app, which isn't the usual behavior.
      
      I seem to recall this issue coming up in discussion once in the past, but I couldn't find the thread.
      
      For the new streaming feature, we'll be paying attention to the size of outgoing messages. Again, here, it would be nice to compute this size by summing segments without doing a tree walk.
      
      So, this commit adds `sizeInWords()` methods that do this.
      76e35a7c
  2. 05 Aug, 2018 1 commit
  3. 16 Apr, 2017 1 commit
  4. 30 Mar, 2017 1 commit
  5. 24 Mar, 2017 1 commit
  6. 03 Jul, 2015 1 commit
    • Kenton Varda's avatar
      Refactor how messages are imbued with a capability table. · 5413038b
      Kenton Varda authored
      **The problem**
      
      The methods MessageReader::initCapTable() and MessageBuilder::getCapTable() always felt rather hacky. initCapTable() in particular feels like something that should be handled by the constructor. However, in practice, the cap table is often initialized based on a table encoded within the message itself. That is, an RPC message contains a "payload" which includes both the application-level message structure and a table of capabilities. The cap table has to be processed first, then initCapTable() is called on the overall message, before the application structure can safely be read.
      
      The really weird part about this is that even though the cap table only applies to one branch of the message (the payload), it is set on the *whole* MessageReader. This implies, for example, that it would be impossible to have a message that contains multiple payloads. We haven't had any need for such a thing, but an implemnetation that has such artificial limitations feels very wrong.
      
      MessageBuilder has similar issues going in the opposite direction.
      
      All of this ugliness potentially gets worse when we introduce "membranes". We want a way to intercept capabilities as they are being read from or written to an RPC payload. Currently, the only plausible way to do that is, again, to apply a transformation to all capabilities in the message. In practice it seems like this would work out OK, but it again feels wrong -- we really want to take a single Reader or Builder and "wrap" it so that transformations are applied on capabilities read/written through it.
      
      **The solution**
      
      This change fixes the problem by adding a new pointer to each struct/list Reader/Builder that tracks the current cap table. So, now a Reader or Builder for a particular sub-object can be "imbued" with a cap table without affecting any other existing Readers/Builders pointing into the same message. The cap table is inherited by child Readers/Builders obtained through the original one.
      
      This approach matches up nicely with membranes, which should make their implementation nice and clean.
      
      This change unfortunately means that Readers and Builders are now bigger, possibly with some performance impact.
      5413038b
  7. 10 Dec, 2014 1 commit
  8. 26 Oct, 2014 1 commit
    • Kenton Varda's avatar
      Implement "lite mode", where reflection is disabled. · c772a700
      Kenton Varda authored
      To use, pass --disable-reflection to the configure script.
      
      This produces a smaller runtime library. However, using it for this purpose is not recommended. The main purpose of lite mode is to define a subset of Cap'n Proto which might plausibly compile under MSVC. MSVC still lacks full support for constexpr and expression SFINAE; luckily, most of our use of these things relates to reflection, and not all users need reflection.
      
      Cap'n Proto lite mode inherits its name from Protocol Buffers' lite mode. However, there are some key differences:
      
      - Protobuf generated code included global constructors related to registering descriptors and extensions. For many people, this was the main reason to use lite mode: to get rid of these global constructors and achieve faster startup times. Cap'n Proto, on the other hand, never had global constructors in the first place.
      
      - Schemas are actually still available in lite mode, though only in their raw (Cap'n Proto structure) form. Only the schema API (which wraps the raw schemas in a more convenient interface) and reflection API (which offers a convenient way to use the schemas) are unavailable.
      
      - Lite mode is enabled in an application by defining CAPNP_LITE rather than by specifying an annotation in the schema file. This better-reflects real-world usage patterns, where you typically want to enable lite mode application-wide anyway.
      
      - We do not build the lite mode library by default. You must request it by passing --disable-reflection to the configure script. Before you can do that, you must have a prebuilt Cap'n Proto compiler binary available, since the compiler can't be built without reflection.
      
      - Relatedly, the lite mode library is built with the same name as the full library. This library is not intended to be installed. If anything it should be statically linked. But, mostly the option only exists on non-MSVC platform to give us a way to test that we haven't broken lite mode.
      c772a700
  9. 20 Jun, 2014 1 commit
    • Kenton Varda's avatar
      Change license to MIT. · 889204fe
      Kenton Varda authored
      For portions currently copyright by Kenton (most of it), transfer copyright to Sandstorm Development Group, Inc. (Kenton's company).
      
      The license change is practically meaningless, as MIT and BSD 2-clause are legally equivalent. However, the BSD 2-clause license is sometimes confused for its ugly siblings, BSD 3-clause and BSD 4-clause. The MIT license is more immediately recognizeable for what it is.
      
      Rémy Blank and Jason Choy (the two non-trivial contributors) are on record as approving this change:
      
      https://groups.google.com/d/msg/capnproto/xXDd2HUOCcc/gbe_COIuXKYJ
      889204fe
  10. 12 Mar, 2014 1 commit
  11. 11 Dec, 2013 1 commit
    • Kenton Varda's avatar
      Eliminate the concept of imbuing messages in favor of the simpler concept of… · 3c7efbb4
      Kenton Varda authored
      Eliminate the concept of imbuing messages in favor of the simpler concept of setting a cap table directly on MessageReader / getting one from MessageBuilder.  This eliminates capability-context entirely.  This was made possible by the earlier change which moved capabilities to a separate table rather than storing CapDescriptors inline, but I didn't realize it at the time.
      3c7efbb4
  12. 10 Dec, 2013 1 commit
  13. 06 Dec, 2013 1 commit
  14. 30 Nov, 2013 2 commits
  15. 28 Nov, 2013 1 commit
    • Kenton Varda's avatar
      Revamp concurrency model, part 1: EventLoop no longer allows cross-thread event… · 7921c854
      Kenton Varda authored
      Revamp concurrency model, part 1:  EventLoop no longer allows cross-thread event queuing, simplifying many things.  Capability clients are no longer thread-safe, so they don't have to be so const.  In the future, explicit ways to communicate between threads will be re-added, but threads will be treated more like separate vats that just happen to have a particularly fat pipe.  Upcoming:  Remove mutexes.
      7921c854
  16. 12 Nov, 2013 2 commits
  17. 31 Oct, 2013 1 commit
  18. 22 Oct, 2013 1 commit
  19. 15 Oct, 2013 2 commits
  20. 12 Oct, 2013 1 commit
  21. 11 Oct, 2013 2 commits
  22. 08 Oct, 2013 1 commit
  23. 04 Oct, 2013 1 commit
  24. 02 Oct, 2013 1 commit
  25. 29 Sep, 2013 1 commit
  26. 25 Sep, 2013 1 commit
  27. 28 Aug, 2013 1 commit
  28. 24 Jul, 2013 1 commit
  29. 22 Jul, 2013 1 commit
  30. 07 Jun, 2013 1 commit
  31. 06 Jun, 2013 5 commits
  32. 04 Jun, 2013 1 commit