# ADR-002: Cap'n Proto over MessagePack **Status:** Accepted --- ## Context quicproquo needs an efficient, typed wire format for client-server communication. The format must support: 1. **Typed messages** with compile-time schema enforcement to eliminate hand-rolled serialisation bugs. 2. **Schema evolution** so that new fields and methods can be added without breaking existing clients. 3. **RPC support** for clean method dispatch, eliminating the need for manual message-type routing. 4. **Efficient encoding** to minimize overhead on constrained networks and high-throughput server paths. 5. **Canonical serialisation** so that identical logical messages produce identical byte sequences, enabling reliable signing. The original M0 prototype used MessagePack (via the `rmp-serde` crate) with hand-rolled dispatch based on integer message-type tags. This approach had several problems: - **No schema enforcement.** The wire format was defined implicitly by Rust `#[derive(Serialize, Deserialize)]` annotations. There was no single source of truth for the wire format, and changes to Rust struct layout silently changed the wire format. - **No RPC.** Message dispatch was a manual `match` on a `MsgType` enum. Adding a new message type required modifying the dispatch table in both client and server, with no compile-time guarantee that all cases were handled. - **No canonical form.** MessagePack's map encoding does not guarantee key ordering, so the same logical message could produce different byte sequences depending on the Rust `HashMap` iteration order. This made signing over serialised data unreliable. - **Deserialization overhead.** MessagePack requires a full decode pass that allocates and copies data. For a messaging system processing many small messages, this overhead is unnecessary. ### Alternatives considered 1. **MessagePack (status quo).** Keep the existing format. Rejected because of the schema, dispatch, and canonicity problems described above. 2. **Protocol Buffers (Protobuf).** Schema-defined, binary, widely used. However: - Protobuf does not guarantee canonical serialisation (default value elision, field ordering, and unknown field handling can vary between implementations). - Protobuf RPC requires a separate framework (gRPC), which brings in HTTP/2 and a heavy runtime. - Protobuf deserialization requires an allocation and copy pass (not zero-copy). 3. **FlatBuffers.** Zero-copy, schema-defined. However: - No built-in RPC framework. - The Rust crate ecosystem was less mature than Cap'n Proto at the time of evaluation. - No canonical serialisation guarantee. 4. **Cap'n Proto.** Zero-copy, schema-defined, canonical serialisation, built-in async RPC. The `capnp` and `capnp-rpc` Rust crates are mature and actively maintained. --- ## Decision Replace MessagePack with **Cap'n Proto** for all wire-format serialisation and RPC dispatch. Define all message types and service interfaces in `.capnp` schema files, and use the `capnpc` compiler for Rust code generation. ### Key properties of Cap'n Proto **Zero-copy deserialization:** Cap'n Proto's wire format is designed so that the byte layout on the wire is identical to the byte layout in memory. A receiver can traverse the message in-place using pointer arithmetic, without allocating or copying data. For a messaging server that processes many small messages per second, this eliminates a significant class of allocation overhead. ```text Traditional serialisation: wire bytes -> decode -> allocate -> application struct Cap'n Proto: wire bytes == application struct (traverse in place) ``` **Schema enforcement:** All messages and RPC interfaces are defined in `.capnp` schema files checked into the repository. The `capnpc` compiler generates Rust code with type-safe builders and readers. A mismatched field type or missing field is caught at compile time, not at runtime. **Canonical serialisation:** Cap'n Proto defines a canonical form for messages: fields are laid out in a deterministic order with deterministic padding. Two implementations that build the same logical message produce identical byte sequences. This property is essential for signing: the MLS layer signs over serialised Cap'n Proto data, and non-deterministic serialisation would make signature verification unreliable. **Built-in async RPC:** The `capnp-rpc` crate provides a full RPC framework built on top of Cap'n Proto serialisation. Features include: - **Method dispatch:** Each interface method has a unique ordinal, and the RPC runtime dispatches incoming calls to the correct handler automatically. - **Promise pipelining:** A client can call a method on the result of a previous call before the first call has returned. The RPC runtime resolves the pipeline when the result is available. - **Cancellation:** An in-flight RPC call can be cancelled by the client, and the server is notified. - **Level 1 RPC:** The `capnp-rpc` crate implements Cap'n Proto's Level 1 RPC protocol, which supports most features needed for client-server communication. **Schema evolution:** Cap'n Proto supports forward-compatible schema evolution: - New fields can be added to structs (with the next available field number). Old readers ignore unknown fields. - New methods can be added to interfaces (with the next available ordinal). Old clients cannot call them; old servers reject unknown method calls. - Fields and methods can never be removed or renumbered, but they can be deprecated. - The `version` field in the `Auth` struct provides application-level versioning on top of structural evolution. ### Schema files The Cap'n Proto schemas are stored in the `schemas/` directory: | File | Content | Documentation | |---|---|---| | `schemas/envelope.capnp` | Legacy `Envelope` struct and `MsgType` enum | [Envelope Schema](../wire-format/envelope-schema.md) | | `schemas/auth.capnp` | `AuthenticationService` interface | [Auth Schema](../wire-format/auth-schema.md) | | `schemas/delivery.capnp` | `DeliveryService` interface | [Delivery Schema](../wire-format/delivery-schema.md) | | `schemas/node.capnp` | `NodeService` interface and `Auth` struct | [NodeService Schema](../wire-format/node-service-schema.md) | --- ## Consequences ### Benefits - **Eliminated hand-rolled dispatch.** The manual `MsgType` match table is replaced by Cap'n Proto RPC's automatic method dispatch. Adding a new operation means adding a method to the `.capnp` schema and implementing the handler -- no dispatch table to update. - **Compile-time type safety.** Schema violations are caught at compile time by the generated Rust code. A field type mismatch or missing required parameter is a compile error, not a runtime panic. - **Zero-copy performance.** The server avoids deserialization overhead for messages it routes but does not inspect (which is most messages, since the DS is MLS-unaware). The server can read the routing fields (recipient key, channel ID) directly from the wire bytes. - **Canonical form for signing.** MLS operations that sign over serialised data can rely on Cap'n Proto producing deterministic byte sequences. - **Schema as documentation.** The `.capnp` files serve as the authoritative specification of the wire format, readable by both humans and tools. ### Costs and trade-offs - **Build-time code generation.** The `capnpc` compiler must run during the build (via `build.rs` in `quicproquo-proto`). This adds a build dependency and increases compile times slightly. - **Learning curve.** Cap'n Proto's builder/reader API is different from typical `serde`-based Rust serialisation. Developers must learn the Cap'n Proto programming model (builders for construction, readers for traversal, owned messages for storage). - **Generated code verbosity.** The generated Rust code is verbose and not intended to be read directly. Application code interacts with it through the builder/reader traits. - **Smaller ecosystem than Protobuf.** Cap'n Proto has fewer users, fewer tutorials, and fewer third-party tools than Protobuf. However, the core Rust crates are well-maintained. - **No dynamic reflection.** Unlike Protobuf (which supports `Any` and `DynamicMessage`), Cap'n Proto does not provide runtime reflection over unknown schemas. This has not been a limitation in practice. ### Residual risks - **Crate maintenance.** The `capnp` and `capnp-rpc` crates are maintained primarily by David Renshaw. If maintenance lapses, the project would need to fork or switch serialisation formats. Mitigated by the crates' maturity and the relatively stable Cap'n Proto specification. - **RPC limitations.** The Rust `capnp-rpc` crate implements Level 1 of the Cap'n Proto RPC protocol. Level 3 features (three-party handoffs) are not supported. This has not been a limitation for quicproquo's client-server architecture. --- ## Code references | File | Relevance | |---|---| | `schemas/envelope.capnp` | Legacy Envelope struct definition | | `schemas/auth.capnp` | AuthenticationService RPC interface | | `schemas/delivery.capnp` | DeliveryService RPC interface | | `schemas/node.capnp` | NodeService unified RPC interface | | `crates/quicproquo-proto/build.rs` | Build script that invokes `capnpc` for code generation | | `crates/quicproquo-proto/src/lib.rs` | Re-exports generated Cap'n Proto modules | --- ## Further reading - [Design Decisions Overview](overview.md) -- index of all ADRs - [Wire Format Overview](../wire-format/overview.md) -- how Cap'n Proto fits in the serialisation pipeline - [Why This Design, Not Signal/Matrix/...](why-not-signal.md) -- serialisation comparison against Protobuf and JSON - [Cap'n Proto encoding specification](https://capnproto.org/encoding.html) -- upstream specification