quicproquo/docs/src/design-rationale/adr-002-capnproto.md

ADR-002: Cap'n Proto over MessagePack

Status: Accepted

---

Context

quicproquo needs an efficient, typed wire format for client-server communication. The format must support:

1. Typed messages with compile-time schema enforcement to eliminate hand-rolled serialisation bugs.
2. Schema evolution so that new fields and methods can be added without breaking existing clients.
3. RPC support for clean method dispatch, eliminating the need for manual message-type routing.
4. Efficient encoding to minimize overhead on constrained networks and high-throughput server paths.
5. Canonical serialisation so that identical logical messages produce identical byte sequences, enabling reliable signing.

The original M0 prototype used MessagePack (via the rmp-serde crate) with hand-rolled dispatch based on integer message-type tags. This approach had several problems:

• No schema enforcement. The wire format was defined implicitly by Rust #[derive(Serialize, Deserialize)] annotations. There was no single source of truth for the wire format, and changes to Rust struct layout silently changed the wire format.
• No RPC. Message dispatch was a manual match on a MsgType enum. Adding a new message type required modifying the dispatch table in both client and server, with no compile-time guarantee that all cases were handled.
• No canonical form. MessagePack's map encoding does not guarantee key ordering, so the same logical message could produce different byte sequences depending on the Rust HashMap iteration order. This made signing over serialised data unreliable.
• Deserialization overhead. MessagePack requires a full decode pass that allocates and copies data. For a messaging system processing many small messages, this overhead is unnecessary.

Alternatives considered

1. MessagePack (status quo). Keep the existing format. Rejected because of the schema, dispatch, and canonicity problems described above.

2. Protocol Buffers (Protobuf). Schema-defined, binary, widely used. However:

   • Protobuf does not guarantee canonical serialisation (default value elision, field ordering, and unknown field handling can vary between implementations).
   • Protobuf RPC requires a separate framework (gRPC), which brings in HTTP/2 and a heavy runtime.
   • Protobuf deserialization requires an allocation and copy pass (not zero-copy).

3. FlatBuffers. Zero-copy, schema-defined. However:

   • No built-in RPC framework.
   • The Rust crate ecosystem was less mature than Cap'n Proto at the time of evaluation.
   • No canonical serialisation guarantee.

4. Cap'n Proto. Zero-copy, schema-defined, canonical serialisation, built-in async RPC. The capnp and capnp-rpc Rust crates are mature and actively maintained.

---

Decision

Replace MessagePack with Cap'n Proto for all wire-format serialisation and RPC dispatch. Define all message types and service interfaces in .capnp schema files, and use the capnpc compiler for Rust code generation.

Key properties of Cap'n Proto

Zero-copy deserialization:

Cap'n Proto's wire format is designed so that the byte layout on the wire is identical to the byte layout in memory. A receiver can traverse the message in-place using pointer arithmetic, without allocating or copying data. For a messaging server that processes many small messages per second, this eliminates a significant class of allocation overhead.

Traditional serialisation:  wire bytes -> decode -> allocate -> application struct
Cap'n Proto:                wire bytes == application struct (traverse in place)

Schema enforcement:

All messages and RPC interfaces are defined in .capnp schema files checked into the repository. The capnpc compiler generates Rust code with type-safe builders and readers. A mismatched field type or missing field is caught at compile time, not at runtime.

Canonical serialisation:

Cap'n Proto defines a canonical form for messages: fields are laid out in a deterministic order with deterministic padding. Two implementations that build the same logical message produce identical byte sequences. This property is essential for signing: the MLS layer signs over serialised Cap'n Proto data, and non-deterministic serialisation would make signature verification unreliable.

Built-in async RPC:

The capnp-rpc crate provides a full RPC framework built on top of Cap'n Proto serialisation. Features include:

• Method dispatch: Each interface method has a unique ordinal, and the RPC runtime dispatches incoming calls to the correct handler automatically.
• Promise pipelining: A client can call a method on the result of a previous call before the first call has returned. The RPC runtime resolves the pipeline when the result is available.
• Cancellation: An in-flight RPC call can be cancelled by the client, and the server is notified.
• Level 1 RPC: The capnp-rpc crate implements Cap'n Proto's Level 1 RPC protocol, which supports most features needed for client-server communication.

Schema evolution:

Cap'n Proto supports forward-compatible schema evolution:

• New fields can be added to structs (with the next available field number). Old readers ignore unknown fields.
• New methods can be added to interfaces (with the next available ordinal). Old clients cannot call them; old servers reject unknown method calls.
• Fields and methods can never be removed or renumbered, but they can be deprecated.
• The version field in the Auth struct provides application-level versioning on top of structural evolution.

Schema files

The Cap'n Proto schemas are stored in the schemas/ directory:

File	Content	Documentation
schemas/envelope.capnp	Legacy Envelope struct and MsgType enum	Envelope Schema
schemas/auth.capnp	AuthenticationService interface	Auth Schema
schemas/delivery.capnp	DeliveryService interface	Delivery Schema
schemas/node.capnp	NodeService interface and Auth struct	NodeService Schema

---

Consequences

Benefits

• Eliminated hand-rolled dispatch. The manual MsgType match table is replaced by Cap'n Proto RPC's automatic method dispatch. Adding a new operation means adding a method to the .capnp schema and implementing the handler -- no dispatch table to update.
• Compile-time type safety. Schema violations are caught at compile time by the generated Rust code. A field type mismatch or missing required parameter is a compile error, not a runtime panic.
• Zero-copy performance. The server avoids deserialization overhead for messages it routes but does not inspect (which is most messages, since the DS is MLS-unaware). The server can read the routing fields (recipient key, channel ID) directly from the wire bytes.
• Canonical form for signing. MLS operations that sign over serialised data can rely on Cap'n Proto producing deterministic byte sequences.
• Schema as documentation. The .capnp files serve as the authoritative specification of the wire format, readable by both humans and tools.

Costs and trade-offs

• Build-time code generation. The capnpc compiler must run during the build (via build.rs in quicproquo-proto). This adds a build dependency and increases compile times slightly.
• Learning curve. Cap'n Proto's builder/reader API is different from typical serde-based Rust serialisation. Developers must learn the Cap'n Proto programming model (builders for construction, readers for traversal, owned messages for storage).
• Generated code verbosity. The generated Rust code is verbose and not intended to be read directly. Application code interacts with it through the builder/reader traits.
• Smaller ecosystem than Protobuf. Cap'n Proto has fewer users, fewer tutorials, and fewer third-party tools than Protobuf. However, the core Rust crates are well-maintained.
• No dynamic reflection. Unlike Protobuf (which supports Any and DynamicMessage), Cap'n Proto does not provide runtime reflection over unknown schemas. This has not been a limitation in practice.

Residual risks

• Crate maintenance. The capnp and capnp-rpc crates are maintained primarily by David Renshaw. If maintenance lapses, the project would need to fork or switch serialisation formats. Mitigated by the crates' maturity and the relatively stable Cap'n Proto specification.
• RPC limitations. The Rust capnp-rpc crate implements Level 1 of the Cap'n Proto RPC protocol. Level 3 features (three-party handoffs) are not supported. This has not been a limitation for quicproquo's client-server architecture.

---

Code references

File	Relevance
schemas/envelope.capnp	Legacy Envelope struct definition
schemas/auth.capnp	AuthenticationService RPC interface
schemas/delivery.capnp	DeliveryService RPC interface
schemas/node.capnp	NodeService unified RPC interface
crates/quicproquo-proto/build.rs	Build script that invokes capnpc for code generation
crates/quicproquo-proto/src/lib.rs	Re-exports generated Cap'n Proto modules

---

Further reading

• Design Decisions Overview -- index of all ADRs
• Wire Format Overview -- how Cap'n Proto fits in the serialisation pipeline
• Why This Design, Not Signal/Matrix/... -- serialisation comparison against Protobuf and JSON
• Cap'n Proto encoding specification -- upstream specification