docs: rewrite mdBook documentation for v2 architecture
Update 25+ files and add 6 new pages to reflect the v2 migration from Cap'n Proto to Protobuf framing over QUIC. Integrates SDK and Operations docs into the mdBook, restructures SUMMARY.md, and rewrites the wire format, architecture, and protocol sections with accurate v2 content.
This commit is contained in:
@@ -10,9 +10,10 @@ These decisions are not immutable. Each ADR has a status field and can be supers
|
||||
|
||||
| ADR | Title | Status | One-line summary |
|
||||
|---|---|---|---|
|
||||
| [ADR-002](adr-002-capnproto.md) | Cap'n Proto over MessagePack | Accepted | Zero-copy, schema-enforced serialisation with built-in async RPC replaces hand-rolled MessagePack dispatch. |
|
||||
| [ADR-002](adr-002-capnproto.md) | Cap'n Proto over MessagePack (v1) | Superseded | Zero-copy, schema-enforced serialisation with built-in async RPC replaced hand-rolled MessagePack dispatch. Superseded by ADR-007. |
|
||||
| [ADR-004](adr-004-mls-unaware-ds.md) | MLS-Unaware Delivery Service | Accepted | The DS routes opaque blobs by recipient key; it never inspects MLS content. |
|
||||
| [ADR-005](adr-005-single-use-keypackages.md) | Single-Use KeyPackages | Accepted | The AS atomically removes a KeyPackage on fetch to preserve MLS forward secrecy. |
|
||||
| [ADR-007](adr-007-protobuf-migration.md) | v1 Cap'n Proto to v2 Protobuf Migration | Accepted | Replace Cap'n Proto RPC with custom Protobuf framing over QUIC streams for better ecosystem support, 44-method surface, and multi-threaded dispatch. |
|
||||
|
||||
---
|
||||
|
||||
@@ -26,7 +27,7 @@ For a broader comparison of quicproquo's design against alternative messaging pr
|
||||
|
||||
Each ADR page follows this structure:
|
||||
|
||||
1. **Status** -- One of: Proposed, Accepted, Deprecated, Superseded. All current ADRs are Accepted.
|
||||
1. **Status** -- One of: Proposed, Accepted, Deprecated, Superseded. All current ADRs are Accepted unless noted.
|
||||
2. **Context** -- The problem or force that motivated the decision. What constraints existed? What alternatives were considered?
|
||||
3. **Decision** -- The specific choice that was made. What was selected and what was rejected?
|
||||
4. **Consequences** -- The trade-offs that result from the decision. What are the benefits? What are the costs? What residual risks remain?
|
||||
@@ -34,13 +35,72 @@ Each ADR page follows this structure:
|
||||
|
||||
---
|
||||
|
||||
## Cross-cutting themes
|
||||
## ADR-007: v1 Cap'n Proto to v2 Protobuf Migration
|
||||
|
||||
Several themes recur across multiple ADRs:
|
||||
**Status**: Accepted
|
||||
|
||||
**Context**
|
||||
|
||||
quicproquo v1 used Cap'n Proto for both serialisation and RPC dispatch via
|
||||
`capnp-rpc`. This worked well for the initial 8-method `NodeService` interface
|
||||
but had several limitations as the protocol expanded:
|
||||
|
||||
- **`!Send` constraint**: `capnp-rpc` uses `Rc<RefCell<>>` internally, requiring
|
||||
all RPC futures to run on a `tokio::task::LocalSet`. This prevented multi-threaded
|
||||
dispatch and added complexity to every connection handler.
|
||||
- **Schema growth friction**: Cap'n Proto's capability-based RPC model does not
|
||||
map cleanly to large flat method tables. Adding the 36 new methods (keys,
|
||||
blob, device, federation, moderation, recovery, etc.) would have required
|
||||
significant schema refactoring.
|
||||
- **ALPN collision**: The `b"capnp"` ALPN identifier is not registered and could
|
||||
conflict with other Cap'n Proto deployments. A project-specific ALPN is cleaner.
|
||||
- **Tooling**: `capnpc` requires a system-wide binary installation or a vendored
|
||||
copy. `prost-build` with `protobuf-src` self-vendors `protoc`, eliminating the
|
||||
build-time dependency.
|
||||
|
||||
**Decision**
|
||||
|
||||
Replace `capnp-rpc` with a custom binary framing layer (`quicproquo-rpc`) and
|
||||
Protocol Buffers (`prost`) for payload serialisation:
|
||||
|
||||
- Three frame types: Request (10-byte header), Response (9-byte header), Push
|
||||
(6-byte header), all carrying Protobuf-encoded payloads.
|
||||
- Method IDs are numeric `u16` constants dispatched via a handler registry.
|
||||
One QUIC bidirectional stream per RPC call; push events on QUIC uni-streams.
|
||||
- ALPN changed from `b"capnp"` to `b"qpq"`. Default port changed from 7000 to 5001.
|
||||
- Cap'n Proto legacy types are retained in `quicproquo-proto` for v1 compatibility
|
||||
but are no longer used for RPC dispatch.
|
||||
|
||||
**Consequences**
|
||||
|
||||
Benefits:
|
||||
- Full `tokio::spawn` concurrency (no `LocalSet` required).
|
||||
- 44-method RPC surface with clean numeric namespace and room to grow.
|
||||
- Self-contained build (no system `protoc` dependency).
|
||||
- Lighter middleware integration via Tower `Service` traits.
|
||||
- Push event delivery without polling.
|
||||
|
||||
Costs:
|
||||
- Lost Cap'n Proto zero-copy reads (Protobuf requires deserialisation). Acceptable
|
||||
because the hot path in the Delivery Service works with opaque `bytes::Bytes`
|
||||
without deserialisation.
|
||||
- Lost promise pipelining from `capnp-rpc`. Not required for the current RPC
|
||||
surface; can be re-added with a future streaming RPC design.
|
||||
- v1 clients are no longer wire-compatible with v2 servers.
|
||||
|
||||
**Code references**
|
||||
|
||||
- Frame format: `crates/quicproquo-rpc/src/framing.rs`
|
||||
- Method IDs: `crates/quicproquo-proto/src/lib.rs` (`method_ids` module)
|
||||
- Proto schemas: `proto/qpq/v1/*.proto`
|
||||
|
||||
---
|
||||
|
||||
## Cross-cutting themes
|
||||
|
||||
### Layered security
|
||||
|
||||
The core principle is that **no single layer is trusted alone**. QUIC/TLS transport encryption protects metadata and provides authentication; MLS provides end-to-end content encryption with forward secrecy and post-compromise security.
|
||||
The core principle is that **no single layer is trusted alone**. QUIC/TLS transport encryption protects metadata and provides server authentication; MLS provides end-to-end content encryption with forward secrecy and post-compromise security; OPAQUE ensures the server never learns the user's password.
|
||||
|
||||
### Server minimalism
|
||||
|
||||
@@ -48,13 +108,16 @@ ADR-004 and ADR-005 reflect a design philosophy where the server does as little
|
||||
|
||||
### Schema-first design
|
||||
|
||||
ADR-002 establishes Cap'n Proto as the single source of truth for the wire format. Every message and RPC call is defined in `.capnp` schema files, which are checked into the repository and used for code generation. This eliminates the class of bugs that arises from hand-rolled serialisation and ensures that the wire format is documented, versioned, and evolvable.
|
||||
The v2 protocol defines all messages and method IDs in checked-in source files
|
||||
(`proto/qpq/v1/*.proto` and `crates/quicproquo-proto/src/lib.rs`). Every wire
|
||||
type is documented, versioned, and evolvable through the standard Protobuf
|
||||
schema evolution rules (adding optional fields, reserving removed field numbers).
|
||||
|
||||
---
|
||||
|
||||
## Further reading
|
||||
|
||||
- [Why This Design, Not Signal/Matrix/...](why-not-signal.md) -- comparative analysis against alternative protocols
|
||||
- [Wire Format Overview](../wire-format/overview.md) -- the serialisation pipeline that implements these decisions
|
||||
- [Wire Format Reference](../wire-format/overview.md) -- the serialisation pipeline that implements these decisions
|
||||
- [Architecture Overview](../architecture/overview.md) -- system-level view
|
||||
- [Protocol Layers Overview](../protocol-layers/overview.md) -- how the protocol layers stack
|
||||
|
||||
Reference in New Issue
Block a user