c/quicproquo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite mdBook documentation for v2 architecture

Browse Source 
Update 25+ files and add 6 new pages to reflect the v2 migration from
Cap'n Proto to Protobuf framing over QUIC. Integrates SDK and Operations
docs into the mdBook, restructures SUMMARY.md, and rewrites the wire
format, architecture, and protocol sections with accurate v2 content.
This commit is contained in:
Christian Nennemann
2026-03-04 22:02:31 +01:00
parent f7a7f672b4
commit d073f614b3
31 changed files with 4423 additions and 2379 deletions
Download Patch File Download Diff File Expand all files Collapse all files

352
docs/src/architecture/crate-responsibilities.md
View File

@@ -1,9 +1,9 @@
# Crate Responsibilities
The quicproquo workspace contains six crates. The main four (proto, core,
The quicproquo workspace contains nine crates. The core four (proto, core,
server, client) follow strict layering rules; each owns one concern and depends
only on the crates below it. The workspace also includes **quicproquo-gui**
(Tauri desktop app) and **quicproquo-p2p** (P2P endpoint resolution). This
only on the crates below it. The workspace also includes dedicated crates for
the RPC framework, client SDK, key transparency, plugin API, and P2P. This
page documents what each crate provides, what it explicitly avoids, and how the
crates relate to one another.
@@ -12,33 +12,47 @@ crates relate to one another.
## Dependency Flow Diagram
```text
┌──────────────────────────┐
│ quicproquo-client │
│ (CLI, QUIC client, │
│ GroupMember orchestr.) │
└─────────┬───────┬────────┘
│ │
┌───────┘ └────────┐
▼ ▼
┌────────────────────────┐ ┌────────────────────────┐
│ quicproquo-core │ │ quicproquo-server │
│ (crypto, MLS, │ │ (QUIC listener, │
│ hybrid KEM) │ │ NodeService RPC, │
│ │ │ storage) │
└──────────┬─────────────┘ └─────────┬──────────────┘
│ │
│ ┌───────────────────┘
▼ ▼
┌────────────────────────┐
│ quicproquo-proto │
│ (Cap'n Proto schemas, │
│ codegen, helpers) │
└────────────────────────┘
+-------------------+ +-------------------+
| quicproquo-client | | quicproquo-sdk |
| (CLI/TUI binary) | | (QpqClient, store)|
+--------+----------+ +--------+----------+
| |
+----------+ +-----------+
| |
v v
+-----------+----------+
| quicproquo-rpc |
| (framing, server, |
| client, middleware) |
+--------+-------------+
|
+---------------+---------------+
| |
v v
+------------------------+ +-----------------------------+
| quicproquo-core | | quicproquo-server |
| (crypto, MLS, | | (RPC server + domain |
| hybrid KEM) | | services) |
+----------+-------------+ +-------------+---------------+
| |
| +-------------------+ |
+------>| quicproquo-proto |<--+
| (capnp legacy + |
| prost v2 types) |
+-------------------+
(separate, no shared deps)
+-------------------+ +-------------------+ +-------------------+
| quicproquo-kt | | quicproquo-p2p | | quicproquo- |
| (key transparency)| | (iroh P2P) | | plugin-api |
+-------------------+ +-------------------+ | (#![no_std] C-ABI)|
+-------------------+
```
**Arrows point from dependant to dependency.** The proto crate sits at the base
of the dependency graph. The core crate depends on proto for envelope
serialisation. The server and client crates both depend on core and proto.
of the dependency graph. The core crate depends on proto for legacy envelope
serialisation. The rpc crate provides the framing and dispatch layer used by
both the sdk and server.
---
@@ -54,48 +68,61 @@ dependency.
| `identity` | `IdentityKeypair` | Ed25519 signing keypair for MLS credentials. Seed stored as `Zeroizing<[u8; 32]>`. Implements `openmls_traits::Signer`. |
| `group` | `GroupMember` | MLS group state machine wrapping `openmls::MlsGroup`. Lifecycle: `new` -> `generate_key_package` -> `create_group` / `join_group` -> `send_message` / `receive_message`. |
| `keypackage` | `generate_key_package` | Standalone KeyPackage generation (returns TLS-encoded bytes + SHA-256 fingerprint). |
| `keystore` | `DiskKeyStore`, `StoreCrypto` | `OpenMlsKeyStore` implementation backed by an in-memory `HashMap` with optional bincode flush to disk. `StoreCrypto` couples `RustCrypto` + `DiskKeyStore` into an `OpenMlsCryptoProvider`. |
| `keystore` | `DiskKeyStore`, `StoreCrypto` | `OpenMlsKeyStore` implementation backed by an in-memory `HashMap` with bincode flush to disk. `StoreCrypto` couples `RustCrypto` + `DiskKeyStore` into an `OpenMlsCryptoProvider`. |
| `hybrid_kem` | `HybridKeypair`, `HybridPublicKey`, `hybrid_encrypt`, `hybrid_decrypt` | X25519 + ML-KEM-768 hybrid KEM. HKDF-SHA256 key derivation, ChaCha20-Poly1305 AEAD. Versioned envelope wire format. |
| `error` | `CoreError`, `MAX_PLAINTEXT_LEN` | Unified error types. `CoreError` covers Cap'n Proto, MLS, and hybrid KEM failures. |
| `error` | `CoreError`, `MAX_PLAINTEXT_LEN` | Unified error types covering MLS and hybrid KEM failures. |
### What this crate does NOT do
- No network I/O.
- No QUIC or TLS -- that is the server and client crates' concern.
- No async runtime setup (it uses Tokio types internally but does not spawn or
manage a runtime).
- No async runtime setup.
- No CLI parsing.
### Key dependencies
`ed25519-dalek`, `openmls`, `openmls_rust_crypto`,
`openmls_traits`, `tls_codec`, `ml-kem`, `x25519-dalek`, `chacha20poly1305`,
`hkdf`, `sha2`, `zeroize`, `capnp`, `quicproquo-proto`, `tokio`,
`serde`, `bincode`, `serde_json`, `thiserror`.
`hkdf`, `sha2`, `zeroize`, `quicproquo-proto`, `serde`, `bincode`, `thiserror`.
---
## quicproquo-proto
**Role:** Cap'n Proto schema definitions, compile-time code generation, and
pure-synchronous serialisation helpers. This crate is the single source of truth
for the wire format.
**Role:** Protocol type definitions for both v1 (legacy Cap'n Proto) and v2
(Protobuf/prost). This crate is the single source of truth for wire types and
method ID constants.
### Contents
| Item | Description |
|---------------------------|-------------|
| `schemas/envelope.capnp` | `Envelope` struct and `MsgType` enum -- top-level wire message. |
| `schemas/auth.capnp` | `AuthenticationService` interface -- `uploadKeyPackage`, `fetchKeyPackage`. |
| `schemas/delivery.capnp` | `DeliveryService` interface -- `enqueue`, `fetch`. |
| `schemas/node.capnp` | `NodeService` interface (unified AS+DS) -- all RPC methods plus `Auth` struct. |
| `build.rs` | Invokes `capnpc` to generate Rust types from the four `.capnp` files. |
| `lib.rs` | `pub mod envelope_capnp`, `auth_capnp`, `delivery_capnp`, `node_capnp` -- re-exports generated modules. |
| `MsgType` | Re-exported enum from `envelope_capnp::envelope::MsgType`. |
| `ParsedEnvelope` | Owned, `Send + 'static` representation of a decoded `Envelope`. All byte fields are eagerly copied out of the Cap'n Proto reader. |
| `build_envelope` | Serialise a `ParsedEnvelope` to unpacked Cap'n Proto wire bytes. |
| `parse_envelope` | Deserialise wire bytes into a `ParsedEnvelope`. |
| `to_bytes` / `from_bytes` | Low-level Cap'n Proto message <-> byte conversions. |
| Item | Description |
|-------------------------------|-------------|
| `schemas/*.capnp` | Legacy Cap'n Proto schemas (auth, delivery, node, federation). |
| `proto/qpq/v1/*.proto` | 14 Protobuf files defining all v2 message types. |
| `build.rs` | Invokes `capnpc` for legacy types and `prost-build` for v2 types. |
| `pub mod qpq::v1` | All Protobuf-generated types, included via `prost` `include!`. |
| `pub mod method_ids` | All 44 RPC method ID constants (u16) plus 4 push event type constants. |
| `auth_capnp`, `node_capnp`... | Re-exported legacy Cap'n Proto generated modules. |
### method_ids ranges
| Range | Category |
|---|---|
| 100-103 | Auth (OPAQUE register/login) |
| 200-205 | Delivery (enqueue, fetch, ack) |
| 300-304 | Keys (key packages, hybrid keys) |
| 400 | Channel creation |
| 410-413 | Group management |
| 420-424 | Moderation |
| 500-501 | User / identity resolution |
| 510-520 | Key transparency |
| 600-601 | Blob storage |
| 700-710 | Device management + push tokens |
| 750-752 | Recovery bundles |
| 800-802 | P2P endpoints + health |
| 900-905 | Federation relay |
| 950 | Account deletion |
| 1000-1003 | Push event types (server-to-client) |
### What this crate does NOT do
@@ -106,155 +133,178 @@ for the wire format.
### Key dependencies
`capnp` (runtime), `capnpc` (build-time only).
`capnp` (runtime), `capnpc` (build-time), `prost`, `prost-build` (build-time),
`bytes`.
---
## quicproquo-rpc
**Role:** v2 RPC framework. Implements the custom binary framing protocol,
server-side dispatch, client-side request/response handling, and Tower
middleware (rate limiting, timeouts, authentication).
### Components
| Component | Description |
|------------------|-------------|
| `framing` | `RequestFrame`, `ResponseFrame`, `PushFrame` encode/decode with big-endian headers. Max payload: 4 MiB. |
| `server` | `RpcServer` accepts QUIC connections, reads request frames, dispatches to registered handlers, writes response frames. |
| `client` | `RpcClient` opens per-RPC QUIC streams, writes request frames, reads response frames. |
| `middleware` | Tower `Service` wrappers: rate limiter, deadline/timeout, auth token injection. |
| `error` | `RpcError`, `RpcStatus` enum (Ok=0, BadRequest=1, Unauthorized=2, ... UnknownMethod=11). |
### Frame format (implemented here)
```
Request: [method_id: u16 BE][request_id: u32 BE][payload_len: u32 BE][protobuf]
Response: [status: u8][request_id: u32 BE][payload_len: u32 BE][protobuf]
Push: [event_type: u16 BE][payload_len: u32 BE][protobuf]
```
### What this crate does NOT do
- No domain logic -- handlers are registered by the server crate.
- No crypto operations.
### Key dependencies
`quinn`, `rustls`, `tokio`, `bytes`, `tower`, `prost`, `quicproquo-proto`,
`tracing`, `thiserror`.
---
## quicproquo-sdk
**Role:** High-level client SDK. `QpqClient` wraps the RPC client with
typed methods, an async event broadcast channel, and a `ConversationStore`
for local conversation state.
### Components
| Component | Description |
|---------------------|-------------|
| `QpqClient` | Authenticated client: `register`, `login`, `send_message`, `fetch_messages`, etc. |
| Event channel | `tokio::sync::broadcast` channel delivering `ClientEvent` variants (NewMessage, Typing, Presence, Membership). |
| `ConversationStore` | SQLCipher-backed local store for message history and group state. |
### What this crate does NOT do
- No raw frame handling -- delegates to `quicproquo-rpc`.
- No MLS group state management -- delegates to `quicproquo-core`.
### Key dependencies
`quicproquo-rpc`, `quicproquo-core`, `quicproquo-proto`, `tokio`, `rusqlite`,
`prost`, `tracing`, `thiserror`, `anyhow`.
---
## quicproquo-server
**Role:** Network-facing server binary. Accepts QUIC + TLS 1.3 connections,
dispatches Cap'n Proto RPC calls to `NodeServiceImpl`, and persists state to
disk via `FileBackedStore`.
dispatches 44 Protobuf RPC methods through registered handlers in `domain/`,
and persists state to SQLCipher.
### Components
| Component | Description |
|----------------------|-------------|
| `NodeServiceImpl` | Implements `node_service::Server` (Cap'n Proto generated trait). Handles all eight RPC methods: `uploadKeyPackage`, `fetchKeyPackage`, `enqueue`, `fetch`, `fetchWait`, `health`, `uploadHybridKey`, `fetchHybridKey`. |
| `FileBackedStore` | Mutex-guarded `HashMap`s for KeyPackages (keyed by Ed25519 public key), delivery queues (keyed by `ChannelKey = (channelId, recipientKey)`), and hybrid public keys. Each mutation flushes the full map to a bincode file on disk. |
| `DashMap` waiters | `DashMap<Vec<u8>, Arc<Notify>>` -- per-recipient `tokio::sync::Notify` instances for `fetchWait` long-polling. `enqueue` calls `notify_waiters()` after appending. |
| TLS config | Self-signed certificate auto-generated on first run (`rcgen`). TLS 1.3 only, ALPN `capnp`. |
| CLI (`clap`) | `--listen` (default `0.0.0.0:7000`), `--data-dir`, `--tls-cert`, `--tls-key`. |
| `v2_handlers/` | One handler module per method category (auth, delivery, keys, channel, group, user, kt, blob, device, p2p, federation, moderation, recovery, account). |
| `domain/` | Protocol-agnostic domain types and service logic (e.g., `AuthService`, `DeliveryService`, `KeyService`). |
| `ServerState` | Shared state: SQLCipher connection pool, DashMap waiters, OPAQUE server state. |
| TLS config | Self-signed certificate auto-generated on first run (`rcgen`). TLS 1.3 only, ALPN `qpq`. |
| CLI (`clap`) | `--listen` (default `0.0.0.0:5001`), `--data-dir`, `--tls-cert`, `--tls-key`. |
### Connection lifecycle
```text
QUIC accept
└─ TLS 1.3 handshake (self-signed cert, ALPN "capnp")
└─ accept_bi() -> bidirectional QUIC stream
└─ tokio_util::compat adapters (AsyncRead/AsyncWrite)
└─ capnp-rpc twoparty::VatNetwork (Side::Server)
└─ RpcSystem drives NodeServiceImpl
```
QUIC accept (ALPN: "qpq")
+- TLS 1.3 handshake (self-signed cert)
+- Per-stream: read RequestFrame -> dispatch to handler -> write ResponseFrame
+- Uni-stream (server -> client): write PushFrame for events
```
Because `capnp-rpc` uses `Rc<RefCell<>>` internally and is therefore `!Send`,
the entire RPC stack runs on a `tokio::task::LocalSet`. Each incoming connection
is handled by `spawn_local`.
Each RPC call gets its own QUIC bidirectional stream; handlers run concurrently
via `tokio::spawn`.
### What this crate does NOT do
- No direct crypto operations (it delegates to `quicproquo-core` types
for fingerprinting and storage only).
- No MLS processing -- all payloads are opaque byte strings.
- No direct crypto beyond OPAQUE server-side operations.
- No MLS processing -- all MLS payloads are opaque byte strings.
### Key dependencies
`quicproquo-core`, `quicproquo-proto`, `quinn`, `quinn-proto`,
`rustls`, `rcgen`, `capnp`, `capnp-rpc`, `tokio`, `tokio-util`, `dashmap`,
`sha2`, `clap`, `tracing`, `anyhow`, `thiserror`, `bincode`, `serde`.
`quicproquo-core`, `quicproquo-proto`, `quicproquo-rpc`, `quinn`, `rustls`,
`rcgen`, `tokio`, `dashmap`, `rusqlite`, `prost`, `clap`, `tracing`, `anyhow`,
`thiserror`.
---
## quicproquo-client
**Role:** CLI client binary. Connects to the server over QUIC + TLS 1.3,
orchestrates MLS group operations via `GroupMember`, and persists identity and
group state to disk.
### Components
| Component | Description |
|-------------------------|-------------|
| `connect_node` | Establishes a QUIC/TLS connection, opens a bidirectional stream, and bootstraps a `capnp-rpc` `RpcSystem` to obtain a `node_service::Client`. |
| CLI subcommands (`clap`)| `ping`, `register`, `fetch-key`, `demo-group`, `register-state`, `create-group`, `invite`, `join`, `send`, `recv`. |
| `GroupMember` usage | The client creates a `GroupMember` (from `quicproquo-core`), calls `generate_key_package` / `create_group` / `add_member` / `join_group` / `send_message` / `receive_message`. |
| State persistence | `StoredState` holds `identity_seed` (32 bytes) and optional serialised `MlsGroup`. A companion `.ks` file stores the `DiskKeyStore` with HPKE init private keys. |
| Auth context | `ClientAuth` bundles an optional bearer token and device ID. Passed to every RPC via the `Auth` struct in `node.capnp`. |
### CLI subcommand summary
| Subcommand | What it does |
|-------------------|--------------|
| `ping` | Call `health()` and print RTT. |
| `register` | Generate a fresh identity + KeyPackage, upload to AS, print identity key. |
| `register-state` | Same as `register` but uses/creates persistent state file. |
| `fetch-key` | Fetch a peer's KeyPackage by hex identity key. |
| `create-group` | Create a new MLS group and save state. |
| `invite` | Fetch peer's KeyPackage, add to group, enqueue Welcome via DS. |
| `join` | Fetch Welcome from DS, join the MLS group. |
| `send` | Encrypt a message with MLS, enqueue via DS. |
| `recv` | Fetch pending payloads from DS, decrypt with MLS. Supports `--stream` for continuous long-polling. |
| `demo-group` | End-to-end Alice+Bob round-trip (ephemeral identities). |
**Role:** CLI/TUI client binary. Connects to the server, orchestrates MLS
group operations via `GroupMember`, and persists identity and group state.
### What this crate does NOT do
- No server-side logic.
- No direct crypto beyond calling `GroupMember` and verifying SHA-256
fingerprints.
- No raw frame parsing -- delegates to `quicproquo-sdk` / `quicproquo-rpc`.
### Key dependencies
`quicproquo-core`, `quicproquo-proto`, `quinn`, `quinn-proto`,
`rustls`, `capnp`, `capnp-rpc`, `tokio`, `tokio-util`, `clap`, `sha2`,
`serde`, `bincode`, `anyhow`, `thiserror`, `tracing`.
`quicproquo-sdk`, `quicproquo-core`, `quicproquo-proto`, `tokio`, `clap`,
`rustyline`, `tracing`, `anyhow`.
---
## quicproquo-bot
## quicproquo-kt
**Role:** High-level SDK for building automated agents (bots) on the
quicproquo network. Wraps the client library into a simple polling-based API.
**Role:** Key transparency. Implements an append-only transparency log for
Ed25519 public keys with revocation checking and audit support.
### Components
| Component | Description |
|------------------|-------------|
| `BotConfig` | Builder-pattern configuration: server address, credentials, TLS, state file path. |
| `Bot` | Connected bot instance. Methods: `connect()`, `send_dm()`, `receive()`, `receive_raw()`, `resolve_user()`. |
| `Message` | Received message struct with `sender`, `text`, and `seq` fields. |
| `run_pipe_mode` | JSON-lines stdin/stdout interface for shell integration (`send`, `recv`, `resolve` actions). |
### Architecture
Each `send_dm` and `receive` call opens a fresh QUIC connection (stateless
reconnect pattern). The bot wraps the client's `cmd_send` and
`receive_pending_plaintexts` functions, handling MLS group state internally.
### What this crate does NOT do
- No server-side logic.
- No raw MLS operations — delegates to `quicproquo-client` high-level functions.
- No persistent QUIC connections — each operation reconnects.
### Key dependencies
`quicproquo-core`, `quicproquo-client`, `tokio`, `anyhow`, `tracing`,
`serde`, `serde_json`, `hex`.
Methods exposed: `RevokeKey` (510), `CheckRevocation` (511),
`AuditKeyTransparency` (520).
---
## Other workspace crates
## quicproquo-plugin-api
| Crate | Role |
|-------------------------|------|
| **quicproquo-gui** | Tauri 2 desktop application; provides a GUI on top of the client/core stack. |
| **quicproquo-p2p** | P2P endpoint publish/resolve; used by the server and clients for direct peer discovery. |
**Role:** `#![no_std]` C-ABI plugin interface. Defines a stable ABI for
dynamically loaded plugins with 6 hook points (on_message_send,
on_message_receive, on_group_join, on_group_leave, on_connect, on_disconnect).
These crates are optional for building and running the server and CLI client.
This crate has no workspace dependencies. It is intentionally `no_std` to
allow plugins compiled for embedded or WASM targets.
---
## quicproquo-p2p
**Role:** P2P endpoint publish and resolve via iroh. Used by the server and
clients for direct peer discovery when the `mesh` feature is enabled on
`quicproquo-client`.
Methods exposed: `PublishEndpoint` (800), `ResolveEndpoint` (801).
This crate is compiled but kept out of the default dependency graph for most
build targets due to iroh's large dependency footprint (~90 extra deps).
---
## Layering Rules
1. **proto** depends on nothing in-workspace. It is pure data definition.
2. **core** depends on **proto** (for `ParsedEnvelope` and envelope helpers).
It does not depend on server or client.
3. **server** depends on **core** and **proto**. It does not depend on client.
4. **client** depends on **core** and **proto**. It does not depend on server.
5. **server** and **client** never depend on each other. They communicate
exclusively via the Cap'n Proto RPC wire protocol.
6. **quicproquo-gui** and **quicproquo-p2p** are optional; they depend
on client/core/proto as needed and do not change the core layering.
2. **core** depends on **proto** (for legacy envelope helpers).
It does not depend on server, rpc, or sdk.
3. **rpc** depends on **proto**. It does not depend on core, server, or client.
4. **sdk** depends on **rpc** and **core**. It does not depend on server.
5. **server** depends on **core**, **proto**, and **rpc**. It does not depend on client or sdk.
6. **client** depends on **sdk**, **core**, and **proto**. It does not depend on server.
7. **server** and **client** never depend on each other. They communicate
exclusively via the v2 Protobuf framing protocol over QUIC.
8. **kt**, **plugin-api**, and **p2p** are optional; they do not change the
core layering.
This layering ensures that:
@@ -268,7 +318,7 @@ This layering ensures that:
## Further Reading
- [Architecture Overview](overview.md) -- high-level system diagram
- [Service Architecture](service-architecture.md) -- NodeService RPC details
- [Wire Format Overview](../wire-format/overview.md) -- Cap'n Proto schema reference
- [Service Architecture](service-architecture.md) -- 44 RPC method details
- [Wire Format Reference](../wire-format/overview.md) -- Protobuf schema reference
- [GroupMember Lifecycle](../internals/group-member-lifecycle.md) -- MLS state machine details
- [Storage Backend](../internals/storage-backend.md) -- FileBackedStore internals
- [Storage Backend](../internals/storage-backend.md) -- SQLCipher storage internals