c/quicproquo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite mdBook documentation for v2 architecture

Browse Source 
Update 25+ files and add 6 new pages to reflect the v2 migration from
Cap'n Proto to Protobuf framing over QUIC. Integrates SDK and Operations
docs into the mdBook, restructures SUMMARY.md, and rewrites the wire
format, architecture, and protocol sections with accurate v2 content.
This commit is contained in:
Christian Nennemann
2026-03-04 22:02:31 +01:00
parent f7a7f672b4
commit d073f614b3
31 changed files with 4423 additions and 2379 deletions
Download Patch File Download Diff File Expand all files Collapse all files

521
docs/src/architecture/service-architecture.md
View File

@@ -1,61 +1,233 @@
# Service Architecture
The quicproquo server exposes a single **NodeService** RPC endpoint that
combines Authentication and Delivery operations. This page documents the RPC
interface, per-connection lifecycle, storage model, long-polling mechanism, and
authentication context.
The quicproquo server exposes 44 RPC methods through a single QUIC + TLS 1.3
endpoint on **port 5001**. Methods are dispatched by numeric method ID using
the v2 Protobuf framing protocol. This page documents the method reference,
connection lifecycle, storage model, and authentication flow.
---
## NodeService Endpoint
## RPC Endpoint
A single QUIC + TLS 1.3 listener on **port 7000** serves all operations.
The schema is defined in `schemas/node.capnp` and documented in
[NodeService Schema](../wire-format/node-service-schema.md).
A single QUIC + TLS 1.3 listener on **port 5001** serves all operations.
The ALPN identifier is `qpq`. Each RPC call uses a dedicated QUIC
bidirectional stream; calls are concurrent and do not block each other.
```text
NodeService (port 7000)
├── Authentication methods
│ ├── uploadKeyPackage(identityKey, package, auth) -> fingerprint
│ ├── fetchKeyPackage(identityKey, auth) -> package
│ ├── uploadHybridKey(identityKey, hybridPublicKey) -> ()
│ └── fetchHybridKey(identityKey) -> hybridPublicKey
│
├── Delivery methods
│ ├── enqueue(recipientKey, payload, channelId, version, auth) -> ()
│ ├── fetch(recipientKey, channelId, version, auth) -> payloads
│ └── fetchWait(recipientKey, channelId, version, timeoutMs, auth) -> payloads
│
└── Operational
└── health() -> status
quicproquo-server (port 5001, ALPN: "qpq")
|
+-- Auth (100-103)
| +-- 100: OpaqueRegisterStart
| +-- 101: OpaqueRegisterFinish
| +-- 102: OpaqueLoginStart
| +-- 103: OpaqueLoginFinish
|
+-- Delivery (200-205)
| +-- 200: Enqueue
| +-- 201: Fetch
| +-- 202: FetchWait
| +-- 203: Peek
| +-- 204: Ack
| +-- 205: BatchEnqueue
|
+-- Keys (300-304)
| +-- 300: UploadKeyPackage
| +-- 301: FetchKeyPackage
| +-- 302: UploadHybridKey
| +-- 303: FetchHybridKey
| +-- 304: FetchHybridKeys
|
+-- Channel (400)
| +-- 400: CreateChannel
|
+-- Group Management (410-413)
| +-- 410: RemoveMember
| +-- 411: UpdateGroupMetadata
| +-- 412: ListGroupMembers
| +-- 413: RotateKeys
|
+-- Moderation (420-424)
| +-- 420: ReportMessage
| +-- 421: BanUser
| +-- 422: UnbanUser
| +-- 423: ListReports
| +-- 424: ListBanned
|
+-- User (500-501)
| +-- 500: ResolveUser
| +-- 501: ResolveIdentity
|
+-- Key Transparency (510-520)
| +-- 510: RevokeKey
| +-- 511: CheckRevocation
| +-- 520: AuditKeyTransparency
|
+-- Blob (600-601)
| +-- 600: UploadBlob
| +-- 601: DownloadBlob
|
+-- Device (700-710)
| +-- 700: RegisterDevice
| +-- 701: ListDevices
| +-- 702: RevokeDevice
| +-- 710: RegisterPushToken
|
+-- Recovery (750-752)
| +-- 750: StoreRecoveryBundle
| +-- 751: FetchRecoveryBundle
| +-- 752: DeleteRecoveryBundle
|
+-- P2P (800-802)
| +-- 800: PublishEndpoint
| +-- 801: ResolveEndpoint
| +-- 802: Health
|
+-- Federation (900-905)
| +-- 900: RelayEnqueue
| +-- 901: RelayBatchEnqueue
| +-- 902: ProxyFetchKeyPackage
| +-- 903: ProxyFetchHybridKey
| +-- 904: ProxyResolveUser
| +-- 905: FederationHealth
|
+-- Account (950)
+-- 950: DeleteAccount
Push event types (server -> client, uni-stream):
1000: PushNewMessage
1001: PushTyping
1002: PushPresence
1003: PushMembership
```
---
## RPC Method Reference
### Authentication Service Methods
### Auth (100-103)
| Method | Params | Returns | Semantics |
|----------------------|-------------------------------------|------------------|-----------|
| `uploadKeyPackage` | `identityKey` (32 B Ed25519 pk), `package` (TLS-encoded KeyPackage), `auth` | `fingerprint` (SHA-256 of package) | Appends the KeyPackage to a per-identity FIFO queue. The fingerprint lets the client detect server-side tampering. Max package size: 1 MB. |
| `fetchKeyPackage` | `identityKey` (32 B), `auth` | `package` (or empty `Data`) | Atomically pops and returns the oldest KeyPackage for the identity. Returns empty bytes if none are stored. Single-use semantics per RFC 9420. |
| `uploadHybridKey` | `identityKey` (32 B), `hybridPublicKey` (X25519 pk + ML-KEM-768 ek) | `()` | Stores (or replaces) the hybrid PQ public key for envelope-level post-quantum encryption. |
| `fetchHybridKey` | `identityKey` (32 B) | `hybridPublicKey` (or empty `Data`) | Returns the stored hybrid public key for a peer, or empty if none. |
OPAQUE password authentication (asymmetric PAKE). The password is never sent
to the server. Method IDs 100-103 implement the 4-step OPAQUE handshake.
### Delivery Service Methods
| ID | Method | Description |
|-----|-------------------------|-------------|
| 100 | `OpaqueRegisterStart` | Client initiates registration with `username` and OPAQUE `registration_request` blob. Server returns `registration_response`. |
| 101 | `OpaqueRegisterFinish` | Client completes registration with `username`, OPAQUE `upload` blob, and Ed25519 `identity_key`. Server stores the OPAQUE record. |
| 102 | `OpaqueLoginStart` | Client initiates login with `username` and OPAQUE `login_request` blob. Server returns `login_response`. |
| 103 | `OpaqueLoginFinish` | Client completes login with `username`, OPAQUE `finalization` blob, and `identity_key`. Server returns a `session_token`. |
| Method | Params | Returns | Semantics |
|--------------|------------------------------------------------------------------------|----------------------|-----------|
| `enqueue` | `recipientKey` (32 B), `payload` (opaque), `channelId`, `version`, `auth` | `()` | Appends `payload` to the recipient's FIFO queue. Max payload: 5 MB. Wakes any `fetchWait` waiter for this recipient. Supported versions: 0 (legacy), 1 (current). |
| `fetch` | `recipientKey` (32 B), `channelId`, `version`, `auth` | `payloads: List(Data)` | Atomically drains and returns the full queue in FIFO order. Returns empty list if nothing is pending. |
| `fetchWait` | `recipientKey` (32 B), `channelId`, `version`, `timeoutMs`, `auth` | `payloads: List(Data)` | Same as `fetch`, but if the queue is empty and `timeoutMs > 0`, blocks up to `timeoutMs` milliseconds waiting for a `Notify` signal from `enqueue`. Returns whatever is in the queue when the wait completes or times out. |
The `session_token` is an opaque bearer token used for subsequent authenticated
RPCs. It is passed in the Protobuf request body (not as a frame-level header).
### Operational Methods
### Delivery (200-205)
| Method | Params | Returns | Semantics |
|----------|--------|-----------------|-----------|
| `health` | none | `status: Text` | Returns `"ok"`. Used for liveness/readiness probes. |
Store-and-forward relay. The server never inspects MLS ciphertext -- it routes
opaque byte blobs by recipient key.
| ID | Method | Description |
|-----|----------------|-------------|
| 200 | `Enqueue` | Append an opaque payload to the recipient's FIFO queue. Wakes `FetchWait` waiters. |
| 201 | `Fetch` | Drain and return all queued payloads in FIFO order. |
| 202 | `FetchWait` | Same as `Fetch`, but long-polls if the queue is empty (up to `timeout_ms`). |
| 203 | `Peek` | Return queued payloads without removing them. |
| 204 | `Ack` | Acknowledge and remove specific payloads by sequence number. |
| 205 | `BatchEnqueue` | Enqueue multiple payloads in a single RPC call. |
### Keys (300-304)
MLS KeyPackage distribution and hybrid PQ public key management.
| ID | Method | Description |
|-----|--------------------|-------------|
| 300 | `UploadKeyPackage` | Append a TLS-encoded MLS KeyPackage to the identity's queue. Single-use: each fetch atomically removes one. |
| 301 | `FetchKeyPackage` | Atomically pop and return the oldest KeyPackage for an identity. Returns empty if none. |
| 302 | `UploadHybridKey` | Store (or replace) the X25519+ML-KEM-768 hybrid public key for an identity. |
| 303 | `FetchHybridKey` | Return the stored hybrid public key for a single identity. |
| 304 | `FetchHybridKeys` | Return hybrid public keys for multiple identities in one call. |
### Group Management (400, 410-413)
| ID | Method | Description |
|-----|-----------------------|-------------|
| 400 | `CreateChannel` | Register a new channel (group) on the server. |
| 410 | `RemoveMember` | Remove a member from a group (server-side record). |
| 411 | `UpdateGroupMetadata` | Update group name, description, or settings. |
| 412 | `ListGroupMembers` | List all members of a group. |
| 413 | `RotateKeys` | Trigger a server-assisted key rotation event. |
### User / Identity (500-501)
| ID | Method | Description |
|-----|-------------------|-------------|
| 500 | `ResolveUser` | Resolve a username to an Ed25519 public key. |
| 501 | `ResolveIdentity` | Resolve an identity key to user profile information. |
### Key Transparency (510-520)
| ID | Method | Description |
|-----|--------------------------|-------------|
| 510 | `RevokeKey` | Append a key revocation record to the transparency log. |
| 511 | `CheckRevocation` | Check whether a given key has been revoked. |
| 520 | `AuditKeyTransparency` | Fetch a transparency log audit proof for a key. |
### Blob Storage (600-601)
| ID | Method | Description |
|-----|----------------|-------------|
| 600 | `UploadBlob` | Store a binary blob (file attachment, avatar, etc.). Returns a content-addressed blob ID. |
| 601 | `DownloadBlob` | Retrieve a blob by ID. |
### Device Management (700-710)
| ID | Method | Description |
|-----|-----------------------|-------------|
| 700 | `RegisterDevice` | Register a new device for a user account. |
| 701 | `ListDevices` | List all registered devices for the authenticated user. |
| 702 | `RevokeDevice` | Revoke a device, invalidating its session. |
| 710 | `RegisterPushToken` | Register a push notification token (APNs / FCM) for a device. |
### Recovery (750-752)
| ID | Method | Description |
|-----|-------------------------|-------------|
| 750 | `StoreRecoveryBundle` | Encrypt and store an account recovery bundle server-side. |
| 751 | `FetchRecoveryBundle` | Retrieve the recovery bundle (requires OPAQUE re-authentication). |
| 752 | `DeleteRecoveryBundle` | Delete the stored recovery bundle. |
### P2P and Health (800-802)
| ID | Method | Description |
|-----|--------------------|-------------|
| 800 | `PublishEndpoint` | Publish a direct P2P endpoint (iroh node address). |
| 801 | `ResolveEndpoint` | Resolve a peer's P2P endpoint by identity key. |
| 802 | `Health` | Liveness/readiness probe. Returns server uptime and status. |
### Federation (900-905)
| ID | Method | Description |
|-----|-------------------------|-------------|
| 900 | `RelayEnqueue` | Relay a single message to a user on another server. |
| 901 | `RelayBatchEnqueue` | Relay multiple messages in one request. |
| 902 | `ProxyFetchKeyPackage` | Fetch a KeyPackage from a remote server on behalf of a local client. |
| 903 | `ProxyFetchHybridKey` | Fetch a hybrid public key from a remote server. |
| 904 | `ProxyResolveUser` | Resolve a username on a remote server. |
| 905 | `FederationHealth` | Check health of the federation link to another server. |
### Moderation (420-424)
| ID | Method | Description |
|-----|----------------|-------------|
| 420 | `ReportMessage` | Submit a content moderation report. |
| 421 | `BanUser` | Ban a user from a channel or server-wide. |
| 422 | `UnbanUser` | Lift a ban. |
| 423 | `ListReports` | List pending moderation reports (admin only). |
| 424 | `ListBanned` | List banned users (admin only). |
### Account (950)
| ID | Method | Description |
|-----|-----------------|-------------|
| 950 | `DeleteAccount` | Permanently delete the authenticated account and all associated data. |
---
@@ -64,196 +236,127 @@ NodeService (port 7000)
Each incoming QUIC connection follows this sequence:
```text
┌──────────────────────────────────────────────────────────────────────┐
│ Client Server │
│ │
│ 1. UDP packet -> │
│ QUIC INITIAL │
│ │
│ 2. <- QUIC HANDSHAKE │
│ TLS 1.3 ServerHello + │
│ Certificate (self-signed) │
│ ALPN: "capnp" │
│ │
│ 3. Client verifies server │
│ cert against pinned CA │
│ cert (--ca-cert flag) │
│ │
│ 4. QUIC connection established │
│ │
│ 5. Client opens bidirectional ──────────> Server accepts bi stream │
│ QUIC stream (open_bi) (accept_bi) │
│ │
│ 6. tokio_util::compat adapters wrap the send/recv halves │
│ into AsyncRead + AsyncWrite │
│ │
│ 7. capnp-rpc twoparty::VatNetwork │
│ Client Side::Client Server Side::Server │
│ │
│ 8. RpcSystem::new() starts │
│ promise-pipelined RPC loop │
│ │
│ 9. Client bootstraps │
│ node_service::Client NodeServiceImpl created │
│ (shares Arc<FileBackedStore>, │
│ Arc<DashMap<..., Notify>>) │
│ │
│ 10. RPC calls flow over the bidirectional stream │
│ until either side closes the connection. │
└──────────────────────────────────────────────────────────────────────┘
Client Server
------ ------
1. UDP QUIC INITIAL ->
2. <- QUIC HANDSHAKE
TLS 1.3 ServerHello +
Certificate (self-signed)
ALPN: "qpq"
3. Client verifies server cert against
pinned CA cert (--ca-cert flag)
4. QUIC connection established
5. Per RPC call:
Client opens bidirectional stream
Client writes RequestFrame:
[method_id: u16][req_id: u32][len: u32][protobuf]
Client marks end-of-write
6. Server reads RequestFrame
Server dispatches to handler by method_id
Handler processes, writes ResponseFrame:
[status: u8][req_id: u32][len: u32][protobuf]
7. For push events (server -> client):
Server opens uni-stream
Server writes PushFrame:
[event_type: u16][len: u32][protobuf]
8. Multiple RPCs run concurrently
(each on its own stream)
```
### LocalSet requirement
### Concurrency model
`capnp-rpc` uses `Rc<RefCell<>>` internally, making it `!Send`. Therefore:
- The server runs the entire accept loop inside a `tokio::task::LocalSet`.
- Each connection handler is `spawn_local`, ensuring all RPC futures stay on a
single thread.
- The client wraps each subcommand invocation in its own `LocalSet::run_until`.
This is a fundamental constraint of the Cap'n Proto RPC runtime in Rust.
Attempts to spawn RPC futures on the multi-threaded Tokio executor will fail
with a compile error.
Unlike the v1 Cap'n Proto RPC (which was `!Send` due to `Rc<RefCell<>>`
internals and required `LocalSet`), the v2 RPC framework uses `Arc`-based
shared state and spawns each handler with `tokio::spawn`. The server can
handle many concurrent requests per connection without a `LocalSet`.
---
## Storage Model
## Status Codes
`NodeServiceImpl` holds two pieces of shared state:
Response frames carry a `status: u8` field:
### FileBackedStore
| Value | Status | Meaning |
|-------|------------------|---------|
| 0 | `Ok` | Success |
| 1 | `BadRequest` | Malformed request or missing required field |
| 2 | `Unauthorized` | Missing or invalid session token |
| 3 | `Forbidden` | Valid token but insufficient permissions |
| 4 | `NotFound` | Requested resource does not exist |
| 5 | `RateLimited` | Request rate limit exceeded; retry after backoff |
| 8 | `DeadlineExceeded` | Request timed out on the server |
| 9 | `Unavailable` | Server temporarily unable to serve the request |
| 10 | `Internal` | Unexpected server error |
| 11 | `UnknownMethod` | The requested method_id is not registered |
---
## Authentication Flow
OPAQUE (RFC-compliant asymmetric PAKE) prevents the password from reaching
the server in any form:
```text
FileBackedStore
├── key_packages: Mutex<HashMap<Vec<u8>, VecDeque<Vec<u8>>>>
│ Key: Ed25519 public key (32 bytes)
│ Value: FIFO queue of TLS-encoded KeyPackage blobs
│ File: data/keypackages.bin (bincode)
│
├── deliveries: Mutex<HashMap<ChannelKey, VecDeque<Vec<u8>>>>
│ ChannelKey: { channel_id: Vec<u8>, recipient_key: Vec<u8> }
│ Value: FIFO queue of opaque payload blobs
│ File: data/deliveries.bin (bincode, v2 format)
│
└── hybrid_keys: Mutex<HashMap<Vec<u8>, Vec<u8>>>
Key: Ed25519 public key (32 bytes)
Value: serialised HybridPublicKey blob
File: data/hybridkeys.bin (bincode)
Client Server
| |
| OpaqueRegisterStart(100): |
| username, registration_request |
| --------------------------------->|
| |
| registration_response |
| <---------------------------------|
| |
| OpaqueRegisterFinish(101): |
| username, upload, identity_key |
| --------------------------------->|
| |
| success |
| <---------------------------------|
| |
| OpaqueLoginStart(102): |
| username, login_request |
| --------------------------------->|
| |
| login_response |
| <---------------------------------|
| |
| OpaqueLoginFinish(103): |
| username, finalization, |
| identity_key |
| --------------------------------->|
| |
| session_token |
| <---------------------------------|
```
Every mutation (upload, fetch, enqueue) acquires the relevant `Mutex`, modifies
the in-memory `HashMap`, and then flushes the entire map to disk as a bincode
blob. This is intentionally simple for MVP-scale workloads. A production
deployment would replace this with an embedded database or external store.
The delivery map supports a **v1 -> v2 upgrade path**: if `deliveries.bin`
contains the legacy `QueueMapV1` format (keyed by `recipientKey` only), the
store transparently upgrades entries by wrapping them in `ChannelKey` with an
empty `channel_id`.
### DashMap Waiters
```text
Arc<DashMap<Vec<u8>, Arc<Notify>>>
Key: recipient Ed25519 public key (32 bytes)
Value: tokio::sync::Notify instance
```
The waiters map is orthogonal to `FileBackedStore`. It lives entirely in
memory and serves the `fetchWait` long-polling mechanism:
1. `enqueue` calls `waiter(&recipient_key).notify_waiters()` after storing the
payload.
2. `fetchWait` first tries a regular `fetch`. If the queue is empty and
`timeoutMs > 0`:
- Look up or insert a `Notify` for the recipient.
- `tokio::time::timeout(Duration::from_millis(timeoutMs), notify.notified())`
- When notified (or on timeout), perform a second `fetch` and return
whatever is available.
This design avoids busy-polling while keeping the implementation lock-free
(DashMap uses sharded RwLocks internally).
---
## Auth Struct
Every RPC method that modifies or reads user-specific state accepts an `Auth`
parameter:
```capnp
struct Auth {
version @0 :UInt16; # 0 = legacy/none, 1 = token-based auth
accessToken @1 :Data; # opaque bearer token
deviceId @2 :Data; # optional UUID for auditing/rate limiting
}
```
### Version semantics
| Version | Meaning |
|---------|------------------------------------------------------------|
| 0 | Legacy / no authentication. The server accepts the request without checking credentials. Suitable for development and testing. |
| 1 | Token-based authentication. The `accessToken` field should contain an opaque bearer token issued at login. The server validates the token against a token store (not yet implemented -- see [Auth, Devices, and Tokens](../roadmap/authz-plan.md)). |
The server validates the `version` field on every request via `validate_auth()`.
Requests with unsupported versions are rejected with a Cap'n Proto error.
### Client-side usage
The client CLI accepts `--access-token` and `--device-id` flags (or the
corresponding environment variables). These are bundled into a `ClientAuth`
struct and injected into every outgoing RPC call via the `set_auth()` helper.
Currently, the client sends `version = 0` with empty token and device ID by
default. When the token-based auth flow is implemented, the client will populate
these fields.
---
## Validation and Limits
The server enforces the following constraints on every RPC call:
| Constraint | Value | Error on violation |
|-----------------------------|--------------------|--------------------|
| `identityKey` / `recipientKey` length | Exactly 32 bytes | Cap'n Proto error: "must be exactly 32 bytes" |
| KeyPackage size | <= 1 MB | Cap'n Proto error: "package exceeds max size" |
| Payload size | <= 5 MB | Cap'n Proto error: "payload exceeds max size" |
| Wire version | 0 or 1 | Cap'n Proto error: "unsupported wire version" |
| Auth version | 0 or 1 | Cap'n Proto error: "unsupported auth version" |
| KeyPackage non-empty | `package.len() > 0`| Cap'n Proto error: "package must not be empty" |
| Payload non-empty | `payload.len() > 0`| Cap'n Proto error: "payload must not be empty" |
The `session_token` is then passed in subsequent Protobuf requests. The server
validates it on every authenticated method call.
---
## Configuration
The server binary is configured via CLI flags or environment variables:
| Flag | Env var | Default | Description |
|----------------|----------------------------|------------------------|-------------|
| `--listen` | `QPQ_LISTEN` | `0.0.0.0:5001` | QUIC listen address (host:port). |
| `--data-dir` | `QPQ_DATA_DIR` | `data` | Directory for persisted state. |
| `--tls-cert` | `QPQ_TLS_CERT` | `data/server-cert.der` | Path to TLS certificate (DER). Auto-generated if missing. |
| `--tls-key` | `QPQ_TLS_KEY` | `data/server-key.der` | Path to TLS private key (DER). Auto-generated if missing. |
| Flag | Env var | Default | Description |
|----------------|----------------------------|----------------------|-------------|
| `--listen` | `QPQ_LISTEN` | `0.0.0.0:7000` | QUIC listen address (host:port). |
| `--data-dir` | `QPQ_DATA_DIR` | `data` | Directory for persisted KeyPackages, delivery queues, and hybrid keys. |
| `--tls-cert` | `QPQ_TLS_CERT` | `data/server-cert.der` | Path to TLS certificate (DER). Auto-generated if missing. |
| `--tls-key` | `QPQ_TLS_KEY` | `data/server-key.der` | Path to TLS private key (DER). Auto-generated if missing. |
If the TLS certificate or key files do not exist at startup, the server
auto-generates a self-signed certificate for `localhost`, `127.0.0.1`, and
`::1` using `rcgen`.
Logging level is controlled by the `RUST_LOG` environment variable (default:
`info`).
Logging level is controlled by the `RUST_LOG` environment variable (default: `info`).
---
## Further Reading
- [Architecture Overview](overview.md) -- two-service model and dual-key overview
- [NodeService Schema](../wire-format/node-service-schema.md) -- full Cap'n Proto schema
- [End-to-End Data Flow](data-flow.md) -- sequence diagrams showing registration, group creation, and messaging
- [Delivery Service Internals](../internals/delivery-service.md) -- queue routing and channel-aware delivery
- [Authentication Service Internals](../internals/authentication-service.md) -- KeyPackage lifecycle
- [Storage Backend](../internals/storage-backend.md) -- FileBackedStore details and upgrade path
- [Auth, Devices, and Tokens](../roadmap/authz-plan.md) -- planned token-based authentication
- [Architecture Overview](overview.md) -- two-service model and system diagram
- [End-to-End Data Flow](data-flow.md) -- sequence diagrams for registration, group creation, and messaging
- [Protobuf Framing](../protocol-layers/capn-proto.md) -- frame format details and method ID constants
- [Wire Format Reference](../wire-format/overview.md) -- full Protobuf schema documentation