c/quicproquo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite mdBook documentation for v2 architecture

Browse Source 
Update 25+ files and add 6 new pages to reflect the v2 migration from
Cap'n Proto to Protobuf framing over QUIC. Integrates SDK and Operations
docs into the mdBook, restructures SUMMARY.md, and rewrites the wire
format, architecture, and protocol sections with accurate v2 content.
This commit is contained in:
Christian Nennemann
2026-03-04 22:02:31 +01:00
parent f7a7f672b4
commit d073f614b3
31 changed files with 4423 additions and 2379 deletions
Download Patch File Download Diff File Expand all files Collapse all files

268
docs/src/wire-format/auth-schema.md
View File

@@ -1,149 +1,215 @@
# Auth Schema
**Schema file:** `schemas/auth.capnp`
**File ID:** `@0xb3a8f1c2e4d97650`
**Proto file:** `proto/qpq/v1/auth.proto`
**Package:** `qpq.v1`
**Method IDs:** 100-103
The `AuthenticationService` interface defines the RPC contract for uploading and fetching MLS KeyPackages. It is the standalone version of the Authentication Service; in the current architecture, these methods are integrated into the unified [NodeService](node-service-schema.md) interface.
The auth proto defines the OPAQUE asymmetric password-authenticated key exchange (PAKE) messages used for user registration and login. OPAQUE never transmits the password to the server; the server learns only a random value derived from the password.
Registration is a two-round-trip flow (start + finish). Login is a two-round-trip flow (start + finish). On successful login, the server returns a `session_token` used to authenticate subsequent RPCs.
See [Authentication Service Internals](../internals/authentication-service.md) for the server-side implementation and the full flow diagram.
---
## Full schema listing
## Full proto listing
```capnp
# auth.capnp -- Authentication Service RPC interface.
#
# Clients call uploadKeyPackage before joining any group so that peers can
# fetch their key material to add them. Each KeyPackage is single-use (MLS
# requirement): fetchKeyPackage removes and returns one package atomically.
#
# The server indexes packages by the raw Ed25519 public key bytes (32 bytes),
# not a fingerprint, so callers must know the target's identity public key
# out-of-band (e.g. from a directory or QR code scan).
#
# ID generated with: capnp id
@0xb3a8f1c2e4d97650;
```protobuf
syntax = "proto3";
package qpq.v1;
interface AuthenticationService {
# Upload a single-use KeyPackage for later retrieval by peers.
#
# identityKey : Ed25519 public key bytes (exactly 32 bytes).
# package : openmls-serialised KeyPackage blob (TLS encoding).
#
# Returns the SHA-256 fingerprint of `package`. Clients should record this
# and compare it against the fingerprint returned by a peer's fetchKeyPackage
# to detect tampering.
uploadKeyPackage @0 (identityKey :Data, package :Data) -> (fingerprint :Data);
// OPAQUE registration + login (4 methods).
// Method IDs: 100-103.
# Fetch and atomically remove one KeyPackage for a given identity key.
#
# Returns empty Data if no KeyPackage is currently stored for this identity.
# Callers should handle the empty case by asking the target to upload more
# packages before retrying.
fetchKeyPackage @1 (identityKey :Data) -> (package :Data);
message OpaqueRegisterStartRequest {
string username = 1;
bytes request = 2;
}
message OpaqueRegisterStartResponse {
bytes response = 1;
}
message OpaqueRegisterFinishRequest {
string username = 1;
bytes upload = 2;
bytes identity_key = 3;
}
message OpaqueRegisterFinishResponse {
bool success = 1;
}
message OpaqueLoginStartRequest {
string username = 1;
bytes request = 2;
}
message OpaqueLoginStartResponse {
bytes response = 1;
}
message OpaqueLoginFinishRequest {
string username = 1;
bytes finalization = 2;
bytes identity_key = 3;
}
message OpaqueLoginFinishResponse {
bytes session_token = 1;
}
```
---
## Method-by-method analysis
## Registration flow (IDs 100-101)
### `uploadKeyPackage @0`
User registration takes two round trips. The `request` and `response` fields carry opaque OPAQUE protocol blobs; their internal structure is defined by the `opaque-ke` crate.
### OpaqueRegisterStart (ID 100)
```
uploadKeyPackage (identityKey :Data, package :Data) -> (fingerprint :Data)
Client Server
| |
| OpaqueRegisterStartRequest |
| username: "alice" |
| request: <OPAQUE blob> |
| -----------------------------> |
| |
| OpaqueRegisterStartResponse |
| response: <OPAQUE blob> |
| <----------------------------- |
```
**Purpose:** A client uploads a single-use MLS KeyPackage so that peers can later fetch it to add the client to a group.
**Request fields:**
**Parameters:**
| Field | Type | Description |
|-------|------|-------------|
| `username` | `string` | The username being registered. Must be unique on the server. |
| `request` | `bytes` | OPAQUE `RegistrationRequest` blob generated by the client using the `opaque-ke` crate. |
| Parameter | Type | Size | Description |
|---|---|---|---|
| `identityKey` | `Data` | Exactly 32 bytes | The uploader's raw Ed25519 public key bytes. This is the index key under which the package is stored. |
| `package` | `Data` | Variable (bounded by transport max) | An openmls-serialised KeyPackage blob in TLS encoding. Contains the client's HPKE init key, credential, and signature. |
**Response fields:**
**Return value:**
| Field | Type | Description |
|-------|------|-------------|
| `response` | `bytes` | OPAQUE `RegistrationResponse` blob generated by the server. Client feeds this into the finish step. |
| Field | Type | Size | Description |
|---|---|---|---|
| `fingerprint` | `Data` | 32 bytes | SHA-256 digest of the uploaded `package` bytes. |
**Fingerprint semantics:** The returned fingerprint allows the uploading client to verify that the server stored the package correctly. More importantly, when a peer later fetches a KeyPackage, it can compare the fetched package's SHA-256 hash against the fingerprint (communicated out-of-band) to detect tampering by a malicious server.
**Idempotency:** Uploading the same package twice appends a second copy to the queue. The server does not deduplicate. Clients should avoid uploading duplicates to conserve their KeyPackage supply.
### `fetchKeyPackage @1`
### OpaqueRegisterFinish (ID 101)
```
fetchKeyPackage (identityKey :Data) -> (package :Data)
Client Server
| |
| OpaqueRegisterFinishRequest |
| username: "alice" |
| upload: <OPAQUE record> |
| identity_key: <32 bytes> |
| -----------------------------> |
| |
| OpaqueRegisterFinishResponse |
| success: true |
| <----------------------------- |
```
**Purpose:** Fetch and atomically remove one KeyPackage for a given identity. This is the mechanism by which a group creator obtains a peer's key material in order to add them to a group via MLS `add_members()`.
**Request fields:**
**Parameters:**
| Field | Type | Description |
|-------|------|-------------|
| `username` | `string` | Must match the username from the start request. |
| `upload` | `bytes` | OPAQUE `RegistrationUpload` blob. The server stores this as the user's OPAQUE record; it contains the password-derived key material without revealing the password. |
| `identity_key` | `bytes` | The user's Ed25519 identity public key (32 bytes). Stored alongside the OPAQUE record and used as the user's long-term identifier for key packages and delivery queues. |
| Parameter | Type | Size | Description |
|---|---|---|---|
| `identityKey` | `Data` | Exactly 32 bytes | The raw Ed25519 public key of the target peer whose KeyPackage is being requested. |
**Response fields:**
**Return value:**
| Field | Type | Size | Description |
|---|---|---|---|
| `package` | `Data` | Variable, or 0 bytes | The fetched KeyPackage blob, or empty `Data` if no packages are stored for this identity. |
**Atomic removal:** The fetch operation is destructive: it removes the returned KeyPackage from the server's store in the same operation that returns it. This guarantees MLS's single-use requirement -- a KeyPackage is never served to two different requesters.
**Empty response handling:** Callers must check for an empty response. An empty `package` means the target has no KeyPackages available. The caller should either:
1. Retry after a delay, hoping the target uploads more packages.
2. Signal the user that the target is unreachable for group addition.
| Field | Type | Description |
|-------|------|-------------|
| `success` | `bool` | `true` if the registration record was stored successfully. `false` if the username is already taken or another error occurred. |
---
## Indexing by raw Ed25519 public key
## Login flow (IDs 102-103)
The Authentication Service indexes KeyPackages by the **raw 32-byte Ed25519 public key**, not by a fingerprint or any higher-level identifier. This design choice has several implications:
User login also takes two round trips. On success, the server issues a `session_token` that the client attaches to subsequent authenticated RPCs.
1. **No directory service required for lookup.** The caller must already know the target's Ed25519 public key (obtained out-of-band via QR code scan, manual exchange, or a future directory service).
### OpaqueLoginStart (ID 102)
2. **Consistent with DS indexing.** The [Delivery Service](delivery-schema.md) uses the same 32-byte Ed25519 key as its queue index, so a single key serves as the universal identifier across both services.
```
Client Server
| |
| OpaqueLoginStartRequest |
| username: "alice" |
| request: <OPAQUE blob> |
| -----------------------------> |
| |
| OpaqueLoginStartResponse |
| response: <OPAQUE blob> |
| <----------------------------- |
```
3. **No ambiguity.** Unlike fingerprints (which could collide if truncated) or human-readable names (which require a mapping layer), the raw public key is the canonical, collision-resistant identifier.
**Request fields:**
| Field | Type | Description |
|-------|------|-------------|
| `username` | `string` | The username logging in. |
| `request` | `bytes` | OPAQUE `CredentialRequest` blob generated by the client. |
**Response fields:**
| Field | Type | Description |
|-------|------|-------------|
| `response` | `bytes` | OPAQUE `CredentialResponse` blob. Contains the server's masked public key and envelope for the client to derive its export key. |
### OpaqueLoginFinish (ID 103)
```
Client Server
| |
| OpaqueLoginFinishRequest |
| username: "alice" |
| finalization: <OPAQUE blob> |
| identity_key: <32 bytes> |
| -----------------------------> |
| |
| OpaqueLoginFinishResponse |
| session_token: <32 bytes> |
| <----------------------------- |
```
**Request fields:**
| Field | Type | Description |
|-------|------|-------------|
| `username` | `string` | Must match the username from the start request. |
| `finalization` | `bytes` | OPAQUE `CredentialFinalization` blob containing the client's proof of knowledge of the password. The server verifies this against its stored OPAQUE record. |
| `identity_key` | `bytes` | The user's Ed25519 identity public key (32 bytes). The server verifies this matches the key registered during `OpaqueRegisterFinish`. |
**Response fields:**
| Field | Type | Description |
|-------|------|-------------|
| `session_token` | `bytes` | Opaque bearer token (32 bytes). Included in subsequent RPC requests to authenticate the session. The server associates this token with the user's identity and device. |
If login fails (wrong password, unknown username, or identity key mismatch), the server returns an error status in the response frame; the `session_token` field is empty.
---
## Single-use semantics
## Session token usage
MLS requires that each KeyPackage be used at most once to preserve the forward secrecy of the initial key exchange. The Authentication Service enforces this by atomically removing the KeyPackage on fetch.
After a successful `OpaqueLoginFinish`, the client uses the `session_token` as a bearer credential for all authenticated RPC methods. The token is passed at the QUIC connection level (not per-frame); the server validates it on connection establishment and maintains the association for the lifetime of the connection.
**Consequences for clients:**
The `Auth` message in `common.proto` carries the token for federation and internal use:
- Clients should **pre-upload multiple KeyPackages** after generating their identity, so that several peers can add them to groups concurrently without exhausting the supply.
- Clients should **monitor their KeyPackage count** on the server (via a future monitoring endpoint or periodic re-upload) and replenish when the supply runs low.
- If a client has zero KeyPackages stored, it is effectively unreachable for new group invitations until it uploads more.
For the design rationale behind single-use KeyPackages, see [ADR-005: Single-Use KeyPackages](../design-rationale/adr-005-single-use-keypackages.md).
---
## Relationship to NodeService
In the current unified architecture, the Authentication Service methods are exposed as part of the [NodeService interface](node-service-schema.md):
| AuthenticationService Method | NodeService Method | Additional Parameters |
|---|---|---|
| `uploadKeyPackage @0` | `uploadKeyPackage @0` | `auth :Auth` |
| `fetchKeyPackage @1` | `fetchKeyPackage @1` | `auth :Auth` |
The standalone `AuthenticationService` interface remains in the schema for documentation purposes and for use in contexts where the full NodeService is not needed.
```protobuf
message Auth {
bytes access_token = 1;
bytes device_id = 2;
}
```
---
## Further reading
- [Wire Format Overview](overview.md) -- serialisation pipeline context
- [NodeService Schema](node-service-schema.md) -- unified interface that subsumes AuthenticationService
- [Delivery Schema](delivery-schema.md) -- the companion service for message routing
- [Envelope Schema](envelope-schema.md) -- legacy framing that used `keyPackageUpload`/`keyPackageFetch` message types
- [ADR-005: Single-Use KeyPackages](../design-rationale/adr-005-single-use-keypackages.md) -- design rationale for atomic removal on fetch
- [ADR-004: MLS-Unaware Delivery Service](../design-rationale/adr-004-mls-unaware-ds.md) -- why the server does not inspect MLS content
- [Wire Format Overview](overview.md) -- frame format and transport parameters
- [Method ID Reference](envelope-schema.md) -- all 44 method IDs
- [Authentication Service Internals](../internals/authentication-service.md) -- server-side OPAQUE flow and session management
- [RPC Reference](node-service-schema.md) -- all proto definitions

468
docs/src/wire-format/delivery-schema.md
View File

@@ -1,193 +1,377 @@
# Delivery Schema
# Delivery and Keys Schema
**Schema file:** `schemas/delivery.capnp`
**File ID:** `@0xc5d9e2b4f1a83076`
**Proto files:** `proto/qpq/v1/delivery.proto`, `proto/qpq/v1/keys.proto`
**Package:** `qpq.v1`
**Method IDs:** 200-205 (delivery), 300-304 (key packages and hybrid keys), 510-520 (key transparency)
The `DeliveryService` interface defines the RPC contract for the store-and-forward message relay. The DS is intentionally MLS-unaware: it routes opaque byte strings by recipient key and optional channel ID without parsing or inspecting the content.
This page documents the Protobuf message definitions for the delivery service (store-and-forward message relay) and the key management service (MLS KeyPackages, hybrid post-quantum keys, and key transparency).
---
## Full schema listing
## delivery.proto
```capnp
# delivery.capnp -- Delivery Service RPC interface.
#
# The Delivery Service is a simple store-and-forward relay. It does not parse
# MLS messages -- all payloads are opaque byte strings routed by recipient key.
#
# Callers are responsible for:
# - Routing Welcome messages to the correct new member after add_members().
# - Routing Commit messages to any existing group members (other than self).
# - Routing Application messages to the intended recipient(s).
#
# The DS indexes queues by the recipient's raw Ed25519 public key (32 bytes),
# matching the indexing scheme used by the Authentication Service.
#
# ID generated with: capnp id
@0xc5d9e2b4f1a83076;
The delivery service is a store-and-forward relay. It is intentionally MLS-unaware: all payloads are opaque byte strings routed by recipient key and channel ID. The server never inspects or decrypts message content.
interface DeliveryService {
# Enqueue an opaque payload for delivery to a recipient.
#
# recipientKey : Ed25519 public key of the intended recipient (exactly 32 bytes).
# payload : Opaque byte string -- a TLS-encoded MlsMessageOut blob or any
# other framed data the application layer wants to deliver.
# channelId : Optional channel identifier (empty for legacy). A 16-byte UUID
# is recommended for 1:1 channels.
# version : Schema/wire version. Must be 0 (legacy) or 1 (this spec).
#
# The payload is appended to the recipient's FIFO queue. Returns immediately;
# the recipient retrieves it via `fetch`.
enqueue @0 (recipientKey :Data, payload :Data, channelId :Data, version :UInt16) -> ();
### Full proto listing
# Fetch and atomically drain all queued payloads for a given recipient.
#
# recipientKey : Ed25519 public key of the caller (exactly 32 bytes).
# channelId : Optional channel identifier (empty for legacy).
# version : Schema/wire version. Must be 0 (legacy) or 1 (this spec).
#
# Returns the complete queue in FIFO order and clears it. Returns an empty
# list if there are no pending messages.
fetch @1 (recipientKey :Data, channelId :Data, version :UInt16) -> (payloads :List(Data));
```protobuf
syntax = "proto3";
package qpq.v1;
// Delivery service: enqueue, fetch, peek, ack, batch (6 methods).
// Method IDs: 200-205.
message Envelope {
uint64 seq = 1;
bytes data = 2;
}
message EnqueueRequest {
bytes recipient_key = 1;
bytes payload = 2;
bytes channel_id = 3;
uint32 ttl_secs = 4;
// Client-generated idempotency key (16 bytes, UUID v7).
// Server deduplicates enqueue requests with the same message_id within a TTL window.
bytes message_id = 5;
}
message EnqueueResponse {
uint64 seq = 1;
bytes delivery_proof = 2;
// True if this was a duplicate enqueue (message_id already seen).
bool duplicate = 3;
}
message FetchRequest {
bytes recipient_key = 1;
bytes channel_id = 2;
uint32 limit = 3;
// Device ID for multi-device scoping.
bytes device_id = 4;
}
message FetchResponse {
repeated Envelope payloads = 1;
}
message FetchWaitRequest {
bytes recipient_key = 1;
bytes channel_id = 2;
uint64 timeout_ms = 3;
uint32 limit = 4;
bytes device_id = 5;
}
message FetchWaitResponse {
repeated Envelope payloads = 1;
}
message PeekRequest {
bytes recipient_key = 1;
bytes channel_id = 2;
uint32 limit = 3;
bytes device_id = 4;
}
message PeekResponse {
repeated Envelope payloads = 1;
}
message AckRequest {
bytes recipient_key = 1;
bytes channel_id = 2;
uint64 seq_up_to = 3;
bytes device_id = 4;
}
message AckResponse {}
message BatchEnqueueRequest {
repeated bytes recipient_keys = 1;
bytes payload = 2;
bytes channel_id = 3;
uint32 ttl_secs = 4;
bytes message_id = 5;
}
message BatchEnqueueResponse {
repeated uint64 seqs = 1;
}
```
---
### Envelope
## Method-by-method analysis
### `enqueue @0`
```
enqueue (recipientKey :Data, payload :Data, channelId :Data, version :UInt16) -> ()
```
**Purpose:** Append an opaque payload to a recipient's delivery queue. The DS stores the payload until the recipient fetches it. The call returns immediately after the payload is enqueued; it does not block until delivery.
**Parameters:**
| Parameter | Type | Size | Description |
|---|---|---|---|
| `recipientKey` | `Data` | Exactly 32 bytes | Ed25519 public key of the intended recipient. Used as the primary queue index. |
| `payload` | `Data` | Variable (bounded by transport max) | Opaque byte string. Typically a TLS-encoded `MlsMessageOut` blob, but the DS does not inspect it. |
| `channelId` | `Data` | 0 bytes (legacy) or 16 bytes (UUID) | Channel identifier for channel-aware routing. Empty `Data` is treated as the legacy default channel. |
| `version` | `UInt16` | 2 bytes | Schema/wire version. `0` = legacy (no channel routing), `1` = current spec (channel-aware). |
**Return value:** Void. The method returns `()` on success. Errors are surfaced as Cap'n Proto RPC exceptions.
**Queue semantics:** Payloads are appended in FIFO order. The DS does not deduplicate, reorder, or inspect payloads. Multiple enqueue calls for the same recipient and channel ID are simply appended to the queue in the order they arrive.
### `fetch @1`
```
fetch (recipientKey :Data, channelId :Data, version :UInt16) -> (payloads :List(Data))
```
**Purpose:** Fetch and atomically drain all queued payloads for a given recipient on a given channel. This is the "pull" side of the store-and-forward relay.
**Parameters:**
| Parameter | Type | Size | Description |
|---|---|---|---|
| `recipientKey` | `Data` | Exactly 32 bytes | Ed25519 public key of the caller. Must match the key used in the enqueue calls. |
| `channelId` | `Data` | 0 bytes (legacy) or 16 bytes (UUID) | Channel identifier. Must match the `channelId` used during enqueue. |
| `version` | `UInt16` | 2 bytes | Schema/wire version. Must match the version used during enqueue. |
**Return value:**
The `Envelope` wrapper is returned by fetch, peek, and fetch-wait operations.
| Field | Type | Description |
|---|---|---|
| `payloads` | `List(Data)` | All queued payloads in FIFO order. Empty list if no messages are pending. |
|-------|------|-------------|
| `seq` | `uint64` | Server-assigned monotonic sequence number for ordering and acknowledgment. |
| `data` | `bytes` | The original payload bytes submitted at enqueue time. |
**Atomic drain:** The fetch operation returns the entire queue and clears it in a single atomic operation. There is no "peek" or partial fetch. This simplifies the concurrency model: the client processes all returned payloads and does not need to track which ones it has already seen.
### Enqueue (ID 200)
Appends an opaque payload to a recipient's queue. Returns immediately.
**Request:**
| Field | Type | Description |
|-------|------|-------------|
| `recipient_key` | `bytes` | Recipient's Ed25519 identity public key (32 bytes). Primary queue index. |
| `payload` | `bytes` | Opaque byte string. Typically a TLS-encoded MLS ciphertext blob. |
| `channel_id` | `bytes` | Channel identifier (16-byte UUID v7 recommended). Empty = default channel. |
| `ttl_secs` | `uint32` | Time-to-live in seconds. Server garbage-collects expired messages. 0 = server default. |
| `message_id` | `bytes` | Client-generated idempotency key (16 bytes, UUID v7). Server deduplicates within the TTL window. |
**Response:**
| Field | Type | Description |
|-------|------|-------------|
| `seq` | `uint64` | Server-assigned sequence number for this message. |
| `delivery_proof` | `bytes` | Cryptographic proof of delivery (reserved for future use). |
| `duplicate` | `bool` | `true` if this `message_id` was already seen within the TTL window; the payload was not stored again. |
### Fetch (ID 201)
Returns and retains queued messages up to `limit`. Does not remove messages from the queue; use `Ack` to advance the read cursor.
**Request:**
| Field | Type | Description |
|-------|------|-------------|
| `recipient_key` | `bytes` | Recipient's Ed25519 identity public key (32 bytes). |
| `channel_id` | `bytes` | Channel identifier. Must match the value used at enqueue time. |
| `limit` | `uint32` | Maximum number of envelopes to return. 0 = server default. |
| `device_id` | `bytes` | Optional device identifier for multi-device queue scoping. |
**Response:**
| Field | Type | Description |
|-------|------|-------------|
| `payloads` | `repeated Envelope` | Messages in FIFO order. Empty list if no messages are pending. |
### FetchWait (ID 202)
Long-poll variant of `Fetch`. Blocks on the server until messages arrive or `timeout_ms` elapses.
**Request:**
| Field | Type | Description |
|-------|------|-------------|
| `recipient_key` | `bytes` | Recipient's Ed25519 identity public key (32 bytes). |
| `channel_id` | `bytes` | Channel identifier. |
| `timeout_ms` | `uint64` | Maximum wait time in milliseconds. 0 = return immediately (equivalent to Fetch). |
| `limit` | `uint32` | Maximum number of envelopes to return. |
| `device_id` | `bytes` | Optional device identifier. |
**Response:** Same as `FetchResponse`.
FetchWait eliminates polling latency: the server holds the RPC open until a `Notify` is signalled by a concurrent `Enqueue` call, or until `timeout_ms` expires.
### Peek (ID 203)
Non-destructive read. Returns messages without removing them and without advancing the acknowledgment cursor.
**Request / Response:** Same field layout as `FetchRequest` / `FetchResponse`.
Peek is useful for inspecting pending messages without marking them as delivered.
### Ack (ID 204)
Advances the delivery cursor, removing all messages with `seq <= seq_up_to` from the queue.
**Request:**
| Field | Type | Description |
|-------|------|-------------|
| `recipient_key` | `bytes` | Recipient's Ed25519 identity public key (32 bytes). |
| `channel_id` | `bytes` | Channel identifier. |
| `seq_up_to` | `uint64` | All messages with sequence number <= this value are removed. |
| `device_id` | `bytes` | Optional device identifier. |
**Response:** Empty (`AckResponse {}`).
### BatchEnqueue (ID 205)
Fan-out: enqueues the same payload to multiple recipients in a single RPC call.
**Request:**
| Field | Type | Description |
|-------|------|-------------|
| `recipient_keys` | `repeated bytes` | List of recipient Ed25519 identity public keys. |
| `payload` | `bytes` | Opaque payload, delivered identically to all recipients. |
| `channel_id` | `bytes` | Channel identifier. |
| `ttl_secs` | `uint32` | Time-to-live in seconds. |
| `message_id` | `bytes` | Idempotency key (16 bytes). |
**Response:**
| Field | Type | Description |
|-------|------|-------------|
| `seqs` | `repeated uint64` | Server-assigned sequence numbers, one per `recipient_key`, in the same order. |
---
## Channel-aware routing
## keys.proto
The `channelId` field enables per-channel queue separation. Each unique `(recipientKey, channelId)` pair maps to an independent FIFO queue on the server.
Key management for MLS KeyPackages, hybrid post-quantum keys, and key transparency audit.
### Compound key structure
### Full proto listing
```text
Queue Key = recipientKey (32 bytes) || channelId (0 or 16 bytes)
```protobuf
syntax = "proto3";
package qpq.v1;
// Key package + hybrid key CRUD (5 methods).
// Method IDs: 300-304.
message UploadKeyPackageRequest {
bytes identity_key = 1;
bytes package = 2;
}
message UploadKeyPackageResponse {
bytes fingerprint = 1;
}
message FetchKeyPackageRequest {
bytes identity_key = 1;
}
message FetchKeyPackageResponse {
bytes package = 1;
}
message UploadHybridKeyRequest {
bytes identity_key = 1;
bytes hybrid_public_key = 2;
}
message UploadHybridKeyResponse {}
message FetchHybridKeyRequest {
bytes identity_key = 1;
}
message FetchHybridKeyResponse {
bytes hybrid_public_key = 1;
}
message FetchHybridKeysRequest {
repeated bytes identity_keys = 1;
}
message FetchHybridKeysResponse {
repeated bytes keys = 1;
}
// Key revocation (method ID 510).
message RevokeKeyRequest {
bytes identity_key = 1;
string reason = 2; // "compromised", "superseded", "user_revoked"
}
message RevokeKeyResponse {
bool success = 1;
uint64 leaf_index = 2;
}
// Check revocation status (method ID 511).
message CheckRevocationRequest {
bytes identity_key = 1;
}
message CheckRevocationResponse {
bool revoked = 1;
string reason = 2;
uint64 timestamp_ms = 3;
}
// KT audit log retrieval (method ID 520).
message AuditKeyTransparencyRequest {
uint64 start = 1;
uint64 end = 2;
}
message AuditKeyTransparencyResponse {
repeated LogEntry entries = 1;
uint64 tree_size = 2;
bytes root = 3;
}
message LogEntry {
uint64 index = 1;
bytes leaf_hash = 2;
}
```
When `channelId` is empty (0 bytes), the queue key degenerates to just the `recipientKey`, preserving backward compatibility with legacy clients that do not use channels.
### UploadKeyPackage (ID 300)
### Channel ID format
Uploads a single-use MLS KeyPackage. KeyPackages are stored in a FIFO queue per identity; each is consumed once by `FetchKeyPackage`.
The recommended format for `channelId` is a 16-byte UUID (128-bit, typically UUID v4). The DS treats the channel ID as an opaque byte string and does not parse its structure. Using UUIDs provides:
| Field | Type | Description |
|-------|------|-------------|
| `identity_key` | `bytes` | Uploader's Ed25519 identity public key (32 bytes). Index key for the queue. |
| `package` | `bytes` | openmls-serialised KeyPackage (bincode format, as required by `DiskKeyStore`). |
1. **Collision resistance** -- 2^122 random bits (for UUID v4) makes accidental collision negligible.
2. **Privacy** -- The channel ID reveals no information about the channel's participants or purpose.
3. **Fixed size** -- 16 bytes is compact and predictable for indexing.
Response: `fingerprint` -- SHA-256 digest of the stored package (32 bytes). Callers should record this to detect tampering.
### Use cases
### FetchKeyPackage (ID 301)
| Scenario | channelId | recipientKey | Result |
|---|---|---|---|
| Legacy client, no channels | Empty (0 bytes) | Alice's Ed25519 key | Single queue for all of Alice's messages |
| 1:1 channel between Alice and Bob | UUID of the 1:1 channel | Alice's Ed25519 key | Separate queue for this specific channel |
| Group channel | UUID of the group channel | Alice's Ed25519 key | Separate queue for this group's messages to Alice |
Fetches and atomically removes one KeyPackage for the given identity. Returns empty bytes if no packages are stored. The removal is atomic; concurrent fetches will not receive the same package.
---
### UploadHybridKey (ID 302)
## Version field
Uploads the client's hybrid (X25519 + ML-KEM-768) public key. Unlike KeyPackages, hybrid keys are not single-use -- each identity stores exactly one, overwriting the previous value.
The `version` field provides a mechanism for wire-level schema evolution without breaking existing clients.
| Field | Type | Description |
|-------|------|-------------|
| `identity_key` | `bytes` | Uploader's Ed25519 identity public key (32 bytes). |
| `hybrid_public_key` | `bytes` | Concatenated X25519 public key (32 bytes) + ML-KEM-768 encapsulation key. |
| Version | Semantics |
|---|---|
| `0` | Legacy mode. `channelId` is ignored (treated as empty). Behaves like the pre-channel DeliveryService. |
| `1` | Current specification. `channelId` is used for channel-aware routing. |
### FetchHybridKey (ID 303)
The server validates the version field and rejects unknown versions as protocol errors. Clients must set the version field to match the schema revision they implement.
Fetches a single peer's hybrid public key. Non-destructive.
---
### FetchHybridKeys (ID 304)
## FIFO queue semantics
Batch variant of `FetchHybridKey`. Returns one key per input identity key, in the same order. Missing keys are returned as empty bytes at the corresponding index.
The Delivery Service provides strict FIFO ordering within each `(recipientKey, channelId)` queue:
### RevokeKey (ID 510)
1. **Enqueue order is preserved.** Payloads are returned by `fetch` in the exact order they were enqueued.
2. **Atomic drain.** Each `fetch` call returns all pending payloads and clears the queue. There is no risk of partial reads or interleaving.
3. **No persistence guarantees (current implementation).** The in-memory queue is lost on server restart. Persistent storage is planned for a future milestone.
4. **No redelivery.** Once a payload is returned by `fetch`, it is permanently removed. If the client crashes before processing it, the payload is lost. Reliable delivery with acknowledgments is a future enhancement.
Revokes an identity key by appending a revocation entry to the key transparency Merkle log.
---
| Field | Type | Description |
|-------|------|-------------|
| `identity_key` | `bytes` | Identity key to revoke (32 bytes). |
| `reason` | `string` | One of: `"compromised"`, `"superseded"`, `"user_revoked"`. |
## MLS-unaware design
Response: `leaf_index` is the index of the revocation entry in the KT Merkle log.
The DS intentionally does not parse, validate, or inspect MLS messages. All payloads are opaque `Data` blobs. This design has several consequences:
### CheckRevocation (ID 511)
- **Security:** The server cannot extract plaintext from MLS ciphertext, even if compromised.
- **Simplicity:** The DS has no dependency on openmls or any MLS library.
- **Flexibility:** The same DS can carry non-MLS payloads (e.g., signaling, metadata) without modification.
- **No server-side optimization:** The DS cannot optimize delivery based on MLS message type (e.g., fanning out a Commit to all group members). The client must enqueue separately for each recipient.
Checks whether an identity key has been revoked.
For the full design rationale, see [ADR-004: MLS-Unaware Delivery Service](../design-rationale/adr-004-mls-unaware-ds.md).
Response fields: `revoked` (bool), `reason` (string), `timestamp_ms` (uint64 unix milliseconds of the revocation event).
---
### AuditKeyTransparency (ID 520)
## Relationship to NodeService
Returns a range of entries from the key transparency append-only Merkle log.
In the current unified architecture, the Delivery Service methods are exposed as part of the [NodeService interface](node-service-schema.md) with additional methods:
| Field | Type | Description |
|-------|------|-------------|
| `start` | `uint64` | First leaf index (inclusive). |
| `end` | `uint64` | Last leaf index (exclusive). 0 = up to current tree size. |
| DeliveryService Method | NodeService Method | Additional Parameters |
|---|---|---|
| `enqueue @0` | `enqueue @2` | `auth :Auth` |
| `fetch @1` | `fetch @3` | `auth :Auth` |
| *(none)* | `fetchWait @4` | `auth :Auth`, `timeoutMs :UInt64` |
The `fetchWait` method is a NodeService extension that provides long-polling semantics: it blocks until either new payloads arrive or the timeout expires. This avoids the latency and bandwidth overhead of repeated `fetch` polling.
Response: `entries` (list of `LogEntry`), `tree_size` (current log size), `root` (Merkle root hash).
---
## Further reading
- [Wire Format Overview](overview.md) -- serialisation pipeline context
- [NodeService Schema](node-service-schema.md) -- unified interface that subsumes DeliveryService
- [Auth Schema](auth-schema.md) -- the companion service for KeyPackage management
- [Envelope Schema](envelope-schema.md) -- legacy framing that used `mlsWelcome`/`mlsCommit`/`mlsApplication` message types
- [ADR-004: MLS-Unaware Delivery Service](../design-rationale/adr-004-mls-unaware-ds.md) -- why the DS does not inspect MLS content
- [Wire Format Overview](overview.md) -- frame format and transport parameters
- [Method ID Reference](envelope-schema.md) -- all 44 method IDs
- [Auth Schema](auth-schema.md) -- OPAQUE authentication proto definitions
- [RPC Reference](node-service-schema.md) -- all proto definitions for all 14 files
- [Storage Backend](../internals/storage-backend.md) -- how KeyPackages and hybrid keys are persisted

301
docs/src/wire-format/envelope-schema.md
View File

@@ -1,149 +1,208 @@
# Envelope Schema
# Method ID Reference
**Schema file:** `schemas/envelope.capnp`
**File ID:** `@0xe4a7f2c8b1d63509`
The v2 RPC protocol dispatches requests by a `u16` method ID encoded in the first two bytes of every request frame. This page is the authoritative reference for all 44 method IDs and their corresponding Protobuf message types.
The Envelope is the legacy top-level wire message used in M1 for all quicproquo traffic. Every frame exchanged between peers was serialised as an Envelope, with the Delivery Service routing by `(groupId, msgType)` without inspecting the payload.
> **Note:** The Envelope is the M1-era framing format. The current M3+ architecture uses Cap'n Proto RPC directly via the [NodeService](node-service-schema.md) interface. The Envelope schema remains in the codebase for backward compatibility and for use in integration tests.
Method IDs are defined in `crates/quicproquo-proto/src/lib.rs` (the `method_ids` module). Proto definitions live in `proto/qpq/v1/`.
---
## Full schema listing
## Auth (100-103)
```capnp
# envelope.capnp -- top-level wire message for all quicproquo traffic.
#
# Every frame is serialised as an Envelope.
# The Delivery Service routes by (groupId, msgType) without inspecting payload.
#
# Field sizing rationale:
# groupId / senderId : 32 bytes -- SHA-256 digest
# payload : opaque -- MLS blob or control data
# timestampMs : UInt64 -- unix epoch milliseconds; sufficient until year 292M
#
# ID generated with: capnp id
@0xe4a7f2c8b1d63509;
OPAQUE asymmetric password-authenticated key exchange. Registration is a two-round trip (start + finish); login is a two-round trip (start + finish). See [Auth Schema](auth-schema.md) for proto definitions and [Authentication Service](../internals/authentication-service.md) for flow diagrams.
struct Envelope {
# Message type discriminant -- determines how payload is interpreted.
msgType @0 :MsgType;
# 32-byte SHA-256 digest of the group name.
# The Delivery Service uses this as its routing key.
# Zero-filled for point-to-point control messages (ping, keyPackageUpload, etc.).
groupId @1 :Data;
# 32-byte SHA-256 digest of the sender's Ed25519 identity public key.
senderId @2 :Data;
# Opaque payload. Interpretation is determined by msgType.
payload @3 :Data;
# Unix timestamp in milliseconds at the time of send.
timestampMs @4 :UInt64;
enum MsgType {
ping @0;
pong @1;
keyPackageUpload @2;
keyPackageFetch @3;
keyPackageResponse @4;
mlsWelcome @5;
mlsCommit @6;
mlsApplication @7;
error @8;
}
}
```
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 100 | `OPAQUE_REGISTER_START` | `OpaqueRegisterStartRequest` | `OpaqueRegisterStartResponse` |
| 101 | `OPAQUE_REGISTER_FINISH` | `OpaqueRegisterFinishRequest` | `OpaqueRegisterFinishResponse` |
| 102 | `OPAQUE_LOGIN_START` | `OpaqueLoginStartRequest` | `OpaqueLoginStartResponse` |
| 103 | `OPAQUE_LOGIN_FINISH` | `OpaqueLoginFinishRequest` | `OpaqueLoginFinishResponse` |
---
## Field-by-field analysis
## Delivery (200-205)
### `msgType @0 :MsgType`
Store-and-forward message relay. The server is MLS-unaware: payloads are opaque byte strings routed by recipient key and channel ID. See [Delivery Schema](delivery-schema.md) for proto definitions.
A 16-bit enum discriminant (Cap'n Proto enums are encoded as UInt16). Determines how the `payload` field should be interpreted. The discriminant is the first field in the struct for efficient dispatch: a router can read the first two bytes of the struct section to decide how to handle the message without parsing any pointer fields.
### `groupId @1 :Data`
A 32-byte `Data` field containing the SHA-256 digest of the group name. The Delivery Service uses this as its primary routing key when the Envelope-based protocol is active.
**Sizing rationale:** SHA-256 produces a 32-byte (256-bit) digest. This is stored as a variable-length `Data` field rather than a fixed-size blob because Cap'n Proto does not have a fixed-size array type. Implementations must validate that the field contains exactly 32 bytes.
**Special case:** For point-to-point control messages (`ping`, `pong`, `keyPackageUpload`, `keyPackageFetch`), the `groupId` is zero-filled (32 zero bytes) because these messages are not associated with any group.
### `senderId @2 :Data`
A 32-byte `Data` field containing the SHA-256 digest of the sender's Ed25519 identity public key. This allows the receiver to identify the sender without inspecting the MLS-layer credentials.
**Sizing rationale:** Same as `groupId` -- SHA-256 digest, 32 bytes.
### `payload @3 :Data`
An opaque byte string whose interpretation depends on `msgType`.
### `timestampMs @4 :UInt64`
Unix epoch timestamp in milliseconds, set by the sender at the time of send. Encoded as a `UInt64`, which provides sufficient range until approximately year 292,000,000 -- effectively unlimited for practical purposes.
The timestamp is sender-asserted and **not** authenticated by the server. Receivers should treat it as advisory (for display ordering) rather than authoritative.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 200 | `ENQUEUE` | `EnqueueRequest` | `EnqueueResponse` |
| 201 | `FETCH` | `FetchRequest` | `FetchResponse` |
| 202 | `FETCH_WAIT` | `FetchWaitRequest` | `FetchWaitResponse` |
| 203 | `PEEK` | `PeekRequest` | `PeekResponse` |
| 204 | `ACK` | `AckRequest` | `AckResponse` |
| 205 | `BATCH_ENQUEUE` | `BatchEnqueueRequest` | `BatchEnqueueResponse` |
---
## MsgType enum
## Keys (300-304)
The `MsgType` enum defines nine message types. Each variant determines how the `payload` field is interpreted:
MLS KeyPackage and hybrid post-quantum key management. See [Delivery Schema](delivery-schema.md) for proto definitions (keys are defined in `keys.proto`).
| Ordinal | Variant | Payload Contents | Direction |
|---|---|---|---|
| 0 | `ping` | Empty | Client -> Server or Peer -> Peer |
| 1 | `pong` | Empty | Server -> Client or Peer -> Peer |
| 2 | `keyPackageUpload` | openmls-serialised KeyPackage blob (TLS encoding) | Client -> Server |
| 3 | `keyPackageFetch` | Target identity key (32 bytes, raw Ed25519 public key) | Client -> Server |
| 4 | `keyPackageResponse` | openmls-serialised KeyPackage blob, or empty if none stored | Server -> Client |
| 5 | `mlsWelcome` | `MLSMessage` blob (Welcome variant) | Peer -> Peer (via DS) |
| 6 | `mlsCommit` | `MLSMessage` blob (PublicMessage / Commit variant) | Peer -> Group (via DS) |
| 7 | `mlsApplication` | `MLSMessage` blob (PrivateMessage / Application variant) | Peer -> Group (via DS) |
| 8 | `error` | UTF-8 error description string | Any direction |
### Control messages (0-1)
`ping` and `pong` are keepalive probes with empty payloads. They serve as health checks over long-lived connections.
### Authentication messages (2-4)
`keyPackageUpload`, `keyPackageFetch`, and `keyPackageResponse` implement the Authentication Service protocol over the Envelope format. In the current architecture, these operations are handled by the [NodeService RPC](node-service-schema.md) methods `uploadKeyPackage` and `fetchKeyPackage` instead.
### MLS messages (5-7)
`mlsWelcome`, `mlsCommit`, and `mlsApplication` carry MLS protocol messages as opaque blobs. The Envelope does not inspect or validate the MLS content; it simply transports the bytes between peers via the Delivery Service.
### Error messages (8)
`error` carries a UTF-8 string describing an error condition. Used for protocol-level error reporting (e.g., "no KeyPackage found for identity").
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 300 | `UPLOAD_KEY_PACKAGE` | `UploadKeyPackageRequest` | `UploadKeyPackageResponse` |
| 301 | `FETCH_KEY_PACKAGE` | `FetchKeyPackageRequest` | `FetchKeyPackageResponse` |
| 302 | `UPLOAD_HYBRID_KEY` | `UploadHybridKeyRequest` | `UploadHybridKeyResponse` |
| 303 | `FETCH_HYBRID_KEY` | `FetchHybridKeyRequest` | `FetchHybridKeyResponse` |
| 304 | `FETCH_HYBRID_KEYS` | `FetchHybridKeysRequest` | `FetchHybridKeysResponse` |
---
## Relationship to NodeService
## Channel (400)
The Envelope schema was the original M1 wire format. With the transition to QUIC + TLS 1.3 and Cap'n Proto RPC in M3, the Envelope's role has been superseded by the [NodeService interface](node-service-schema.md), which provides typed RPC methods for each operation.
Direct-message channel creation. Returns a deterministic channel ID for a given peer key pair, with deduplication.
The key differences:
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 400 | `CREATE_CHANNEL` | `CreateChannelRequest` | `CreateChannelResponse` |
| Aspect | Envelope (M1) | NodeService RPC (M3+) |
|---|---|---|
| Dispatch | Manual, based on `msgType` enum | Automatic, Cap'n Proto RPC method dispatch |
| Type safety | Payload is opaque `Data` | Each method has typed parameters and return values |
| Transport | QUIC + TLS 1.3 | QUIC + TLS 1.3 |
| Auth | None | Explicit `Auth` struct per method call |
---
## Group Management (410-413)
MLS group operations: member removal, metadata updates, member listing, and key rotation.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 410 | `REMOVE_MEMBER` | `RemoveMemberRequest` | `RemoveMemberResponse` |
| 411 | `UPDATE_GROUP_METADATA` | `UpdateGroupMetadataRequest` | `UpdateGroupMetadataResponse` |
| 412 | `LIST_GROUP_MEMBERS` | `ListGroupMembersRequest` | `ListGroupMembersResponse` |
| 413 | `ROTATE_KEYS` | `RotateKeysRequest` | `RotateKeysResponse` |
---
## Moderation (420-424)
Content moderation: encrypted reports, user bans, and audit lists. Admin-only methods require elevated session privileges.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 420 | `REPORT_MESSAGE` | `ReportMessageRequest` | `ReportMessageResponse` |
| 421 | `BAN_USER` | `BanUserRequest` | `BanUserResponse` |
| 422 | `UNBAN_USER` | `UnbanUserRequest` | `UnbanUserResponse` |
| 423 | `LIST_REPORTS` | `ListReportsRequest` | `ListReportsResponse` |
| 424 | `LIST_BANNED` | `ListBannedRequest` | `ListBannedResponse` |
---
## User / Identity (500-501)
Forward and reverse user resolution. `ResolveUser` returns the identity key with a key-transparency inclusion proof.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 500 | `RESOLVE_USER` | `ResolveUserRequest` | `ResolveUserResponse` |
| 501 | `RESOLVE_IDENTITY` | `ResolveIdentityRequest` | `ResolveIdentityResponse` |
---
## Key Transparency (510-520)
Key revocation and audit log access for the Merkle-based key transparency log.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 510 | `REVOKE_KEY` | `RevokeKeyRequest` | `RevokeKeyResponse` |
| 511 | `CHECK_REVOCATION` | `CheckRevocationRequest` | `CheckRevocationResponse` |
| 520 | `AUDIT_KEY_TRANSPARENCY` | `AuditKeyTransparencyRequest` | `AuditKeyTransparencyResponse` |
---
## Blob Storage (600-601)
Content-addressed binary object storage with chunked upload and ranged download.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 600 | `UPLOAD_BLOB` | `UploadBlobRequest` | `UploadBlobResponse` |
| 601 | `DOWNLOAD_BLOB` | `DownloadBlobRequest` | `DownloadBlobResponse` |
---
## Device Management (700-702, 710)
Multi-device registration, listing, and revocation. Method 710 registers a platform push notification token.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 700 | `REGISTER_DEVICE` | `RegisterDeviceRequest` | `RegisterDeviceResponse` |
| 701 | `LIST_DEVICES` | `ListDevicesRequest` | `ListDevicesResponse` |
| 702 | `REVOKE_DEVICE` | `RevokeDeviceRequest` | `RevokeDeviceResponse` |
| 710 | `REGISTER_PUSH_TOKEN` | `RegisterPushTokenRequest` | `RegisterPushTokenResponse` |
---
## Recovery (750-752)
Encrypted account recovery bundle storage. The server stores an opaque blob indexed by `SHA-256(recovery_token)`; the plaintext is never visible to the server.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 750 | `STORE_RECOVERY_BUNDLE` | `StoreRecoveryBundleRequest` | `StoreRecoveryBundleResponse` |
| 751 | `FETCH_RECOVERY_BUNDLE` | `FetchRecoveryBundleRequest` | `FetchRecoveryBundleResponse` |
| 752 | `DELETE_RECOVERY_BUNDLE` | `DeleteRecoveryBundleRequest` | `DeleteRecoveryBundleResponse` |
---
## P2P / Health (800-802)
iroh P2P node address exchange and server health probe.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 800 | `PUBLISH_ENDPOINT` | `PublishEndpointRequest` | `PublishEndpointResponse` |
| 801 | `RESOLVE_ENDPOINT` | `ResolveEndpointRequest` | `ResolveEndpointResponse` |
| 802 | `HEALTH` | `HealthRequest` | `HealthResponse` |
---
## Federation (900-905)
Cross-server relay for messages, key packages, and user resolution. All federation methods include a `FederationAuth` struct carrying the origin server domain.
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 900 | `RELAY_ENQUEUE` | `RelayEnqueueRequest` | `RelayEnqueueResponse` |
| 901 | `RELAY_BATCH_ENQUEUE` | `RelayBatchEnqueueRequest` | `RelayBatchEnqueueResponse` |
| 902 | `PROXY_FETCH_KEY_PACKAGE` | `ProxyFetchKeyPackageRequest` | `ProxyFetchKeyPackageResponse` |
| 903 | `PROXY_FETCH_HYBRID_KEY` | `ProxyFetchHybridKeyRequest` | `ProxyFetchHybridKeyResponse` |
| 904 | `PROXY_RESOLVE_USER` | `ProxyResolveUserRequest` | `ProxyResolveUserResponse` |
| 905 | `FEDERATION_HEALTH` | `FederationHealthRequest` | `FederationHealthResponse` |
---
## Account (950)
| ID | Constant | Request | Response |
|----|----------|---------|---------|
| 950 | `DELETE_ACCOUNT` | `DeleteAccountRequest` | `DeleteAccountResponse` |
---
## Push Event Types (1000+)
Push events are sent by the server on QUIC uni-streams using the push frame format. They are not RPC methods (no `request_id`), but share the same event type namespace.
| ID | Constant | Payload |
|----|----------|---------|
| 1000 | `PUSH_NEW_MESSAGE` | `NewMessage` |
| 1001 | `PUSH_TYPING` | `TypingIndicator` |
| 1002 | `PUSH_PRESENCE` | `PresenceUpdate` |
| 1003 | `PUSH_MEMBERSHIP` | `GroupMembershipChange` |
Push payload messages are defined in `proto/qpq/v1/push.proto` and wrapped in a `PushEvent` oneof. See [RPC Reference](node-service-schema.md) for the full proto listing.
---
## Method ID assignment policy
Method IDs are stable across versions. Once assigned, an ID is never reused. New methods are assigned the next available ID in their logical category. Gaps in the numbering are reserved for future use within a category.
---
## Further reading
- [Wire Format Overview](overview.md) -- serialisation pipeline context
- [NodeService Schema](node-service-schema.md) -- the current RPC interface that replaced Envelope-based dispatch
- [Auth Schema](auth-schema.md) -- standalone Authentication Service interface
- [Delivery Schema](delivery-schema.md) -- standalone Delivery Service interface
- [ADR-002: Cap'n Proto over MessagePack](../design-rationale/adr-002-capnproto.md) -- why Cap'n Proto was chosen for the wire format
- [Wire Format Overview](overview.md) -- frame format and transport parameters
- [Auth Schema](auth-schema.md) -- OPAQUE proto definitions (IDs 100-103)
- [Delivery Schema](delivery-schema.md) -- delivery + keys proto definitions (IDs 200-304)
- [RPC Reference](node-service-schema.md) -- all proto definitions for all 14 files

983
docs/src/wire-format/node-service-schema.md
View File

File diff suppressed because it is too large Load Diff

176
docs/src/wire-format/overview.md
View File

@@ -1,6 +1,6 @@
# Wire Format Overview
This section documents the serialisation pipeline that transforms application-level data structures into encrypted bytes on the wire. Every byte exchanged between quicproquo clients and the server passes through this pipeline, so understanding it is prerequisite to reading the protocol deep dives or the server/client source code.
This section documents the v2 serialisation pipeline that transforms application-level data structures into bytes on the wire. Every byte exchanged between quicproquo clients and the server passes through this pipeline, so understanding it is prerequisite to reading the protocol deep dives or the server and client source code.
---
@@ -9,69 +9,171 @@ This section documents the serialisation pipeline that transforms application-le
Data flows through three stages on the send path. The receive path reverses the order.
```text
Stage 1 Stage 2 Stage 3
-------- -------- --------
Application Cap'n Proto Transport
data serialisation encryption
Stage 1 Stage 2 Stage 3
-------- -------- --------
Application Protobuf Transport
data serialisation encryption
RPC call capnp::serialize QUIC/TLS 1.3
(zero-copy bytes)
RPC call prost::encode() QUIC/TLS 1.3
+ binary frame header
| | |
v v v
Rust structs Canonical byte Encrypted
& method representation ciphertext
invocations (no deserialization on the wire
needed on receive)
| | |
v v v
Rust structs 10-byte (request) or Encrypted
& method 9-byte (response) ciphertext
invocations binary header + on the wire
protobuf payload
```
### Stage 1: Application creates a message or RPC call
At the application layer, the client or server constructs a typed Cap'n Proto message. In the legacy Envelope path (M1), this means building an `Envelope` struct with a `MsgType` discriminant, group ID, sender ID, and opaque payload. In the current NodeService path (M3+), this means invoking a Cap'n Proto RPC method such as `enqueue()` or `fetchKeyPackage()`.
At the application layer, the client or server constructs a typed Protobuf message defined in `proto/qpq/v1/*.proto`. Each RPC method has a corresponding request and response message type.
- **Envelope** (legacy): see [Envelope Schema](envelope-schema.md)
- **NodeService** (current): see [NodeService Schema](node-service-schema.md)
- **AuthenticationService** (standalone): see [Auth Schema](auth-schema.md)
- **DeliveryService** (standalone): see [Delivery Schema](delivery-schema.md)
- **Auth methods** (IDs 100-103): see [Auth Schema](auth-schema.md)
- **Delivery methods** (IDs 200-205): see [Delivery Schema](delivery-schema.md)
- **All methods**: see [Method ID Reference](envelope-schema.md)
- **Full RPC reference**: see [RPC Reference](node-service-schema.md)
### Stage 2: Cap'n Proto serialises to bytes
### Stage 2: Binary framing + Protobuf serialisation
Cap'n Proto converts the in-memory message to its canonical wire representation. This is a **zero-copy** format: the byte layout in memory is identical to the byte layout on the wire. No serialisation or deserialisation pass is required; readers can traverse the bytes in-place using pointer arithmetic.
The v2 protocol defines three frame types, each with a compact binary header followed by a Protobuf-encoded payload. All multi-byte integers are **big-endian**.
The wire representation consists of:
#### Request frame (client to server, bidirectional stream)
1. A **segment table** -- a list of segment sizes encoded as little-endian 32-bit integers.
2. One or more **segments** -- contiguous runs of 8-byte aligned words containing struct data, list data, and far pointers.
```text
0 1 2 3 4 5 6 7 8 9
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| method_id (u16) | request_id (u32) | len (u32) |
+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| len (cont.) | protobuf payload ...
+-------+-------+-------...
```
Cap'n Proto's canonical form is deterministic for a given message, which makes it suitable for signing: two implementations that build the same logical message will produce identical bytes.
| Offset | Field | Type | Description |
|--------|-------|------|-------------|
| 0-1 | `method_id` | `u16 BE` | RPC method identifier |
| 2-5 | `request_id` | `u32 BE` | Client-generated correlation ID |
| 6-9 | `payload_len` | `u32 BE` | Length of the protobuf payload in bytes |
| 10+ | payload | bytes | Protobuf-encoded request message |
Header size: **10 bytes**.
#### Response frame (server to client, same bidirectional stream)
```text
0 1 2 3 4 5 6 7 8
+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| status| request_id (u32) | len (u32) |
+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| protobuf payload ...
+-------...
```
| Offset | Field | Type | Description |
|--------|-------|------|-------------|
| 0 | `status` | `u8` | RPC status code (0 = OK) |
| 1-4 | `request_id` | `u32 BE` | Echoes the request correlation ID |
| 5-8 | `payload_len` | `u32 BE` | Length of the protobuf payload in bytes |
| 9+ | payload | bytes | Protobuf-encoded response message |
Header size: **9 bytes**.
#### Push frame (server to client, QUIC uni-stream)
```text
0 1 2 3 4 5
+-------+-------+-------+-------+-------+-------+
| event_type (u16) | payload_len (u32) |
+-------+-------+-------+-------+-------+-------+
| protobuf payload ...
+-------...
```
| Offset | Field | Type | Description |
|--------|-------|------|-------------|
| 0-1 | `event_type` | `u16 BE` | Push event type identifier |
| 2-5 | `payload_len` | `u32 BE` | Length of the protobuf payload in bytes |
| 6+ | payload | bytes | Protobuf-encoded push event message |
Header size: **6 bytes**.
#### Limits
| Constraint | Value |
|------------|-------|
| Maximum payload size | 4 MiB (4,194,304 bytes) |
| Payloads exceeding this limit | rejected with `PayloadTooLarge` error |
Source: `crates/quicproquo-rpc/src/framing.rs`.
### Stage 3: Transport encryption
The serialised bytes are encrypted by the QUIC/TLS 1.3 transport layer. The QUIC transport uses native QUIC stream framing, which provides its own length delimitation. Cap'n Proto RPC over QUIC relies on the `capnp-rpc` crate's built-in stream adapter.
The framed bytes are encrypted by the QUIC/TLS 1.3 transport layer. QUIC provides stream framing and length delimitation; the RPC framework reads exactly `header_size + payload_len` bytes per frame.
| Transport | Encryption | Authentication |
|---|---|---|
| **QUIC + TLS 1.3** | AES-128-GCM or ChaCha20-Poly1305 (negotiated by TLS) | Server cert (rustls/quinn) |
|-----------|------------|----------------|
| QUIC + TLS 1.3 | AES-128-GCM or ChaCha20-Poly1305 (negotiated by TLS) | Server cert (rustls/quinn) |
The transport layer treats the payload as opaque bytes. It does not inspect or interpret the Cap'n Proto content. This clean separation means the serialisation format can evolve independently of the transport.
The transport layer treats the framed payload as opaque bytes. Serialisation format and transport evolve independently.
---
## QUIC stream model
Each RPC call uses a **dedicated QUIC bidirectional stream**:
1. Client opens a new bidirectional stream.
2. Client sends one request frame and closes the write end.
3. Server reads the request, dispatches by `method_id`, and sends one response frame.
4. Server closes the write end.
Push events (server-initiated) are sent on **QUIC uni-streams** opened by the server. There is no request correlation ID in push frames.
This design allows unlimited concurrent RPCs with no head-of-line blocking.
---
## Connection parameters
| Parameter | Value |
|-----------|-------|
| Protocol | QUIC (RFC 9000) |
| ALPN | `"qpq"` |
| Default port | 5001 |
| TLS version | 1.3 only |
| Certificate | Server presents a TLS certificate; clients verify against a CA cert |
---
## Schema index
The Cap'n Proto schemas that define the wire-level messages are documented on dedicated pages:
Protobuf schemas are defined in `proto/qpq/v1/` and documented on dedicated pages:
| Schema File | Documentation Page | Purpose |
|---|---|---|
| `schemas/envelope.capnp` | [Envelope Schema](envelope-schema.md) | Legacy message envelope (M1) |
| `schemas/auth.capnp` | [Auth Schema](auth-schema.md) | Authentication Service RPC interface |
| `schemas/delivery.capnp` | [Delivery Schema](delivery-schema.md) | Delivery Service RPC interface |
| `schemas/node.capnp` | [NodeService Schema](node-service-schema.md) | Unified node RPC (current) |
| Proto File | Documentation | Purpose |
|------------|---------------|---------|
| `auth.proto` | [Auth Schema](auth-schema.md) | OPAQUE registration and login (IDs 100-103) |
| `delivery.proto` | [Delivery Schema](delivery-schema.md) | Message delivery (IDs 200-205) |
| `keys.proto` | [Delivery Schema](delivery-schema.md) | Key packages and hybrid keys (IDs 300-304, 510-520) |
| `channel.proto` | [RPC Reference](node-service-schema.md) | Channel creation (ID 400) |
| `group.proto` | [RPC Reference](node-service-schema.md) | Group management (IDs 410-413) |
| `moderation.proto` | [RPC Reference](node-service-schema.md) | Content moderation (IDs 420-424) |
| `user.proto` | [RPC Reference](node-service-schema.md) | User resolution (IDs 500-501) |
| `blob.proto` | [RPC Reference](node-service-schema.md) | Blob storage (IDs 600-601) |
| `device.proto` | [RPC Reference](node-service-schema.md) | Device management (IDs 700-702, 710) |
| `recovery.proto` | [RPC Reference](node-service-schema.md) | Account recovery (IDs 750-752) |
| `p2p.proto` | [RPC Reference](node-service-schema.md) | P2P endpoints and health (IDs 800-802) |
| `federation.proto` | [RPC Reference](node-service-schema.md) | Cross-server relay (IDs 900-905) |
| `push.proto` | [RPC Reference](node-service-schema.md) | Push event types (IDs 1000+) |
| `common.proto` | [RPC Reference](node-service-schema.md) | Auth context, account deletion (ID 950) |
Method ID assignment: `crates/quicproquo-proto/src/lib.rs` (`method_ids` module).
---
## Further reading
- [Architecture Overview](../architecture/overview.md) -- system-level view of how services compose
- [Protocol Layers Overview](../protocol-layers/overview.md) -- how transport, framing, and E2E encryption stack
- [ADR-002: Cap'n Proto over MessagePack](../design-rationale/adr-002-capnproto.md) -- why Cap'n Proto was chosen
- [Method ID Reference](envelope-schema.md) -- complete table of all 44 RPC methods
- [Auth Schema](auth-schema.md) -- OPAQUE authentication proto definitions
- [Delivery Schema](delivery-schema.md) -- message delivery proto definitions
- [RPC Reference](node-service-schema.md) -- complete proto definitions for all 14 files