feat: add post-quantum hybrid KEM + SQLCipher persistence

Feature 1 — Post-Quantum Hybrid KEM (X25519 + ML-KEM-768): - Create hybrid_kem.rs with keygen, encrypt, decrypt + 11 unit tests - Wire format: version(1) | x25519_eph_pk(32) | mlkem_ct(1088) | nonce(12) | ct - Add uploadHybridKey/fetchHybridKey RPCs to node.capnp schema - Server: hybrid key storage in FileBackedStore + RPC handlers - Client: hybrid keypair in StoredState, auto-wrap/unwrap in send/recv/invite/join - demo-group runs full hybrid PQ envelope round-trip Feature 2 — SQLCipher Persistence: - Extract Store trait from FileBackedStore API - Create SqlStore (rusqlite + bundled-sqlcipher) with encrypted-at-rest SQLite - Schema: key_packages, deliveries, hybrid_keys tables with indexes - Server CLI: --store-backend=sql, --db-path, --db-key flags - 5 unit tests for SqlStore (FIFO, round-trip, upsert, channel isolation) Also includes: client lib.rs refactor, auth config, TOML config file support, mdBook documentation, and various cleanups by user. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 08:07:48 +01:00
parent d1ddef4cea
commit f334ed3d43
81 changed files with 14502 additions and 2289 deletions
--- a/README.md
+++ b/README.md
@@ -8,10 +8,6 @@ and ratcheted group key agreement across any number of participants. Messages
 are framed with **Cap'n Proto**, keeping serialisation zero-copy and
 schema-versioned.

---
-
-## Protocol stack
-
 ```
 ┌─────────────────────────────────────────────┐
 │          Application / MLS ciphertext        │  <- group key ratchet (RFC 9420)
@@ -33,165 +29,64 @@ schema-versioned.

 ---

-## Repository layout
+## Documentation

-```
-quicnprotochat/
-├── crates/
-│   ├── quicnprotochat-core/        # Crypto primitives, QUIC/TLS client helpers, MLS group state machine
-│   │   ├── src/codec.rs     # LengthPrefixedCodec — Tokio Encoder + Decoder
-│   │   ├── src/keypair.rs   # Transport key helpers (X25519, zeroize-on-drop)
-│   │   ├── src/identity.rs  # IdentityKeypair — Ed25519 identity + MLS Signer
-│   │   ├── src/keypackage.rs# generate_key_package — standalone KeyPackage helper
-│   │   └── src/group.rs     # GroupMember — full MLS group lifecycle
-│   │
-│   ├── quicnprotochat-proto/       # Cap'n Proto schemas + generated types + serde helpers
-│   │   └── schemas/ →       # (symlinked to workspace root schemas/)
-│   │
-│   ├── quicnprotochat-server/      # Authentication Service (AS) + Delivery Service (DS) binary
-│   └── quicnprotochat-client/      # CLI client (ping, register, fetch-key, …)
-│
-└── schemas/
-    ├── envelope.capnp       # Top-level wire envelope (MsgType discriminant + payload)
-    ├── auth.capnp           # AuthenticationService RPC (KeyPackage upload / fetch)
-    └── delivery.capnp       # DeliveryService RPC (enqueue / fetch MLS messages)
-```
-
---
-
-## Services
-
-### Node Service (Auth + Delivery) — port 4201
-
-Single QUIC + TLS 1.3 endpoint exposing Cap'n Proto `NodeService` that combines
-Authentication (KeyPackage upload/fetch) and Delivery (enqueue/fetch) operations.
-
-```
-uploadKeyPackage(identityKey: Data, package: Data) -> (fingerprint: Data)
-fetchKeyPackage(identityKey: Data)                 -> (package: Data)
-```
-
-Packages are indexed by the raw Ed25519 public key (32 bytes) and consumed
-exactly once on fetch, matching the MLS single-use KeyPackage requirement.
-
-A simple store-and-forward relay for MLS messages. The server never inspects
-payloads — it routes opaque blobs by recipient public key.
-
-```
-enqueue(recipientKey: Data, payload: Data) -> ()
-fetch(recipientKey: Data)                  -> (payloads: List(Data))
-```
-
-`fetch` atomically drains the entire queue in FIFO order.
-
---
-
-## MLS group lifecycle
-
-```
-GroupMember::new(identity)
-  │
-  ├─ generate_key_package()    → upload bytes to AS
-  │
-  ├─ create_group(group_id)    → epoch 0, sole member
-  │     └─ add_member(kp_bytes)→ (commit_bytes, welcome_bytes)
-  │              ↑                  │            │
-  │         fetched from AS    discard      send to joiner via DS
-  │
-  └─ join_group(welcome_bytes) → joined; ready to encrypt
-        ├─ send_message(plain) → TLS-encoded PrivateMessage → DS
-        └─ receive_message(ct) → Some(plaintext) | None (Commit)
-```
-
-The `OpenMlsRustCrypto` backend is **persistent across calls** on the same
-`GroupMember` instance — it holds the HPKE init private key in its in-memory
-key store between `generate_key_package` and `join_group`.
-
---
-
-## Building
-
-**Prerequisites:**
-
- Rust (stable, 1.77+)
- `capnp` CLI — the Cap'n Proto schema compiler
+Full documentation is available as an **mdBook** wiki in [`docs/`](docs/):

 ```bash
-# Debian / Ubuntu
-apt-get install capnproto
+# Install mdBook (once)
+cargo install mdbook

-# macOS
-brew install capnp
+# Build and serve locally
+mdbook serve docs
+# Open http://localhost:3000
 ```

-**Build everything:**
+### Highlights
+
+- **[Architecture Overview](docs/src/architecture/overview.md)** — Two-service model, dual-key design, crate layout
+- **[Protocol Deep Dives](docs/src/protocol-layers/overview.md)** — QUIC/TLS, Noise_XX, Cap'n Proto, MLS, Hybrid KEM
+- **[Cryptographic Properties](docs/src/cryptography/overview.md)** — Forward secrecy, post-compromise security, PQ readiness, threat model
+- **[Design Rationale](docs/src/design-rationale/overview.md)** — Why MLS over Signal/Matrix, ADRs for all key decisions
+- **[Wire Format Reference](docs/src/wire-format/overview.md)** — Annotated Cap'n Proto schemas
+- **[Getting Started](docs/src/getting-started/prerequisites.md)** — Build, run, demo walkthrough
+- **[Roadmap](docs/src/roadmap/milestones.md)** — Milestones, production readiness, future research
+
+---
+
+## Quick start

 ```bash
+# Prerequisites: Rust 1.77+, capnp CLI
+brew install capnp          # macOS
+# apt-get install capnproto # Debian/Ubuntu
+
+# Build and test
 cargo build --workspace
-```
-
-**Run tests:**
-
-```bash
 cargo test --workspace
-```

---
-
-## Running
-
-**Start the server** (NodeService on :4201):
-
-```bash
+# Start the server (port 7000 by default)
 cargo run -p quicnprotochat-server
-# or with a custom port:
-cargo run -p quicnprotochat-server -- --listen 0.0.0.0:4201
-```

-Current TLS defaults (development): self-signed cert/key written to `data/` if
-missing. Override via CLI flags or env vars:
+# Or via a config file (TOML)
+cat > quicnprotochat-server.toml <<'EOF'
+listen = "0.0.0.0:7000"
+data_dir = "data"
+tls_cert = "data/server-cert.der"
+tls_key = "data/server-key.der"
+auth_token = "devtoken"
+store_backend = "file" # or "sql"
+db_path = "data/quicnprotochat.db"
+db_key = ""
+EOF
+cargo run -p quicnprotochat-server -- --config quicnprotochat-server.toml

-| Purpose | Flag | Env var | Default |
-|---|---|---|---|
-| Listen address | `--listen` | `QUICNPROTOCHAT_LISTEN` | `0.0.0.0:4201` |
-| TLS cert (DER) | `--tls-cert` | `QUICNPROTOCHAT_TLS_CERT` | `data/server-cert.der` |
-| TLS key (DER)  | `--tls-key`  | `QUICNPROTOCHAT_TLS_KEY`  | `data/server-key.der` |
-
-**Client commands:**
-
-```bash
-# Check connectivity
-cargo run -p quicnprotochat-client -- ping
-
-# Generate a fresh identity + KeyPackage, upload to AS
-# Prints your identity_key (hex) — share this with peers
-cargo run -p quicnprotochat-client -- register
-
-# Fetch a peer's KeyPackage (they must have registered first)
-cargo run -p quicnprotochat-client -- fetch-key <64-hex-char identity key>
-
-# Run an end-to-end Alice↔Bob demo against live AS + DS
+# Run the Alice/Bob demo
 cargo run -p quicnprotochat-client -- demo-group \
-  --server 127.0.0.1:4201 \
-  --ds-server 127.0.0.1:4201
-
-# Persistent group CLI (stateful)
-cargo run -p quicnprotochat-client -- register-state --state state.bin --server 127.0.0.1:4201
-cargo run -p quicnprotochat-client -- create-group --state state.bin --group-id my-group
-cargo run -p quicnprotochat-client -- invite --state state.bin --peer-key <peer hex> --server 127.0.0.1:4201 --ds-server 127.0.0.1:4201
-cargo run -p quicnprotochat-client -- join --state state.bin --ds-server 127.0.0.1:4201
-cargo run -p quicnprotochat-client -- send --state state.bin --peer-key <peer hex> --msg "hello" --ds-server 127.0.0.1:4201
-cargo run -p quicnprotochat-client -- recv --state state.bin --ds-server 127.0.0.1:4201
+  --server 127.0.0.1:7000 --ds-server 127.0.0.1:7000
 ```

-Server address defaults to `127.0.0.1:4201`; override with `--server` or
-`QUICNPROTOCHAT_SERVER`. The same endpoint serves both Authentication and
-Delivery.
-
-State file notes: the persisted state stores your identity and MLS group state
-after you have joined. If you generate a KeyPackage (`register-state`) and then
-restart before consuming the Welcome, the join may fail because the HPKE init
-key is not retained; run join in the same session you register.
+See the [full demo walkthrough](docs/src/getting-started/demo-walkthrough.md) for a step-by-step guide.

 ---

@@ -199,45 +94,21 @@ key is not retained; run join in the same session you register.

 | # | Name | Status | What it adds |
 |---|------|--------|--------------|
-| M1 | QUIC/TLS transport | ✅ | QUIC + TLS 1.3 endpoint, length-prefixed framing, Ping/Pong |
-| M2 | Authentication Service | ✅ | Ed25519 identity, KeyPackage generation, AS upload/fetch |
-| M3 | Delivery Service + MLS groups | ✅ | DS relay, `GroupMember` create/join/add/send/recv |
-| M4 | Group CLI subcommands | 🔜 | Persistent CLI (`create-group`, `invite`, `join`, `send`, `recv`); demo-group already available |
-| M5 | Multi-party groups | 🔜 | N > 2 members, Commit fan-out, Proposal handling |
-| M6 | Persistence | 🔜 | SQLite key store, durable group state |
-| M7 | Post-quantum | 🔜 | PQ hybrid for MLS/HPKE |
-
---
-
-## Production hardening roadmap (high level)
-
-1) **Transport & identity**: ACME/Let’s Encrypt, pinned identities, TLS policy
-  hardening, server identity via CA.
-2) **Persistence**: Move AS/DS and MLS state to Postgres; encrypted at rest;
-  retention/TTL and migrations.
-3) **AuthZ & accounts**: User/device accounts (OIDC/passwordless), device
-  binding, revocation/recovery; bind MLS credentials to issued identities.
-4) **Delivery semantics**: Message IDs, idempotent enqueue/fetch, ordering per
-  conversation, backpressure/retries; attachment pipeline via encrypted
-  object storage.
-5) **Observability & ops**: Structured logs with correlation IDs; Prometheus
-  metrics; tracing; alerting + SLOs; audit logs for auth/key events.
-6) **Client resilience**: Reconnect/resume, offline queue, multi-device key
-  handling; key verification UX (QR/safety numbers); recovery flows.
-7) **Security & compliance**: Dependency audits, fuzzing, SAST/DAST, pentest;
-  SBOM/signed releases; PII minimization and retention controls.
+| M1 | QUIC/TLS transport | Done | QUIC + TLS 1.3 endpoint, length-prefixed framing, Ping/Pong |
+| M2 | Authentication Service | Done | Ed25519 identity, KeyPackage generation, AS upload/fetch |
+| M3 | Delivery Service + MLS groups | Done | DS relay, `GroupMember` create/join/add/send/recv |
+| M4 | Group CLI subcommands | Next | Persistent CLI (`create-group`, `invite`, `join`, `send`, `recv`) |
+| M5 | Multi-party groups | Planned | N > 2 members, Commit fan-out, Proposal handling |
+| M6 | Persistence | Planned | SQLite key store, durable group state |
+| M7 | Post-quantum | Planned | PQ hybrid for MLS/HPKE (X25519 + ML-KEM-768) |

 ---

 ## Security notes

- This is a **proof-of-concept**. It has not been audited.
- The server uses a self-signed TLS cert by default; clients trust it via a
-  local DER file. No pinning or CA-based identity is enforced yet.
- MLS credentials use `CredentialType::Basic` (public key only). A real
-  deployment would bind credentials to a certificate authority.
- The Delivery operation does no authentication of the `recipientKey` field —
-  anyone can enqueue for any recipient. Access control is a future milestone.
+This is a **proof-of-concept research project**. It has not been audited.
+See the [threat model](docs/src/cryptography/threat-model.md) for a detailed
+analysis of what is and isn't protected.

 ---