c/quicproquo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add post-quantum hybrid KEM + SQLCipher persistence

Browse Source 
Feature 1 — Post-Quantum Hybrid KEM (X25519 + ML-KEM-768):
- Create hybrid_kem.rs with keygen, encrypt, decrypt + 11 unit tests
- Wire format: version(1) | x25519_eph_pk(32) | mlkem_ct(1088) | nonce(12) | ct
- Add uploadHybridKey/fetchHybridKey RPCs to node.capnp schema
- Server: hybrid key storage in FileBackedStore + RPC handlers
- Client: hybrid keypair in StoredState, auto-wrap/unwrap in send/recv/invite/join
- demo-group runs full hybrid PQ envelope round-trip

Feature 2 — SQLCipher Persistence:
- Extract Store trait from FileBackedStore API
- Create SqlStore (rusqlite + bundled-sqlcipher) with encrypted-at-rest SQLite
- Schema: key_packages, deliveries, hybrid_keys tables with indexes
- Server CLI: --store-backend=sql, --db-path, --db-key flags
- 5 unit tests for SqlStore (FIFO, round-trip, upsert, channel isolation)

Also includes: client lib.rs refactor, auth config, TOML config file support,
mdBook documentation, and various cleanups by user.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Christian Nennemann
2026-02-22 08:07:48 +01:00
parent d1ddef4cea
commit f334ed3d43
81 changed files with 14502 additions and 2289 deletions
Download Patch File Download Diff File Expand all files Collapse all files

350
docs/src/architecture/data-flow.md Normal file
View File

@@ -0,0 +1,350 @@
# End-to-End Data Flow
This page traces the three core data flows through the quicnprotochat system:
registration, group creation, and message exchange. Each flow is illustrated
with an ASCII sequence diagram showing control-plane (AS) and data-plane (DS)
traffic.
Throughout these flows the server is **MLS-unaware** -- it stores and forwards
opaque byte blobs without parsing their MLS content.
---
## 1. Registration Flow
Before a client can join any MLS group, it must generate an Ed25519 identity
keypair and upload at least one KeyPackage to the Authentication Service. Peers
fetch these KeyPackages to add the client to groups.
### Sequence Diagram
```text
Client (Alice) NodeService (AS)
────────────── ────────────────
│ │
│ 1. Generate Ed25519 identity keypair │
│ (IdentityKeypair::generate) │
│ │
│ 2. Generate MLS KeyPackage │
│ (GroupMember::generate_key_package) │
│ - Creates HPKE init keypair │
│ - Embeds Ed25519 pk in credential │
│ - Signs leaf node with Ed25519 sk │
│ - TLS-encodes the KeyPackage │
│ │
│ 3. QUIC connect + TLS 1.3 handshake │
│ ────────────────────────────────────────>│
│ │
│ 4. uploadKeyPackage(identityKey, pkg) │
│ ────────────────────────────────────────>│
│ │ 5. Validate:
│ │ - identityKey == 32 bytes
│ │ - package non-empty, <= 1 MB
│ │ - auth version allowed
│ │
│ │ 6. Compute SHA-256(package)
│ │
│ │ 7. Append to per-identity queue:
│ │ keyPackages[identityKey].push(pkg)
│ │
│ │ 8. Flush keypackages.bin to disk
│ │
│ fingerprint (SHA-256) │
│ <────────────────────────────────────────│
│ │
│ 9. Compare local fingerprint with │
│ server-returned fingerprint │
│ (tamper detection) │
│ │
```
### Key Points
- **KeyPackages are single-use** (RFC 9420 requirement). Each `fetchKeyPackage`
call atomically removes and returns one package. The client should upload
multiple KeyPackages if it expects to be added to several groups.
- The `identityKey` used as the AS index is the **raw 32-byte Ed25519 public
key**, not a fingerprint or hash. Peers must know Alice's public key out-of-
band (QR code, directory, etc.) to fetch her KeyPackage.
- The HPKE init private key generated during `generate_key_package` is stored
in the client's `DiskKeyStore`. The **same `GroupMember` instance** (or a
restored instance with the same key store) must later call `join_group` to
decrypt the Welcome message.
- The optional hybrid public key (`uploadHybridKey`) can also be uploaded
during registration for post-quantum envelope encryption.
---
## 2. Group Creation Flow
Alice creates a new MLS group, fetches Bob's KeyPackage from the AS, adds Bob
to the group (producing a Commit and a Welcome), and delivers the Welcome to
Bob via the DS.
### Sequence Diagram
```text
Alice NodeService (AS+DS) Bob
───── ────────────────── ───
│ │ │
│ 1. create_group("my-group") │ │
│ (local MLS operation -- │ │
│ Alice is sole member, │ │
│ epoch 0) │ │
│ │ │
│ 2. fetchKeyPackage(bob_pk) │ │
│ ───────────────────────────────>│ │
│ │ 3. Pop bob's KeyPackage │
│ │ from queue (atomic) │
│ bob_kp bytes │ │
│ <───────────────────────────────│ │
│ │ │
│ 4. add_member(bob_kp) │ │
│ Local MLS operations: │ │
│ a. Deserialise & validate │ │
│ Bob's KeyPackage │ │
│ b. Produce Commit message │ │
│ (adds Bob to ratchet │ │
│ tree, advances epoch) │ │
│ c. Produce Welcome message │ │
│ (encrypted to Bob's │ │
│ HPKE init key, contains │ │
│ group secrets + tree) │ │
│ d. merge_pending_commit() │ │
│ (Alice advances to │ │
│ epoch 1 locally) │ │
│ │ │
│ 5. enqueue(bob_pk, welcome) │ │
│ ───────────────────────────────>│ │
│ │ 6. Append welcome to │
│ │ deliveries[(ch, bob_pk)] │
│ │ │
│ │ 7. Notify bob_pk waiters │
│ │ │
│ │ │
│ │ 8. Bob connects and fetches │
│ │ <─────────────────────────────│
│ │ fetch(bob_pk) │
│ │ │
│ │ 9. Drain bob's queue │
│ │ (returns [welcome]) │
│ │ │
│ │ [welcome_bytes] │
│ │ ─────────────────────────────>│
│ │ │
│ │ │ 10. join_group(welcome)
│ │ │ - Decrypt Welcome with
│ │ │ HPKE init private key
│ │ │ - Extract ratchet tree
│ │ │ from GroupInfo ext
│ │ │ - Initialise MlsGroup
│ │ │ at epoch 1
│ │ │
│ │ │ Bob is now a group member
│ │ │
```
### Key Points
- The **Commit** message is relevant for groups with more than two members. In
the two-party case, Alice is the sole existing member and merges the commit
herself. In a multi-member group, the Commit would be sent to all existing
members via the DS so they can advance their epoch.
- The **Welcome** message is encrypted to Bob's HPKE init key (derived from
the KeyPackage). Only the `GroupMember` instance that generated that
KeyPackage holds the corresponding private key.
- The `use_ratchet_tree_extension = true` MLS config embeds the full ratchet
tree in the Welcome's `GroupInfo` extension. This means Bob does not need a
separate tree fetch -- `new_from_welcome` extracts it automatically.
- The DS routes solely by `recipientKey` (Bob's Ed25519 public key). It does
not parse the Welcome, the Commit, or any MLS structure.
---
## 3. Message Exchange Flow
After both Alice and Bob are group members, they exchange MLS Application
messages through the DS.
### Sequence Diagram
```text
Alice NodeService (DS) Bob
───── ────────────────── ───
│ │ │
│ ─── Alice sends a message to Bob ─── │
│ │ │
│ 1. send_message("hello bob") │ │
│ MLS create_message(): │ │
│ - Derive message key from │ │
│ epoch secret + gen counter│ │
│ - Encrypt plaintext with │ │
│ AES-128-GCM │ │
│ - Produce MlsMessageOut │ │
│ (PrivateMessage variant) │ │
│ - TLS-encode to bytes │ │
│ │ │
│ 2. enqueue(bob_pk, ciphertext) │ │
│ ───────────────────────────────>│ │
│ │ 3. Store in bob's queue │
│ │ 4. Notify bob_pk waiters │
│ │ │
│ │ (time passes) │
│ │ │
│ │ 5. Bob polls for messages │
│ │ <─────────────────────────────│
│ │ fetchWait(bob_pk, 30000) │
│ │ │
│ │ 6. Drain bob's queue │
│ │ [ciphertext] │
│ │ ─────────────────────────────>│
│ │ │
│ │ │ 7. receive_message(ct)
│ │ │ MLS process_message():
│ │ │ - Identify sender from
│ │ │ PrivateMessage header
│ │ │ - Derive decryption key
│ │ │ from epoch secret
│ │ │ - Decrypt AES-128-GCM
│ │ │ - Return plaintext:
│ │ │ "hello bob"
│ │ │
│ ─── Bob replies to Alice ─── │
│ │ │
│ │ │ 8. send_message("hello alice")
│ │ │ (same MLS encrypt flow)
│ │ │
│ │ 9. enqueue(alice_pk, ct) │
│ │ <─────────────────────────────│
│ │ 10. Store + notify │
│ │ │
│ 11. fetch(alice_pk) │ │
│ ───────────────────────────────>│ │
│ [ciphertext] │ │
│ <───────────────────────────────│ │
│ │ │
│ 12. receive_message(ct) │ │
│ -> "hello alice" │ │
│ │ │
```
### Key Points
- **MLS provides forward secrecy**: each message is encrypted with a key
derived from the current epoch secret and a per-sender generation counter.
Compromising a future key does not reveal past messages.
- **The DS is a dumb relay**: it does not decrypt, inspect, or reorder
messages. It stores opaque byte blobs in a FIFO queue keyed by recipient.
- **Long-polling** via `fetchWait` avoids the need for persistent connections
or WebSocket-style push. The client specifies a timeout in milliseconds; the
server blocks up to that duration using `tokio::sync::Notify`. The `recv
--stream` CLI flag loops `fetchWait` indefinitely for continuous message
reception.
- **Channel-aware routing** is supported: the `channelId` field in `enqueue`
and `fetch` allows scoping queues by channel (e.g., a 16-byte UUID for
1:1 conversations). When `channelId` is empty, messages go to the default
(legacy) queue.
---
## Control-Plane vs. Data-Plane Summary
```text
┌─────────────────────────────────────────────────────────────────────┐
│ Control Plane (AS) │
│ │
│ uploadKeyPackage ────> Store KeyPackage for identity │
│ fetchKeyPackage <──── Pop and return one KeyPackage │
│ uploadHybridKey ────> Store hybrid PQ public key │
│ fetchHybridKey <──── Return hybrid PQ public key │
│ │
│ Traffic: Infrequent. Once per group join (upload before, │
│ fetch during group add). │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Data Plane (DS) │
│ │
│ enqueue ────> Append payload to recipient queue │
│ fetch <──── Drain and return all queued payloads │
│ fetchWait <──── Long-poll drain with timeout │
│ │
│ Traffic: High-frequency. Every MLS message (Welcome, Commit, │
│ Application) flows through the DS. │
└─────────────────────────────────────────────────────────────────────┘
```
The separation means the AS can be rate-limited or placed behind stricter
access controls without affecting message throughput on the DS.
---
## State Transitions
The following diagram summarises the client-side state machine across all three
flows:
```text
┌──────────────┐
│ No State │
└──────┬───────┘
│
IdentityKeypair::generate()
│
▼
┌──────────────┐
│ Identity │ Ed25519 keypair exists
│ Generated │ No KeyPackage, no group
└──────┬───────┘
│
generate_key_package() + uploadKeyPackage()
│
▼
┌──────────────┐
│ Registered │ KeyPackage on AS
│ │ HPKE init key in DiskKeyStore
└──────┬───────┘
│
┌──────────────┴──────────────┐
│ │
create_group() join_group(welcome)
│ │
▼ ▼
┌─────────────┐ ┌──────────────┐
│ Group Owner │ │ Group Member │
│ (epoch 0) │ │ (epoch N) │
└──────┬──────┘ └──────┬───────┘
│ │
add_member() │
│ │
▼ ▼
┌──────────────────────────────────────────┐
│ Active Group Member │
│ │
│ send_message() -> enqueue via DS │
│ receive_message() <- fetch from DS │
│ │
│ Epoch advances on each Commit │
└──────────────────────────────────────────┘
```
---
## Further Reading
- [Architecture Overview](overview.md) -- system diagram and two-service model
- [Service Architecture](service-architecture.md) -- RPC method details and long-polling internals
- [GroupMember Lifecycle](../internals/group-member-lifecycle.md) -- detailed MLS state machine
- [KeyPackage Exchange Flow](../internals/keypackage-exchange.md) -- single-use semantics and AS internals
- [MLS (RFC 9420)](../protocol-layers/mls.md) -- key schedule, ratchet tree, and ciphersuite details
- [Forward Secrecy](../cryptography/forward-secrecy.md) -- how MLS provides forward secrecy
- [Post-Compromise Security](../cryptography/post-compromise-security.md) -- group healing after key compromise