Christian Nennemann
f334ed3d43
Feature 1 — Post-Quantum Hybrid KEM (X25519 + ML-KEM-768): - Create hybrid_kem.rs with keygen, encrypt, decrypt + 11 unit tests - Wire format: version(1) | x25519_eph_pk(32) | mlkem_ct(1088) | nonce(12) | ct - Add uploadHybridKey/fetchHybridKey RPCs to node.capnp schema - Server: hybrid key storage in FileBackedStore + RPC handlers - Client: hybrid keypair in StoredState, auto-wrap/unwrap in send/recv/invite/join - demo-group runs full hybrid PQ envelope round-trip Feature 2 — SQLCipher Persistence: - Extract Store trait from FileBackedStore API - Create SqlStore (rusqlite + bundled-sqlcipher) with encrypted-at-rest SQLite - Schema: key_packages, deliveries, hybrid_keys tables with indexes - Server CLI: --store-backend=sql, --db-path, --db-key flags - 5 unit tests for SqlStore (FIFO, round-trip, upsert, channel isolation) Also includes: client lib.rs refactor, auth config, TOML config file support, mdBook documentation, and various cleanups by user. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
22 KiB
End-to-End Data Flow
This page traces the three core data flows through the quicnprotochat system: registration, group creation, and message exchange. Each flow is illustrated with an ASCII sequence diagram showing control-plane (AS) and data-plane (DS) traffic.
Throughout these flows the server is MLS-unaware -- it stores and forwards opaque byte blobs without parsing their MLS content.
1. Registration Flow
Before a client can join any MLS group, it must generate an Ed25519 identity keypair and upload at least one KeyPackage to the Authentication Service. Peers fetch these KeyPackages to add the client to groups.
Sequence Diagram
Client (Alice) NodeService (AS)
────────────── ────────────────
│ │
│ 1. Generate Ed25519 identity keypair │
│ (IdentityKeypair::generate) │
│ │
│ 2. Generate MLS KeyPackage │
│ (GroupMember::generate_key_package) │
│ - Creates HPKE init keypair │
│ - Embeds Ed25519 pk in credential │
│ - Signs leaf node with Ed25519 sk │
│ - TLS-encodes the KeyPackage │
│ │
│ 3. QUIC connect + TLS 1.3 handshake │
│ ────────────────────────────────────────>│
│ │
│ 4. uploadKeyPackage(identityKey, pkg) │
│ ────────────────────────────────────────>│
│ │ 5. Validate:
│ │ - identityKey == 32 bytes
│ │ - package non-empty, <= 1 MB
│ │ - auth version allowed
│ │
│ │ 6. Compute SHA-256(package)
│ │
│ │ 7. Append to per-identity queue:
│ │ keyPackages[identityKey].push(pkg)
│ │
│ │ 8. Flush keypackages.bin to disk
│ │
│ fingerprint (SHA-256) │
│ <────────────────────────────────────────│
│ │
│ 9. Compare local fingerprint with │
│ server-returned fingerprint │
│ (tamper detection) │
│ │
Key Points
-
KeyPackages are single-use (RFC 9420 requirement). Each
fetchKeyPackagecall atomically removes and returns one package. The client should upload multiple KeyPackages if it expects to be added to several groups. -
The
identityKeyused as the AS index is the raw 32-byte Ed25519 public key, not a fingerprint or hash. Peers must know Alice's public key out-of- band (QR code, directory, etc.) to fetch her KeyPackage. -
The HPKE init private key generated during
generate_key_packageis stored in the client'sDiskKeyStore. The sameGroupMemberinstance (or a restored instance with the same key store) must later calljoin_groupto decrypt the Welcome message. -
The optional hybrid public key (
uploadHybridKey) can also be uploaded during registration for post-quantum envelope encryption.
2. Group Creation Flow
Alice creates a new MLS group, fetches Bob's KeyPackage from the AS, adds Bob to the group (producing a Commit and a Welcome), and delivers the Welcome to Bob via the DS.
Sequence Diagram
Alice NodeService (AS+DS) Bob
───── ────────────────── ───
│ │ │
│ 1. create_group("my-group") │ │
│ (local MLS operation -- │ │
│ Alice is sole member, │ │
│ epoch 0) │ │
│ │ │
│ 2. fetchKeyPackage(bob_pk) │ │
│ ───────────────────────────────>│ │
│ │ 3. Pop bob's KeyPackage │
│ │ from queue (atomic) │
│ bob_kp bytes │ │
│ <───────────────────────────────│ │
│ │ │
│ 4. add_member(bob_kp) │ │
│ Local MLS operations: │ │
│ a. Deserialise & validate │ │
│ Bob's KeyPackage │ │
│ b. Produce Commit message │ │
│ (adds Bob to ratchet │ │
│ tree, advances epoch) │ │
│ c. Produce Welcome message │ │
│ (encrypted to Bob's │ │
│ HPKE init key, contains │ │
│ group secrets + tree) │ │
│ d. merge_pending_commit() │ │
│ (Alice advances to │ │
│ epoch 1 locally) │ │
│ │ │
│ 5. enqueue(bob_pk, welcome) │ │
│ ───────────────────────────────>│ │
│ │ 6. Append welcome to │
│ │ deliveries[(ch, bob_pk)] │
│ │ │
│ │ 7. Notify bob_pk waiters │
│ │ │
│ │ │
│ │ 8. Bob connects and fetches │
│ │ <─────────────────────────────│
│ │ fetch(bob_pk) │
│ │ │
│ │ 9. Drain bob's queue │
│ │ (returns [welcome]) │
│ │ │
│ │ [welcome_bytes] │
│ │ ─────────────────────────────>│
│ │ │
│ │ │ 10. join_group(welcome)
│ │ │ - Decrypt Welcome with
│ │ │ HPKE init private key
│ │ │ - Extract ratchet tree
│ │ │ from GroupInfo ext
│ │ │ - Initialise MlsGroup
│ │ │ at epoch 1
│ │ │
│ │ │ Bob is now a group member
│ │ │
Key Points
-
The Commit message is relevant for groups with more than two members. In the two-party case, Alice is the sole existing member and merges the commit herself. In a multi-member group, the Commit would be sent to all existing members via the DS so they can advance their epoch.
-
The Welcome message is encrypted to Bob's HPKE init key (derived from the KeyPackage). Only the
GroupMemberinstance that generated that KeyPackage holds the corresponding private key. -
The
use_ratchet_tree_extension = trueMLS config embeds the full ratchet tree in the Welcome'sGroupInfoextension. This means Bob does not need a separate tree fetch --new_from_welcomeextracts it automatically. -
The DS routes solely by
recipientKey(Bob's Ed25519 public key). It does not parse the Welcome, the Commit, or any MLS structure.
3. Message Exchange Flow
After both Alice and Bob are group members, they exchange MLS Application messages through the DS.
Sequence Diagram
Alice NodeService (DS) Bob
───── ────────────────── ───
│ │ │
│ ─── Alice sends a message to Bob ─── │
│ │ │
│ 1. send_message("hello bob") │ │
│ MLS create_message(): │ │
│ - Derive message key from │ │
│ epoch secret + gen counter│ │
│ - Encrypt plaintext with │ │
│ AES-128-GCM │ │
│ - Produce MlsMessageOut │ │
│ (PrivateMessage variant) │ │
│ - TLS-encode to bytes │ │
│ │ │
│ 2. enqueue(bob_pk, ciphertext) │ │
│ ───────────────────────────────>│ │
│ │ 3. Store in bob's queue │
│ │ 4. Notify bob_pk waiters │
│ │ │
│ │ (time passes) │
│ │ │
│ │ 5. Bob polls for messages │
│ │ <─────────────────────────────│
│ │ fetchWait(bob_pk, 30000) │
│ │ │
│ │ 6. Drain bob's queue │
│ │ [ciphertext] │
│ │ ─────────────────────────────>│
│ │ │
│ │ │ 7. receive_message(ct)
│ │ │ MLS process_message():
│ │ │ - Identify sender from
│ │ │ PrivateMessage header
│ │ │ - Derive decryption key
│ │ │ from epoch secret
│ │ │ - Decrypt AES-128-GCM
│ │ │ - Return plaintext:
│ │ │ "hello bob"
│ │ │
│ ─── Bob replies to Alice ─── │
│ │ │
│ │ │ 8. send_message("hello alice")
│ │ │ (same MLS encrypt flow)
│ │ │
│ │ 9. enqueue(alice_pk, ct) │
│ │ <─────────────────────────────│
│ │ 10. Store + notify │
│ │ │
│ 11. fetch(alice_pk) │ │
│ ───────────────────────────────>│ │
│ [ciphertext] │ │
│ <───────────────────────────────│ │
│ │ │
│ 12. receive_message(ct) │ │
│ -> "hello alice" │ │
│ │ │
Key Points
-
MLS provides forward secrecy: each message is encrypted with a key derived from the current epoch secret and a per-sender generation counter. Compromising a future key does not reveal past messages.
-
The DS is a dumb relay: it does not decrypt, inspect, or reorder messages. It stores opaque byte blobs in a FIFO queue keyed by recipient.
-
Long-polling via
fetchWaitavoids the need for persistent connections or WebSocket-style push. The client specifies a timeout in milliseconds; the server blocks up to that duration usingtokio::sync::Notify. Therecv --streamCLI flag loopsfetchWaitindefinitely for continuous message reception. -
Channel-aware routing is supported: the
channelIdfield inenqueueandfetchallows scoping queues by channel (e.g., a 16-byte UUID for 1:1 conversations). WhenchannelIdis empty, messages go to the default (legacy) queue.
Control-Plane vs. Data-Plane Summary
┌─────────────────────────────────────────────────────────────────────┐
│ Control Plane (AS) │
│ │
│ uploadKeyPackage ────> Store KeyPackage for identity │
│ fetchKeyPackage <──── Pop and return one KeyPackage │
│ uploadHybridKey ────> Store hybrid PQ public key │
│ fetchHybridKey <──── Return hybrid PQ public key │
│ │
│ Traffic: Infrequent. Once per group join (upload before, │
│ fetch during group add). │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Data Plane (DS) │
│ │
│ enqueue ────> Append payload to recipient queue │
│ fetch <──── Drain and return all queued payloads │
│ fetchWait <──── Long-poll drain with timeout │
│ │
│ Traffic: High-frequency. Every MLS message (Welcome, Commit, │
│ Application) flows through the DS. │
└─────────────────────────────────────────────────────────────────────┘
The separation means the AS can be rate-limited or placed behind stricter access controls without affecting message throughput on the DS.
State Transitions
The following diagram summarises the client-side state machine across all three flows:
┌──────────────┐
│ No State │
└──────┬───────┘
│
IdentityKeypair::generate()
│
▼
┌──────────────┐
│ Identity │ Ed25519 keypair exists
│ Generated │ No KeyPackage, no group
└──────┬───────┘
│
generate_key_package() + uploadKeyPackage()
│
▼
┌──────────────┐
│ Registered │ KeyPackage on AS
│ │ HPKE init key in DiskKeyStore
└──────┬───────┘
│
┌──────────────┴──────────────┐
│ │
create_group() join_group(welcome)
│ │
▼ ▼
┌─────────────┐ ┌──────────────┐
│ Group Owner │ │ Group Member │
│ (epoch 0) │ │ (epoch N) │
└──────┬──────┘ └──────┬───────┘
│ │
add_member() │
│ │
▼ ▼
┌──────────────────────────────────────────┐
│ Active Group Member │
│ │
│ send_message() -> enqueue via DS │
│ receive_message() <- fetch from DS │
│ │
│ Epoch advances on each Commit │
└──────────────────────────────────────────┘
Further Reading
- Architecture Overview -- system diagram and two-service model
- Service Architecture -- RPC method details and long-polling internals
- GroupMember Lifecycle -- detailed MLS state machine
- KeyPackage Exchange Flow -- single-use semantics and AS internals
- MLS (RFC 9420) -- key schedule, ratchet tree, and ciphersuite details
- Forward Secrecy -- how MLS provides forward secrecy
- Post-Compromise Security -- group healing after key compromise