c/quicproquo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add post-quantum hybrid KEM + SQLCipher persistence

Browse Source 
Feature 1 — Post-Quantum Hybrid KEM (X25519 + ML-KEM-768):
- Create hybrid_kem.rs with keygen, encrypt, decrypt + 11 unit tests
- Wire format: version(1) | x25519_eph_pk(32) | mlkem_ct(1088) | nonce(12) | ct
- Add uploadHybridKey/fetchHybridKey RPCs to node.capnp schema
- Server: hybrid key storage in FileBackedStore + RPC handlers
- Client: hybrid keypair in StoredState, auto-wrap/unwrap in send/recv/invite/join
- demo-group runs full hybrid PQ envelope round-trip

Feature 2 — SQLCipher Persistence:
- Extract Store trait from FileBackedStore API
- Create SqlStore (rusqlite + bundled-sqlcipher) with encrypted-at-rest SQLite
- Schema: key_packages, deliveries, hybrid_keys tables with indexes
- Server CLI: --store-backend=sql, --db-path, --db-key flags
- 5 unit tests for SqlStore (FIFO, round-trip, upsert, channel isolation)

Also includes: client lib.rs refactor, auth config, TOML config file support,
mdBook documentation, and various cleanups by user.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Christian Nennemann
2026-02-22 08:07:48 +01:00
parent d1ddef4cea
commit f334ed3d43
81 changed files with 14502 additions and 2289 deletions
Download Patch File Download Diff File Expand all files Collapse all files

107
docs/src/wire-format/overview.md Normal file
View File

@@ -0,0 +1,107 @@
# Wire Format Overview
This section documents the serialisation pipeline that transforms application-level data structures into encrypted bytes on the wire. Every byte exchanged between quicnprotochat clients and the server passes through this pipeline, so understanding it is prerequisite to reading the protocol deep dives or the server/client source code.
---
## Serialisation pipeline
Data flows through four stages on the send path. The receive path reverses the order.
```text
Stage 1 Stage 2 Stage 3 Stage 4
-------- -------- -------- --------
Application Cap'n Proto Length-prefixed Transport
data serialisation framing encryption
ParsedEnvelope capnp::serialize [u32 LE len][payload] Noise ChaCha20-Poly1305
or RPC call (zero-copy bytes) or QUIC/TLS 1.3
| | | |
v v v v
Rust structs Canonical byte Framed byte stream Encrypted
& method representation ready for transport ciphertext
invocations (no deserialization on the wire
needed on receive)
```
### Stage 1: Application creates a message or RPC call
At the application layer, the client or server constructs a typed Cap'n Proto message. In the legacy Envelope path (M1), this means building an `Envelope` struct with a `MsgType` discriminant, group ID, sender ID, and opaque payload. In the current NodeService path (M3+), this means invoking a Cap'n Proto RPC method such as `enqueue()` or `fetchKeyPackage()`.
- **Envelope** (legacy): see [Envelope Schema](envelope-schema.md)
- **NodeService** (current): see [NodeService Schema](node-service-schema.md)
- **AuthenticationService** (standalone): see [Auth Schema](auth-schema.md)
- **DeliveryService** (standalone): see [Delivery Schema](delivery-schema.md)
### Stage 2: Cap'n Proto serialises to bytes
Cap'n Proto converts the in-memory message to its canonical wire representation. This is a **zero-copy** format: the byte layout in memory is identical to the byte layout on the wire. No serialisation or deserialisation pass is required; readers can traverse the bytes in-place using pointer arithmetic.
The wire representation consists of:
1. A **segment table** -- a list of segment sizes encoded as little-endian 32-bit integers.
2. One or more **segments** -- contiguous runs of 8-byte aligned words containing struct data, list data, and far pointers.
Cap'n Proto's canonical form is deterministic for a given message, which makes it suitable for signing: two implementations that build the same logical message will produce identical bytes.
### Stage 3: Length-prefixed framing
Before the serialised bytes enter the transport, they are wrapped in a length-prefixed frame:
```text
+----------------------------+--------------------------------------+
| length (4 bytes, LE u32) | payload (length bytes) |
+----------------------------+--------------------------------------+
```
The length prefix is encoded as a **little-endian** 32-bit unsigned integer. Little-endian was chosen for consistency with Cap'n Proto's own segment table encoding, which also uses little-endian integers. This avoids byte-order confusion when the same buffer contains both framing headers and Cap'n Proto data.
The maximum payload size is **65,535 bytes**, matching the Noise protocol's maximum message size. Frames exceeding this limit are rejected as protocol violations. See [Framing Codec](framing-codec.md) for the full `LengthPrefixedCodec` implementation.
> **Note:** This framing stage applies only to the Noise transport path. The QUIC transport uses native QUIC stream framing, which provides its own length delimitation. Cap'n Proto RPC over QUIC relies on the `capnp-rpc` crate's built-in stream adapter rather than `LengthPrefixedCodec`.
### Stage 4: Transport encryption
The framed byte stream is encrypted by the transport layer:
| Transport | Encryption | Authentication | When Used |
|---|---|---|---|
| **Noise\_XX over TCP** | ChaCha20-Poly1305 (per-session key from XX handshake) | Mutual, via static X25519 keys | M1 stack, peer-to-peer, integration tests |
| **QUIC + TLS 1.3** | AES-128-GCM or ChaCha20-Poly1305 (negotiated by TLS) | Server cert (rustls/quinn) | M3+ primary transport |
In both cases, the transport layer treats the payload as opaque bytes. It does not inspect or interpret the Cap'n Proto content. This clean separation means the serialisation format can evolve independently of the transport.
---
## Little-endian framing rationale
Cap'n Proto uses little-endian encoding for its segment table (the header that precedes each serialised message). The `LengthPrefixedCodec` uses the same byte order for its 4-byte length field. This consistency means:
1. A developer inspecting a raw byte dump sees uniform endianness throughout.
2. On little-endian architectures (x86-64, AArch64 in LE mode), both the framing header and the Cap'n Proto header can be read without byte-swapping.
3. There is no risk of accidentally mixing big-endian and little-endian headers in the same stream.
---
## Schema index
The Cap'n Proto schemas that define the wire-level messages are documented on dedicated pages:
| Schema File | Documentation Page | Purpose |
|---|---|---|
| `schemas/envelope.capnp` | [Envelope Schema](envelope-schema.md) | Legacy message envelope (M1) |
| `schemas/auth.capnp` | [Auth Schema](auth-schema.md) | Authentication Service RPC interface |
| `schemas/delivery.capnp` | [Delivery Schema](delivery-schema.md) | Delivery Service RPC interface |
| `schemas/node.capnp` | [NodeService Schema](node-service-schema.md) | Unified node RPC (current) |
The length-prefixed framing codec that wraps serialised messages is documented at [Framing Codec](framing-codec.md).
---
## Further reading
- [Architecture Overview](../architecture/overview.md) -- system-level view of how services compose
- [Protocol Layers Overview](../protocol-layers/overview.md) -- how transport, framing, and E2E encryption stack
- [ADR-002: Cap'n Proto over MessagePack](../design-rationale/adr-002-capnproto.md) -- why Cap'n Proto was chosen
- [ADR-003: RPC Inside the Noise Tunnel](../design-rationale/adr-003-rpc-inside-noise.md) -- why RPC runs inside the encrypted channel