feat: M1 — Noise transport, Cap'n Proto framing, Ping/Pong
Establishes the foundational transport layer for noiseml: - Noise_XX_25519_ChaChaPoly_BLAKE2s handshake (initiator + responder) via `snow`; mutual authentication of static X25519 keys guaranteed before any application data flows. - Length-prefixed frame codec (4-byte LE u32, max 65 535 B per Noise spec) implemented as a Tokio Encoder/Decoder pair. - Cap'n Proto Envelope schema with MsgType enum (Ping, Pong, and future MLS message types defined but not yet dispatched). - Server: TCP listener, one Tokio task per connection, Ping→Pong handler, fresh X25519 keypair logged at startup. - Client: `ping` subcommand — handshake, send Ping, receive Pong, print RTT, exit 0. - Integration tests: bidirectional Ping/Pong with mutual-auth verification; server keypair reuse across sequential connections. - Docker multi-stage build (rust:bookworm → debian:bookworm-slim, non-root) and docker-compose with TCP healthcheck. No MLS group state, no AS/DS, no persistence — out of scope for M1.
This commit is contained in:
329
master-prompt.md
Normal file
329
master-prompt.md
Normal file
@@ -0,0 +1,329 @@
|
||||
# noiseml — Master Project Prompt
|
||||
|
||||
## Project Identity
|
||||
|
||||
You are building **noiseml**, a production-grade end-to-end encrypted group messenger in Rust. It uses the MLS protocol (RFC 9420) for group key agreement, ML-KEM-768 (NIST FIPS 203) hybrid post-quantum key exchange, the Noise Protocol Framework (Noise_XX pattern) over raw TCP as the transport layer, and Cap'n Proto for wire serialisation and RPC. There is no TLS, no HTTP, no WebSocket, no MessagePack.
|
||||
|
||||
This is not a prototype. Every milestone produces production-ready, tested, deployable code.
|
||||
|
||||
---
|
||||
|
||||
## Non-Negotiable Engineering Standards
|
||||
|
||||
- **Production-ready only.** No stubs, mocks, `todo!()`, `unimplemented!()`, or placeholder logic in deliverables. If something is out of scope for the current milestone, it is explicitly omitted with a documented reason, not silently stubbed.
|
||||
- **YAGNI / KISS / DRY.** Do not add features, abstractions, or generics that are not required by the current milestone. Favour clarity over cleverness.
|
||||
- **Spec-first.** Document the design (ADR or inline doc comment) before implementing it. Every public API must have a doc comment explaining what it does, its invariants, and any error conditions.
|
||||
- **Security-by-design.** Secrets use `zeroize`. No secret material in logs. No `unwrap()` on cryptographic operations — all errors are typed and propagated. Constant-time comparisons where required.
|
||||
- **Containerised.** The server runs in Docker. `docker-compose.yml` is always kept up to date.
|
||||
- **Dependency hygiene.** Pin major versions. Prefer the `dalek` ecosystem for classical crypto, `snow` for Noise, `openmls` for MLS, `ml-kem` for post-quantum, `capnp`/`capnp-rpc` for serialisation and RPC. Do not introduce new dependencies without justification.
|
||||
- **Review before presenting.** Before presenting any code, review it for: missing error handling, security gaps, incomplete implementations, and deviation from these standards. Fix all issues found before output.
|
||||
|
||||
---
|
||||
|
||||
## Git Standards
|
||||
|
||||
- GPG-signed commits only.
|
||||
- Conventional commits: `feat:`, `fix:`, `chore:`, `docs:`, `test:`, `refactor:`.
|
||||
- Feature branches per milestone: `feat/m1-noise-transport`, `feat/m2-keypackage-as`, etc.
|
||||
- No `Co-authored-by` trailers.
|
||||
- Commit messages describe *why*, not just *what*.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Workspace Layout
|
||||
|
||||
```
|
||||
noiseml/
|
||||
├── Cargo.toml # workspace root
|
||||
├── crates/
|
||||
│ ├── noiseml-core/ # crypto primitives, MLS wrapper, Noise framing codec
|
||||
│ ├── noiseml-proto/ # Cap'n Proto schemas + generated types, no crypto, no I/O
|
||||
│ ├── noiseml-server/ # Delivery Service (DS) + Authentication Service (AS)
|
||||
│ └── noiseml-client/ # CLI client
|
||||
├── schemas/ # .capnp schema files (canonical source of truth)
|
||||
│ ├── envelope.capnp
|
||||
│ ├── auth.capnp
|
||||
│ └── delivery.capnp
|
||||
├── docker/
|
||||
│ └── Dockerfile
|
||||
├── docker-compose.yml
|
||||
└── docs/
|
||||
└── architecture.md
|
||||
```
|
||||
|
||||
### Crate Responsibilities
|
||||
|
||||
**noiseml-core**
|
||||
- Noise_XX handshake initiator and responder (via `snow`)
|
||||
- Length-prefixed Cap'n Proto frame codec (Tokio `Encoder`/`Decoder` traits)
|
||||
- MLS group state machine wrapper around `openmls`
|
||||
- Hybrid PQ ciphersuite (X25519 + ML-KEM-768)
|
||||
- Key generation and zeroize-on-drop key types
|
||||
|
||||
**noiseml-proto**
|
||||
- Cap'n Proto `.capnp` schemas in `schemas/` (workspace root, shared)
|
||||
- `build.rs` invokes `capnpc` to generate Rust types into `src/generated/`
|
||||
- Re-exports generated types with ergonomic builder/reader helpers
|
||||
- Canonical serialisation helpers for signing (uses `capnp::message::Builder::canonicalize()`)
|
||||
- No crypto, no I/O, no async
|
||||
|
||||
**noiseml-server**
|
||||
- Authentication Service: KeyPackage store (DashMap → SQLite at M6)
|
||||
- Delivery Service: Cap'n Proto RPC interface, fan-out router, per-group append-only message log
|
||||
- Tokio TCP listener, Noise handshake per connection, then Cap'n Proto RPC over the encrypted channel
|
||||
- Structured logging (tracing)
|
||||
|
||||
**noiseml-client**
|
||||
- Tokio TCP connection to server
|
||||
- Noise handshake, then Cap'n Proto RPC client stub
|
||||
- CLI interface (clap)
|
||||
- Drives noiseml-core for all crypto operations
|
||||
- Displays received messages to stdout
|
||||
|
||||
### Transport Stack
|
||||
|
||||
```
|
||||
TCP connection
|
||||
└── Noise_XX handshake (snow)
|
||||
└── Authenticated encrypted channel (ChaCha20-Poly1305)
|
||||
└── [u32 frame_len][Cap'n Proto encoded message]
|
||||
└── Cap'n Proto RPC (capnp-rpc, M2+)
|
||||
```
|
||||
|
||||
Both sides hold static X25519 keypairs for the Noise handshake and Ed25519 keypairs for MLS identity. After Noise_XX, mutual authentication is complete. All subsequent frames are Noise-encrypted. Cap'n Proto RPC runs inside the encrypted channel — it has no knowledge of the transport security.
|
||||
|
||||
### Cap'n Proto Schemas
|
||||
|
||||
```capnp
|
||||
# schemas/envelope.capnp
|
||||
@0xDEADBEEFCAFEBABE; # unique file ID (generate with: capnp id)
|
||||
|
||||
struct Envelope {
|
||||
msgType @0 :MsgType;
|
||||
groupId @1 :Data; # 32 bytes, SHA-256 of group name
|
||||
senderId @2 :Data; # 32 bytes, SHA-256 of sender identity key
|
||||
payload @3 :Data; # opaque: MLS blob or control payload
|
||||
timestampMs @4 :UInt64; # unix milliseconds
|
||||
|
||||
enum MsgType {
|
||||
ping @0;
|
||||
pong @1;
|
||||
keyPackageUpload @2;
|
||||
keyPackageFetch @3;
|
||||
keyPackageResponse @4;
|
||||
mlsWelcome @5;
|
||||
mlsCommit @6;
|
||||
mlsApplication @7;
|
||||
error @8;
|
||||
}
|
||||
}
|
||||
|
||||
# schemas/auth.capnp
|
||||
@0xAAAABBBBCCCCDDDD;
|
||||
|
||||
interface AuthenticationService {
|
||||
# Upload a KeyPackage for later retrieval by peers adding this client to a group.
|
||||
# identityKey: Ed25519 public key bytes (32 bytes)
|
||||
# package: openmls-serialised KeyPackage blob
|
||||
# Returns the SHA-256 fingerprint of the package on success.
|
||||
uploadKeyPackage @0 (identityKey :Data, package :Data) -> (fingerprint :Data);
|
||||
|
||||
# Fetch one KeyPackage for a given identity key.
|
||||
# Consuming: the server removes the returned KeyPackage (one-time use, MLS spec).
|
||||
# Returns empty Data if no KeyPackage is available for this identity.
|
||||
fetchKeyPackage @1 (identityKey :Data) -> (package :Data);
|
||||
}
|
||||
|
||||
# schemas/delivery.capnp
|
||||
@0x1111222233334444;
|
||||
|
||||
interface DeliveryService {
|
||||
# Fan out an MLS message to all current members of a group.
|
||||
# groupId: 32-byte group identifier
|
||||
# message: serialised MLSMessage blob
|
||||
# Returns count of recipients the message was queued for.
|
||||
fanOut @0 (groupId :Data, message :Data) -> (recipientCount :UInt32);
|
||||
|
||||
# Subscribe to incoming messages for a group.
|
||||
# memberId: 32-byte identity key fingerprint of the subscribing client.
|
||||
# Returns a capability stream; server pushes MLS blobs as they arrive.
|
||||
subscribe @1 (groupId :Data, memberId :Data) -> (stream :MessageStream);
|
||||
}
|
||||
|
||||
interface MessageStream {
|
||||
# Pull the next available message for this subscriber.
|
||||
# Blocks (promise does not resolve) until a message is available.
|
||||
# sequenceNo is monotonically increasing per group, used for gap detection.
|
||||
next @0 () -> (message :Data, sequenceNo :UInt64);
|
||||
}
|
||||
```
|
||||
|
||||
### MLS Design
|
||||
|
||||
- Ciphersuite: `MLS_128_DHKEMX25519_AES128GCM_SHA256_Ed25519` as baseline (M1–M4), replaced with hybrid PQ ciphersuite at M5.
|
||||
- DS is MLS-unaware: routes `MLSMessage` blobs by `group_id`. Does not inspect epoch or content.
|
||||
- AS stores `KeyPackage` blobs indexed by `(identity_key_fingerprint, package_id)`. One KeyPackage consumed per member-add operation (MLS requirement: KeyPackages are single-use).
|
||||
- Welcome messages routed by the DS to the target client using `sender_id` → `target_id` mapping in the Envelope.
|
||||
- Cap'n Proto canonical form used when serialising any structure that is subsequently signed (MLS Commit signatures, AS KeyPackage fingerprints).
|
||||
|
||||
### Post-Quantum (M5)
|
||||
|
||||
Hybrid KEM construction:
|
||||
```
|
||||
SharedSecret = HKDF-SHA256(
|
||||
ikm = X25519_ss || ML-KEM-768_ss,
|
||||
info = "noiseml-hybrid-v1",
|
||||
len = 32
|
||||
)
|
||||
```
|
||||
Follows the combiner approach from draft-ietf-tls-hybrid-design. Implemented as a custom `openmls` `OpenMlsCryptoProvider` trait implementation in `noiseml-core`.
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
### M1 — Noise Transport ✦ current
|
||||
**Goal:** Two processes complete Noise_XX handshake over TCP and exchange typed Cap'n Proto frames.
|
||||
|
||||
Deliverables:
|
||||
- `schemas/envelope.capnp`: `Envelope` + `MsgType` (Ping/Pong only needed at this stage)
|
||||
- `noiseml-proto`: `build.rs` with `capnpc`, generated type re-exports, canonical helper
|
||||
- `noiseml-core`: static X25519 keypair generation, Noise_XX initiator + responder, length-prefixed Cap'n Proto frame codec
|
||||
- `noiseml-server`: TCP listener, Noise handshake, Ping→Pong handler, one tokio task per connection
|
||||
- `noiseml-client`: connects, Noise handshake, sends Ping, receives Pong, exits 0
|
||||
- Integration test: server and client in same test binary using `tokio::spawn`
|
||||
- `docker-compose.yml` running the server
|
||||
|
||||
### M2 — Authentication Service + KeyPackage Exchange
|
||||
**Goal:** Clients register identity and publish/fetch MLS KeyPackages via Cap'n Proto RPC.
|
||||
|
||||
Deliverables:
|
||||
- `schemas/auth.capnp`: `AuthenticationService` interface
|
||||
- `noiseml-proto`: generated RPC stubs + client/server bootstrap helpers
|
||||
- `noiseml-core`: MLS KeyPackage generation (openmls)
|
||||
- `noiseml-server`: AS RPC server implementation with DashMap store
|
||||
- `noiseml-client`: `register` and `fetch-key` CLI subcommands
|
||||
- Test: Alice uploads KeyPackage, Bob fetches it, fingerprints match
|
||||
|
||||
### M3 — MLS Group Create + Welcome
|
||||
**Goal:** Alice creates a group and adds Bob via MLS Welcome. Both hold valid epoch 1 state.
|
||||
|
||||
Deliverables:
|
||||
- `schemas/delivery.capnp`: `DeliveryService` + `MessageStream` interfaces
|
||||
- `noiseml-core`: group create, add member, process Welcome
|
||||
- `noiseml-server`: DS RPC server, Welcome routing by identity
|
||||
- `noiseml-client`: `create-group` and `join` CLI subcommands
|
||||
- Test: two clients reach identical epoch 1 group state, verified by comparing group context hashes
|
||||
|
||||
### M4 — Encrypted Group Messaging
|
||||
**Goal:** Alice and Bob exchange MLS Application messages through the DS.
|
||||
|
||||
Deliverables:
|
||||
- `noiseml-core`: send/receive application message, epoch rotation on Commit
|
||||
- `noiseml-server`: DS fan-out via `MessageStream` capability stream, per-group ordered log (in-memory)
|
||||
- `noiseml-client`: `send` subcommand, live receive loop via `MessageStream.next()`
|
||||
- Test: round-trip message integrity, forward secrecy verified by confirming distinct key material across epochs
|
||||
|
||||
### M5 — Hybrid PQ Ciphersuite
|
||||
**Goal:** Replace MLS crypto backend with X25519 + ML-KEM-768 hybrid.
|
||||
|
||||
Deliverables:
|
||||
- `noiseml-core`: custom `OpenMlsCryptoProvider` with hybrid KEM
|
||||
- All M3/M4 tests pass unchanged with new ciphersuite
|
||||
- Criterion benchmarks: key generation, encap/decap, group-add latency (10/100/1000 members)
|
||||
|
||||
### M6 — Persistence + Production Docker
|
||||
**Goal:** Server survives restart. Full containerised deployment.
|
||||
|
||||
Deliverables:
|
||||
- `noiseml-server`: SQLite via `sqlx` for AS key store and DS message log, `migrations/` directory
|
||||
- `docker/Dockerfile`: multi-stage build (rust:bookworm builder → debian:bookworm-slim runtime)
|
||||
- `docker-compose.yml`: server + SQLite volume, healthcheck
|
||||
- Client reconnect with session resume (re-handshake + rejoin group epoch from DS log)
|
||||
|
||||
---
|
||||
|
||||
## Dependencies (pinned majors)
|
||||
|
||||
```toml
|
||||
# Crypto
|
||||
openmls = "0.6"
|
||||
openmls_rust_crypto = "0.6"
|
||||
ml-kem = "0.3"
|
||||
x25519-dalek = "2"
|
||||
ed25519-dalek = "2"
|
||||
snow = "0.9"
|
||||
chacha20poly1305 = "0.10"
|
||||
sha2 = "0.10"
|
||||
hkdf = "0.12"
|
||||
zeroize = { version = "1", features = ["derive"] }
|
||||
rand = "0.8"
|
||||
|
||||
# Serialisation + RPC
|
||||
capnp = "0.19"
|
||||
capnp-rpc = "0.19"
|
||||
|
||||
# Build-time only
|
||||
capnpc = "0.19" # build-dependency in noiseml-proto
|
||||
|
||||
# Async / networking
|
||||
tokio = { version = "1", features = ["full"] }
|
||||
tokio-util = { version = "0.7", features = ["codec"] }
|
||||
|
||||
# Server utilities
|
||||
dashmap = "5"
|
||||
sqlx = { version = "0.7", features = ["sqlite", "runtime-tokio"] } # M6+
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
anyhow = "1"
|
||||
thiserror = "1"
|
||||
|
||||
# CLI
|
||||
clap = { version = "4", features = ["derive"] }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Design Decisions (ADR Summary)
|
||||
|
||||
**ADR-001: Noise_XX for transport authentication**
|
||||
Both parties hold static keys registered with the AS. XX pattern provides mutual authentication and identity hiding for the initiator. No TLS dependency, no certificate infrastructure.
|
||||
|
||||
**ADR-002: Cap'n Proto replaces MessagePack**
|
||||
Cap'n Proto provides: zero-copy reads, schema-enforced types, canonical serialisation for cryptographic signing, and a built-in async RPC system (capnp-rpc) that eliminates hand-rolled message dispatch. The build-time codegen overhead is accepted as worthwhile.
|
||||
|
||||
**ADR-003: Cap'n Proto RPC runs inside the Noise tunnel**
|
||||
Cap'n Proto RPC has no transport security of its own. It operates over the byte stream produced by the Noise session. Separation of concerns: Noise owns authentication and confidentiality, Cap'n Proto owns framing and dispatch.
|
||||
|
||||
**ADR-004: DS is MLS-unaware**
|
||||
The Delivery Service routes opaque `MLSMessage` blobs by `group_id`. It never decrypts or inspects MLS content. This is the correct MLS architecture (RFC 9420 §4) and is a natural Audit-Core integration point: the DS log is an append-only sequence of authenticated blobs.
|
||||
|
||||
**ADR-005: Single-use KeyPackages**
|
||||
MLS requires that each KeyPackage be used at most once (to preserve forward secrecy of the initial key exchange). The AS consumes a KeyPackage on fetch. Clients should pre-upload multiple KeyPackages. The AS warns when a client's supply runs low (M2+).
|
||||
|
||||
**ADR-006: PQ gap in Noise transport is accepted**
|
||||
The MLS content layer is PQ-protected from M5. The Noise transport (X25519) remains classical — PQ-Noise (draft-noise-pq) is not yet supported by `snow`. Harvest-now-decrypt-later against the handshake metadata is an accepted residual risk for M1–M5. No long-lived content secrets transit the Noise handshake, so the practical impact is limited to identity/timing metadata.
|
||||
|
||||
---
|
||||
|
||||
## How to Use This Prompt
|
||||
|
||||
Paste this document at the start of any session working on noiseml. Then state which milestone you are working on and what specific task you need. The assistant will:
|
||||
|
||||
1. Confirm the current milestone and task.
|
||||
2. State any design decisions being made (ADR format if significant).
|
||||
3. Produce complete, production-ready code for the task.
|
||||
4. Review the code internally for gaps before presenting.
|
||||
5. State what the next logical task is.
|
||||
|
||||
When asking for code, always specify:
|
||||
- Which crate(s) are affected.
|
||||
- Whether this is a new file or modification to existing.
|
||||
- Any constraints or context the assistant may not have (e.g. existing types already defined).
|
||||
|
||||
---
|
||||
|
||||
*noiseml — MLS + Post-Quantum + Noise/TCP + Cap'n Proto messenger in Rust*
|
||||
*Architecture version: 1.1 | Last updated: 2026-02-19*
|
||||
Reference in New Issue
Block a user