c/quicproquo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update getting-started and contributing docs for v2

Browse Source 
Remove the capnp compiler requirement from prerequisites (protobuf-src
vendors protoc automatically). Update building.md for 9 crates and the
justfile commands. Rewrite running-the-server.md with accurate v2 flags
(--allow-insecure-auth, --sealed-sender, --plugin-dir, --ws-listen,
--webtransport-listen, --federation-enabled, QPQ_PRODUCTION). Update
docker.md to remove capnproto install from builder stage description.
Delete bot-sdk.md and generators.md (removed crates). Update testing.md
with the accurate 301-test breakdown across 9 crates and the AUTH_LOCK
note for E2E tests. Update coding-standards.md dependency table to list
prost as primary serialisation, capnp as legacy-only, and add opaque-ke.
This commit is contained in:
Christian Nennemann
2026-03-04 22:00:23 +01:00
parent 189534c511
commit f7a7f672b4
8 changed files with 405 additions and 675 deletions
Download Patch File Download Diff File Expand all files Collapse all files

174
docs/src/contributing/testing.md
View File

@@ -1,8 +1,8 @@
# Testing Strategy
This page describes the testing structure, conventions, and current coverage for
quicproquo. All tests run with `cargo test --workspace` and must pass before
any code is merged.
quicproquo. All tests run with `cargo test --workspace` (or `just test`) and
must pass before any code is merged.
For the coding standards that tests must follow, see
[Coding Standards](coding-standards.md).
@@ -17,55 +17,103 @@ Unit tests live alongside the code they test, in `#[cfg(test)] mod tests` blocks
at the bottom of each source file. They test individual functions and types in
isolation.
**quicproquo-core:**
**quicproquo-core (96 tests):**
| Module | Tests | What they cover |
|--------|-------|----------------|
| `codec` | 7 tests | Length-prefixed frame encoding/decoding, edge cases (empty payload, max size, partial frame, exact boundary) |
| `keypair` | 3 tests | Ed25519 keypair generation, public key extraction, deterministic re-derivation |
| `group` | 2 tests | Group round-trip (create + add + join + send + recv), group\_id lifecycle |
| `hybrid_kem` | 11 tests | Encapsulate/decapsulate round-trip, key generation, combiner correctness, wrong-key rejection, serialisation |
| `codec` | 7 | Length-prefixed frame encoding/decoding, edge cases (empty payload, max size, partial frame, exact boundary) |
| `keypair` | 3 | Ed25519 keypair generation, public key extraction, deterministic re-derivation |
| `group` | 2 | Group round-trip (create + add + join + send + recv), group\_id lifecycle |
| `hybrid_kem` | 11 | Encapsulate/decapsulate round-trip, key generation, combiner correctness, wrong-key rejection, serialisation |
| `opaque_auth` | 12 | OPAQUE registration + login full flow, bad password rejection |
| `mls_*` | 61 | MLS key schedule, member add/remove, Welcome processing, key exhaustion |
**quicproquo-proto:**
**quicproquo-rpc (18 tests):**
| Module | Tests | What they cover |
|--------|-------|----------------|
| `lib` | 3 tests | Cap'n Proto builder/reader round-trip, canonical serialisation, schema validation |
| `framing` | 8 | Wire framing round-trips, method ID encoding, length-prefix correctness |
| `dispatch` | 10 | Handler dispatch, method not found, middleware chain, timeout enforcement |
### Integration Tests
**quicproquo-sdk (30 tests):**
Integration tests live in `crates/quicproquo-client/tests/` and test the
full client-server interaction. Each test spawns a server using `tokio::spawn`
within the same test binary, then runs client operations against it.
| Module | Tests | What they cover |
|--------|-------|----------------|
| `client` | 15 | `QpqClient` connect, send, receive, event broadcast |
| `conversation_store` | 15 | `ConversationStore` CRUD, pagination, message ordering |
| File | Milestone | What it covers |
|------|-----------|---------------|
| `auth_service.rs` | M2 | KeyPackage upload via AS, KeyPackage fetch (single-use consume semantics), identity key validation |
| `mls_group.rs` | M3 | Full MLS round-trip: register state, create group, add member via Welcome, send encrypted message, receive and decrypt |
**quicproquo-server (65 tests):**
| Module | Tests | What they cover |
|--------|-------|----------------|
| `auth` | 20 | OPAQUE registration, login, session management, rate limiting |
| `node_service` | 20 | KeyPackage upload/fetch, message enqueue/deliver, sealed sender |
| `storage` | 15 | `FileBackedStore` and `SqlStore` CRUD, MLS entity serialisation |
| `federation` | 10 | Federation peer relay, mTLS validation, domain routing |
**quicproquo-kt (21 tests):**
| Module | Tests | What they cover |
|--------|-------|----------------|
| `merkle_log` | 21 | Merkle tree insertion, consistency proofs, root hash correctness |
**quicproquo-p2p (34 tests):**
| Module | Tests | What they cover |
|--------|-------|----------------|
| iroh mesh | 34 | P2P peer discovery, relay, mesh join/leave |
### Integration and E2E Tests
E2E tests live in `crates/quicproquo-client/tests/e2e.rs` (20 tests) and
exercise the full client-server stack in-process. Each test spawns a real server
using `tokio::spawn`, runs client operations against it, and asserts on the
results.
**quicproquo-client unit (16 tests):**
| File | What it covers |
|------|---------------|
| `src/lib.rs` | CLI command parsing, client state machine, error formatting |
**quicproquo-client E2E (20 tests):**
| Test | What it covers |
|------|---------------|
| `auth_failure` | Rejected OPAQUE login (wrong password) |
| `message_ordering` | Sequential message delivery order preserved |
| `opaque_flow` | Full OPAQUE registration + login round-trip |
| `key_exhaustion` | Behaviour when KeyPackage queue is empty |
| `rate_limit` | Rate limiting rejects excess requests |
| `mls_group_round_trip` | Full MLS group: create, add member, send, receive |
| `keypackage_single_use` | KeyPackage consumed on first fetch |
| and 13 more | Additional protocol scenarios |
### Test Pattern
All integration tests follow the same pattern:
All E2E tests follow the same pattern:
```rust
#[tokio::test]
async fn test_something() {
// 1. Start server in background
// 1. Acquire shared lock to avoid port conflicts
let _lock = AUTH_LOCK.lock().await;
// 2. Start server in background
let server_handle = tokio::spawn(async move {
server::run(config).await.unwrap();
server::run(config).await.expect("server failed");
});
// 2. Wait for server to be ready
// 3. Wait for server to be ready
tokio::time::sleep(Duration::from_millis(100)).await;
// 3. Run client operations
// 4. Run client operations
let result = client::do_something(server_addr).await;
// 4. Assert
// 5. Assert
assert!(result.is_ok());
// ...
// 5. Cleanup
// 6. Cleanup
server_handle.abort();
}
```
@@ -80,25 +128,41 @@ server process.
### Full Workspace
```bash
just test
# or
cargo test --workspace
```
This runs all unit tests and integration tests across all four crates.
This runs all unit tests and integration tests across all nine crates (301 tests total).
### E2E Tests (serialised)
The E2E test suite shares an `AUTH_LOCK` `tokio::Mutex` to prevent port binding
conflicts when tests run in parallel. Always run E2E tests with a single thread:
```bash
cargo test -p quicproquo-client --test e2e -- --test-threads 1
```
Running without `--test-threads 1` may cause intermittent bind errors if two
tests try to use the same port concurrently.
### Single Crate
```bash
cargo test -p quicproquo-core
cargo test -p quicproquo-proto
cargo test -p quicproquo-rpc
cargo test -p quicproquo-sdk
cargo test -p quicproquo-server
cargo test -p quicproquo-client
cargo test -p quicproquo-kt
cargo test -p quicproquo-p2p
```
### Single Test
```bash
cargo test -p quicproquo-core -- codec::tests::test_round_trip
cargo test -p quicproquo-client --test mls_group
cargo test -p quicproquo-client --test e2e -- opaque_flow --test-threads 1
```
### With Output
@@ -111,17 +175,18 @@ cargo test --workspace -- --nocapture
## Current Results
All tests pass as of the M3 milestone on branch `feat/m1-noise-transport`.
All 301 tests pass on branch `v2`.
Summary:
| Crate | Unit Tests | Integration Tests | Total |
|-------|-----------|-------------------|-------|
| `quicproquo-core` | 23 | -- | 23 |
| `quicproquo-proto` | 3 | -- | 3 |
| `quicproquo-server` | 0 | -- | 0 |
| `quicproquo-client` | 0 | 5 | 5 |
| **Total** | **26** | **5** | **31** |
| Crate | Unit / Integration Tests | E2E Tests | Total |
|-------|--------------------------|-----------|-------|
| `quicproquo-core` | 96 | -- | 96 |
| `quicproquo-rpc` | 18 | -- | 18 |
| `quicproquo-sdk` | 30 | -- | 30 |
| `quicproquo-server` | 65 | -- | 65 |
| `quicproquo-kt` | 21 | -- | 21 |
| `quicproquo-p2p` | 34 | -- | 34 |
| `quicproquo-client` | 16 unit + 1 doctest | 20 | 37 |
| **Total** | **281** | **20** | **301** |
---
@@ -155,37 +220,41 @@ fn fetch_consumes_keypackage_single_use() { ... }
Tests must not depend on external services, network access, or filesystem state
outside the test's temporary directory. The `tokio::spawn` pattern for
client-server tests ensures everything runs in-process.
E2E tests ensures everything runs in-process.
### Determinism
Tests must be deterministic. If randomness is needed (e.g., key generation),
the test must not depend on specific random values -- only on the properties of
the test must not depend on specific random values — only on the properties of
the output (correct length, successful round-trip, etc.).
### No `.unwrap()` in Test Setup
`.unwrap()` is acceptable in test assertions, but test setup that fails silently
is not. Use `expect("descriptive message")` on setup operations so failures
report clearly.
---
## Planned Testing Enhancements
The following testing improvements are planned for future milestones:
### Fuzzing Targets (M5+)
Fuzz testing for parser and deserialisation code:
- **Cap'n Proto message parser:** Feed arbitrary bytes to the Cap'n Proto reader
and verify it either parses correctly or returns a typed error (no panics,
no undefined behaviour).
- **Protobuf message parser:** Feed arbitrary bytes to `prost::Message::decode`
on each generated type and verify it either parses correctly or returns a
typed error (no panics, no undefined behaviour).
- **MLS message handler:** Feed arbitrary `MLSMessage` bytes to the
`GroupMember::receive_message` path.
Tool: `cargo-fuzz` with `libfuzzer`.
### Golden-Wire Fixtures (M5+)
Serialised test vectors for regression testing across versions:
- Capture the wire bytes of known-good Cap'n Proto messages (Envelope, Auth,
Delivery structs) at the current version.
- Capture the wire bytes of known-good Protobuf messages at the current version.
- Store as `.bin` files in `tests/fixtures/`.
- Each test deserialises the fixture and verifies the expected field values.
- When the wire format changes, fixtures are updated with a version bump.
@@ -200,8 +269,7 @@ version N-1 (and vice versa):
- Build two versions of the binary (current and previous release).
- Run the older server with the newer client and verify all RPCs succeed.
- Run the newer server with the older client and verify graceful degradation
(legacy mode works, new features return clean errors).
- Run the newer server with the older client and verify graceful degradation.
### Criterion Benchmarks (M5)
@@ -210,15 +278,14 @@ Performance benchmarks using [Criterion.rs](https://docs.rs/criterion/):
- Key generation latency (Ed25519, X25519, ML-KEM-768).
- MLS encap/decap (KeyPackage generation, Welcome processing).
- Group-add latency scaling: 2, 10, 100, 1000 members.
- Cap'n Proto serialise/deserialise throughput.
- Protobuf serialise/deserialise throughput.
Benchmarks run separately from tests (`cargo bench`) and are not part of the
CI gate, but are tracked for regression detection.
### Docker-based E2E Tests (Phase 5)
End-to-end tests using `testcontainers-rs` (see
[Future Research: Testcontainers-rs](../roadmap/future-research.md#testcontainers-rs)):
End-to-end tests using `testcontainers-rs`:
- Spin up server container from the Docker image.
- Run client operations from the test process against the containerised server.
@@ -232,4 +299,3 @@ End-to-end tests using `testcontainers-rs` (see
- [Coding Standards](coding-standards.md) -- quality requirements for test code
- [Milestones](../roadmap/milestones.md) -- which tests were added at each milestone
- [Production Readiness WBS](../roadmap/production-readiness.md) -- Phase 5 (E2E Harness and Security Tests)
- [Future Research: Testcontainers-rs](../roadmap/future-research.md#testcontainers-rs) -- Docker-based testing