c/quicproquo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update getting-started and contributing docs for v2

Browse Source 
Remove the capnp compiler requirement from prerequisites (protobuf-src
vendors protoc automatically). Update building.md for 9 crates and the
justfile commands. Rewrite running-the-server.md with accurate v2 flags
(--allow-insecure-auth, --sealed-sender, --plugin-dir, --ws-listen,
--webtransport-listen, --federation-enabled, QPQ_PRODUCTION). Update
docker.md to remove capnproto install from builder stage description.
Delete bot-sdk.md and generators.md (removed crates). Update testing.md
with the accurate 301-test breakdown across 9 crates and the AUTH_LOCK
note for E2E tests. Update coding-standards.md dependency table to list
prost as primary serialisation, capnp as legacy-only, and add opaque-ke.
This commit is contained in:
Christian Nennemann
2026-03-04 22:00:23 +01:00
parent 189534c511
commit f7a7f672b4
8 changed files with 405 additions and 675 deletions
Download Patch File Download Diff File Expand all files Collapse all files

110
docs/src/getting-started/building.md
View File

@@ -1,6 +1,6 @@
# Building from Source
This page covers compiling the workspace, running the test suite, and understanding the build-time Cap'n Proto code generation step.
This page covers compiling the workspace, running the test suite, and the `just` convenience commands available for common development tasks.
---
@@ -12,14 +12,25 @@ From the repository root:
cargo build --workspace
```
This compiles all four crates:
Or using the `just` shortcut:
```bash
just build
```
This compiles all nine crates in the workspace:
| Crate | Type | Purpose |
|---|---|---|
| `quicproquo-core` | library | Crypto primitives, MLS `GroupMember` state machine, hybrid KEM |
| `quicproquo-proto` | library | Cap'n Proto schemas, generated types, envelope serialisation helpers |
| `quicproquo-server` | binary | Unified Authentication + Delivery Service (`NodeService`) |
| `quicproquo-client` | binary | CLI client with subcommands (`ping`, `register`, `send`, `recv`, etc.) |
| `quicproquo-proto` | library | Protobuf schemas (prost), generated types, method ID constants |
| `quicproquo-kt` | library | Key Transparency Merkle log |
| `quicproquo-plugin-api` | library | `#![no_std]` C-ABI plugin interface (`HookVTable`) |
| `quicproquo-rpc` | library | QUIC RPC framing, server dispatcher, client, Tower middleware |
| `quicproquo-sdk` | library | `QpqClient`, event broadcast, `ConversationStore` |
| `quicproquo-server` | binary | Unified Authentication + Delivery Service (`qpq-server`) |
| `quicproquo-client` | binary | CLI client (`qpq`) with REPL and subcommands |
| `quicproquo-p2p` | library | iroh P2P layer (compiled when the `mesh` feature is enabled) |
For a release build with LTO, symbol stripping, and single codegen unit:
@@ -39,55 +50,72 @@ strip = "symbols"
---
## `just` commands
A `justfile` at the repository root provides shortcuts for common tasks:
| Command | Equivalent | Description |
|---|---|---|
| `just build` | `cargo build --workspace` | Build all crates |
| `just test` | `cargo test --workspace` | Run full test suite |
| `just lint` | `cargo clippy --workspace -- -D warnings` | Check for warnings (CI-strict) |
| `just fmt` | `cargo fmt --all -- --check` | Check formatting |
| `just fmt-fix` | `cargo fmt --all` | Auto-format |
| `just proto` | `cargo build -p quicproquo-proto` | Trigger Protobuf codegen |
| `just rpc` | `cargo build -p quicproquo-rpc` | Build RPC framework only |
| `just sdk` | `cargo build -p quicproquo-sdk` | Build client SDK only |
| `just server` | `cargo build -p quicproquo-server` | Build server only |
| `just client` | `cargo build -p quicproquo-client` | Build CLI client only |
| `just clean` | `cargo clean` | Remove build artifacts |
---
## Running the test suite
```bash
just test
# or
cargo test --workspace
```
The test suite includes:
The E2E tests use a shared `AUTH_LOCK` mutex to prevent port conflicts. Run them
with a single thread to avoid flaky failures:
- **`quicproquo-proto`**: Round-trip serialisation tests for Cap'n Proto `Envelope` messages (Ping, Pong, corrupted-input error handling).
- **`quicproquo-core`**: Two-party MLS round-trip (`create_group` / `add_member` / `send_message` / `receive_message`), group ID lifecycle assertions.
- **`quicproquo-client`**: Integration tests for MLS group operations and auth service interactions (require a running server or use in-process mocks).
```bash
cargo test --workspace -- --test-threads 1
```
To run tests for a single crate:
```bash
cargo test -p quicproquo-core
cargo test -p quicproquo-server
cargo test -p quicproquo-rpc
```
---
## Cap'n Proto code generation
## Protobuf code generation
The `quicproquo-proto` crate does not contain hand-written Rust types for wire messages. Instead, its `build.rs` script invokes the `capnp` compiler at build time to generate Rust source from the `.capnp` schema files.
The `quicproquo-proto` crate does not contain hand-written Rust types for wire messages. Instead, its `build.rs` script uses `prost-build` to generate Rust source from the `.proto` schema files in `proto/qpq/v1/`.
### How it works
1. `build.rs` locates the workspace-root `schemas/` directory (two levels above `crates/quicproquo-proto/`).
2. It invokes `capnpc::CompilerCommand` on all four schema files:
- `schemas/envelope.capnp` -- top-level wire envelope with `MsgType` discriminant
- `schemas/auth.capnp` -- `AuthenticationService` RPC interface
- `schemas/delivery.capnp` -- `DeliveryService` RPC interface
- `schemas/node.capnp` -- `NodeService` RPC interface (unified AS + DS)
1. `build.rs` invokes `prost_build::Config::new()` on the 11 schema files under `proto/`.
2. `prost-build` locates the `protoc` binary via the `protobuf-src` crate, which compiles and vendors a compatible `protoc` binary at build time. **No system installation of `protoc` is required.**
3. The generated Rust source is written to `$OUT_DIR` (Cargo's build output directory).
4. `src/lib.rs` includes the generated code via `include!(concat!(env!("OUT_DIR"), "/envelope_capnp.rs"))` and similar macros for each schema.
4. `src/lib.rs` includes the generated code via `include!(concat!(env!("OUT_DIR"), "/..."))` macros.
### Rebuild triggers
The `build.rs` script emits `cargo:rerun-if-changed` directives for each schema file. If you modify a `.capnp` file, the next `cargo build` will automatically re-run code generation.
The `build.rs` script emits `cargo:rerun-if-changed` directives for each `.proto` file. Modifying a schema triggers automatic re-generation on the next `cargo build`.
### Schema include path
The `src_prefix` is set to the `schemas/` directory so that inter-schema imports (e.g., `using Auth = import "auth.capnp".Auth;` inside `node.capnp`) resolve correctly.
### Design constraints of quicproquo-proto
### Design constraints of `quicproquo-proto`
The proto crate is intentionally restricted:
- **No crypto** -- key material never enters this crate.
- **No I/O** -- callers own the transport; this crate only converts bytes to types and back.
- **No I/O** -- callers own the transport; this crate converts bytes to types and back.
- **No async** -- pure synchronous data-layer code.
For details on the wire format, see the [Wire Format Reference](../wire-format/overview.md).
@@ -96,27 +124,9 @@ For details on the wire format, see the [Wire Format Reference](../wire-format/o
## Troubleshooting
### `capnp` binary not found
### Slow first build
**Symptom:**
```
Cap'n Proto schema compilation failed.
Is `capnp` installed? (apt-get install capnproto / brew install capnp)
```
**Fix:** Install the Cap'n Proto compiler for your platform. See [Prerequisites](prerequisites.md) for platform-specific instructions.
Verify it is on your `PATH`:
```bash
which capnp
capnp --version
```
### Version mismatch between `capnp` CLI and `capnpc` Rust crate
The workspace uses `capnpc = "0.19"` (the Rust bindings for the Cap'n Proto compiler). If your system `capnp` binary is significantly older or newer, generated code may be incompatible. The recommended approach is to use a `capnp` binary whose major version matches the `capnpc` crate version. On most systems, the package manager version is compatible.
The first build downloads and compiles all dependencies (including `openmls`, `quinn`, `rustls`, `prost-build`, `protobuf-src`, etc.). This can take several minutes depending on your hardware. The `protobuf-src` compilation step is the most time-consuming on a cold cache. Subsequent builds are incremental and much faster.
### linker errors on macOS with Apple Silicon
@@ -126,14 +136,18 @@ If you see linker errors related to `ring` or `aws-lc-sys` (used transitively by
xcode-select --install
```
### Slow first build
### E2E tests failing with port conflicts
The first build downloads and compiles all dependencies (including `openmls`, `quinn`, `rustls`, `capnp-rpc`, etc.). This can take several minutes depending on your hardware. Subsequent builds are incremental and much faster.
Run E2E tests with `--test-threads 1` to serialise the tests and avoid bind conflicts on the shared test port:
```bash
cargo test -p quicproquo-client --test e2e -- --test-threads 1
```
---
## Next steps
- [Running the Server](running-the-server.md) -- start the NodeService endpoint
- [Running the Server](running-the-server.md) -- start the server endpoint
- [Running the Client](running-the-client.md) -- CLI subcommands and usage examples
- [Docker Deployment](docker.md) -- build and run in containers