feat: add in-flight RPC tracking, plugin shutdown hooks, and graceful drain

Replace the fixed 30s sleep-based shutdown drain with actual in-flight RPC
tracking using an Arc<AtomicUsize> counter and RAII InFlightGuard. On
SIGTERM/SIGINT the server now:

1. Stops accepting new client and federation connections
2. Sends QUIC CONNECTION_CLOSE with reason "server shutting down"
3. Polls the in-flight counter until it reaches 0 (or drain timeout)
4. Logs drain progress as RPCs complete
5. Calls plugin on_shutdown hooks before exit

Also adds:
- on_shutdown hook to HookVTable (C-ABI plugin API) and ServerHooks trait
- server_in_flight_rpcs Prometheus gauge metric
- Federation connection tracking via shared in-flight counter
This commit is contained in:
2026-03-08 17:56:34 +01:00
parent a05da9b751
commit 66eca065e0
5 changed files with 116 additions and 9 deletions

View File

@@ -56,6 +56,13 @@ pub fn record_storage_latency(operation: &'static str, duration: std::time::Dura
.record(duration.as_secs_f64());
}
// ── In-flight RPCs ────────────────────────────────────────────────────────
/// Record the current number of in-flight RPCs (connections being served).
pub fn record_in_flight_rpcs(count: usize) {
metrics::gauge!("server_in_flight_rpcs").set(count as f64);
}
// ── Server info ────────────────────────────────────────────────────────────
/// Record the server uptime in seconds (set periodically).