Platform upgrade: semantic search, citations, readiness, tests, Docker

Major features added by 5 parallel agent teams:
- Semantic "Ask" (NL queries via FTS5 + embeddings + Claude synthesis)
- Global search across drafts, ideas, authors, gaps
- REST API expansion (14 endpoints, up from 3) with CSV/JSON export
- Citation graph visualization (D3.js, 440 nodes, 2422 edges)
- Standards readiness scoring (0-100 composite from 6 factors)
- Side-by-side draft comparison view with shared/unique analysis
- Annotation system (notes + tags per draft, DB-persisted)
- Docker deployment (Dockerfile + docker-compose with Ollama)
- Scheduled updates (cron script with log rotation)
- Pipeline health dashboard (stage progress bars, cost tracking)
- Test suite foundation (54 pytest tests covering DB, models, web data)

Fixes: compare_drafts() stubbed→working, get_authors_for_draft() bug,
source-aware analysis prompts, config env var overrides + validation,
resilient batch error handling with --retry-failed, observatory --dry-run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-07 20:52:56 +01:00
parent da2a989744
commit 757b781c67
33 changed files with 4253 additions and 170 deletions

View File

@@ -0,0 +1,153 @@
{% extends "base.html" %}
{% set active_page = "ask" %}
{% block title %}Ask — IETF Draft Analyzer{% endblock %}
{% block extra_head %}
<style>
.ask-input {
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
backdrop-filter: blur(10px);
}
.answer-card {
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
backdrop-filter: blur(10px);
}
.source-row {
transition: all 0.15s ease;
}
.source-row:hover {
background: rgba(59, 130, 246, 0.05);
}
.loading-spinner {
border: 3px solid rgba(59, 130, 246, 0.2);
border-top-color: #3b82f6;
border-radius: 50%;
width: 24px;
height: 24px;
animation: spin 0.8s linear infinite;
display: inline-block;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
</style>
{% endblock %}
{% block content %}
<!-- Header -->
<div class="mb-8 text-center">
<h1 class="text-3xl font-bold text-white">Ask the Draft Corpus</h1>
<p class="text-slate-400 text-sm mt-2">Ask natural language questions about IETF AI/agent drafts. Answers are synthesized from the most relevant documents.</p>
</div>
<!-- Search Bar -->
<div class="max-w-3xl mx-auto mb-8">
<form method="get" action="/ask" id="askForm">
<div class="ask-input rounded-xl border border-slate-700 p-2 flex items-center gap-2">
<svg class="w-5 h-5 text-slate-500 ml-3 flex-shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M8.228 9c.549-1.165 2.03-2 3.772-2 2.21 0 4 1.343 4 3 0 1.4-1.278 2.575-3.006 2.907-.542.104-.994.54-.994 1.093m0 3h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z"/>
</svg>
<input type="text" name="q" value="{{ question }}" placeholder="Which drafts address agent authentication? What approaches exist for agent delegation?"
class="flex-1 bg-transparent border-0 px-3 py-3 text-base text-slate-200 placeholder-slate-500 focus:outline-none"
autofocus>
<button type="submit" class="px-6 py-3 bg-blue-600 text-white rounded-lg text-sm font-medium hover:bg-blue-500 transition-colors flex-shrink-0">
Ask
</button>
</div>
<div class="flex items-center gap-4 mt-3 px-2">
<label class="text-xs text-slate-500 flex items-center gap-1.5">
<span>Sources:</span>
<select name="top" class="bg-slate-800/60 border border-slate-700 rounded px-2 py-1 text-xs text-slate-300 focus:outline-none">
<option value="3" {% if request.args.get('top', '5') == '3' %}selected{% endif %}>3</option>
<option value="5" {% if request.args.get('top', '5') == '5' %}selected{% endif %}>5</option>
<option value="10" {% if request.args.get('top', '5') == '10' %}selected{% endif %}>10</option>
</select>
</label>
<div class="text-xs text-slate-600">Combines keyword search + semantic similarity</div>
</div>
</form>
</div>
<!-- Example questions (show when no query) -->
{% if not question %}
<div class="max-w-3xl mx-auto">
<div class="text-xs text-slate-500 uppercase tracking-wide mb-3 font-medium">Example questions</div>
<div class="grid grid-cols-1 md:grid-cols-2 gap-2">
{% set examples = [
"Which drafts address agent authentication and identity?",
"What are the competing approaches to agent-to-agent communication?",
"How do safety mechanisms work across different proposals?",
"What protocols exist for AI model serving and inference?",
"Which drafts propose agent discovery or registration systems?",
"What are the main gaps in autonomous network operations?",
] %}
{% for q in examples %}
<a href="/ask?q={{ q | urlencode }}" class="ask-input rounded-lg border border-slate-800 px-4 py-3 text-sm text-slate-400 hover:text-blue-400 hover:border-slate-700 transition">
{{ q }}
</a>
{% endfor %}
</div>
</div>
{% endif %}
<!-- Answer -->
{% if result %}
<div class="max-w-3xl mx-auto">
<!-- Synthesized answer -->
<div class="answer-card rounded-xl border border-slate-800 p-6 mb-6">
<div class="flex items-center gap-2 mb-4">
<svg class="w-5 h-5 text-blue-400" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9.663 17h4.673M12 3v1m6.364 1.636l-.707.707M21 12h-1M4 12H3m3.343-5.657l-.707-.707m2.828 9.9a5 5 0 117.072 0l-.548.547A3.374 3.374 0 0014 18.469V19a2 2 0 11-4 0v-.531c0-.895-.356-1.754-.988-2.386l-.548-.547z"/>
</svg>
<h2 class="text-lg font-semibold text-white">Answer</h2>
</div>
<div class="text-slate-300 text-sm leading-relaxed whitespace-pre-line">{{ result.answer }}</div>
</div>
<!-- Source drafts -->
{% if result.sources %}
<div class="answer-card rounded-xl border border-slate-800 overflow-hidden">
<div class="px-6 py-4 border-b border-slate-800">
<h3 class="text-sm font-semibold text-slate-300">Source Drafts ({{ result.sources|length }})</h3>
</div>
<table class="w-full text-sm">
<thead>
<tr class="border-b border-slate-800/50 bg-slate-900/40">
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-500 uppercase w-8">#</th>
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-500 uppercase">Draft</th>
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-500 uppercase w-20">Match</th>
<th class="px-4 py-2.5 text-right text-xs font-medium text-slate-500 uppercase w-16">Score</th>
</tr>
</thead>
<tbody class="divide-y divide-slate-800/30">
{% for src in result.sources %}
<tr class="source-row">
<td class="px-4 py-3 text-slate-600">{{ loop.index }}</td>
<td class="px-4 py-3">
<a href="/drafts/{{ src.name }}" class="text-blue-400 hover:text-blue-300 font-medium transition">
{{ src.title }}
</a>
<div class="text-xs text-slate-600 mt-0.5 font-mono">{{ src.name }}</div>
</td>
<td class="px-4 py-3">
{% if src.match_type == 'both' %}
<span class="inline-block px-2 py-0.5 rounded text-xs font-medium bg-green-900/30 text-green-400 border border-green-800/30">both</span>
{% elif src.match_type == 'semantic' %}
<span class="inline-block px-2 py-0.5 rounded text-xs font-medium bg-purple-900/30 text-purple-400 border border-purple-800/30">semantic</span>
{% else %}
<span class="inline-block px-2 py-0.5 rounded text-xs font-medium bg-slate-800/50 text-slate-400 border border-slate-700/30">keyword</span>
{% endif %}
</td>
<td class="px-4 py-3 text-right font-mono text-xs text-slate-400">
{{ "%.3f"|format(src.similarity) if src.similarity else "-" }}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
</div>
{% endif %}
{% endblock %}