Idea quality pipeline, web UI features, academic paper

- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing
  1,907 ideas to 468 across 434 drafts (78% reduction)
- Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity
- Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter)
  using Claude to rate ideas 1-5, removing 49 generic building blocks
- Final count: 419 high-quality ideas (avg 1.1/draft)
- Web UI: gap explorer with live draft generation and pre-generated demos
- Web UI: D3.js author collaboration network (498 nodes, 1142 edges,
  68 clusters, org filtering, interactive zoom/pan)
- Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft
  AI agent standards landscape
- Save improvement ideas backlog to data/reports/improvement-ideas.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-06 22:17:57 +01:00
parent 3c3d7e649f
commit 6e3a387778
29 changed files with 6575 additions and 240 deletions

View File

@@ -0,0 +1,65 @@
{% extends "base.html" %}
{% set active_page = "about" %}
{% block title %}About — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="max-w-3xl">
<h1 class="text-2xl font-bold text-white mb-6">About IETF Draft Analyzer</h1>
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6 mb-6">
<h2 class="text-lg font-semibold text-white mb-3">What is this?</h2>
<p class="text-sm text-slate-400 leading-relaxed mb-4">
A tool for tracking, categorizing, rating, and mapping IETF Internet-Drafts
focused on AI and agent-related topics. It uses Claude for analysis and rating,
Ollama for embeddings, and SQLite for storage.
</p>
<p class="text-sm text-slate-400 leading-relaxed">
The dashboard provides interactive visualizations of the draft landscape,
including category breakdowns, rating distributions, author networks,
extracted ideas, and gap analysis.
</p>
</div>
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6 mb-6">
<h2 class="text-lg font-semibold text-white mb-3">Current Data</h2>
<div class="grid grid-cols-2 gap-4 text-sm">
<div>
<div class="text-slate-500">Total Drafts</div>
<div class="text-xl font-bold text-blue-400">{{ stats.total_drafts }}</div>
</div>
<div>
<div class="text-slate-500">Rated Drafts</div>
<div class="text-xl font-bold text-green-400">{{ stats.rated_count }}</div>
</div>
<div>
<div class="text-slate-500">Authors Tracked</div>
<div class="text-xl font-bold text-purple-400">{{ stats.author_count }}</div>
</div>
<div>
<div class="text-slate-500">Ideas Extracted</div>
<div class="text-xl font-bold text-amber-400">{{ stats.idea_count }}</div>
</div>
<div>
<div class="text-slate-500">Gaps Identified</div>
<div class="text-xl font-bold text-red-400">{{ stats.gap_count }}</div>
</div>
<div>
<div class="text-slate-500">API Tokens Used</div>
<div class="text-xl font-bold text-slate-300">{{ "{:,}".format(stats.input_tokens + stats.output_tokens) }}</div>
</div>
</div>
</div>
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6">
<h2 class="text-lg font-semibold text-white mb-3">Tech Stack</h2>
<ul class="text-sm text-slate-400 space-y-2">
<li><span class="text-slate-200 font-medium">Analysis:</span> Claude (Sonnet for analysis, Haiku for bulk)</li>
<li><span class="text-slate-200 font-medium">Embeddings:</span> Ollama (nomic-embed-text)</li>
<li><span class="text-slate-200 font-medium">Storage:</span> SQLite with FTS5 full-text search</li>
<li><span class="text-slate-200 font-medium">Dashboard:</span> Flask, Tailwind CSS, Plotly.js</li>
<li><span class="text-slate-200 font-medium">Data source:</span> IETF Datatracker API</li>
</ul>
</div>
</div>
{% endblock %}