Idea quality pipeline, web UI features, academic paper

- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing
  1,907 ideas to 468 across 434 drafts (78% reduction)
- Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity
- Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter)
  using Claude to rate ideas 1-5, removing 49 generic building blocks
- Final count: 419 high-quality ideas (avg 1.1/draft)
- Web UI: gap explorer with live draft generation and pre-generated demos
- Web UI: D3.js author collaboration network (498 nodes, 1142 edges,
  68 clusters, org filtering, interactive zoom/pan)
- Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft
  AI agent standards landscape
- Save improvement ideas backlog to data/reports/improvement-ideas.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-06 22:17:57 +01:00
parent 3c3d7e649f
commit 6e3a387778
29 changed files with 6575 additions and 240 deletions

View File

@@ -0,0 +1,369 @@
{% extends "base.html" %}
{% set active_page = "drafts" %}
{% block title %}Draft Explorer — IETF Draft Analyzer{% endblock %}
{% block extra_head %}
<style>
.filter-bar {
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
backdrop-filter: blur(10px);
}
.draft-row {
transition: all 0.15s ease;
}
.draft-row:hover {
background: rgba(59, 130, 246, 0.05);
}
.dim-bar-bg {
display: inline-block;
width: 40px;
height: 6px;
border-radius: 3px;
background: rgba(51, 65, 85, 0.6);
vertical-align: middle;
position: relative;
overflow: hidden;
}
.dim-bar-fill {
display: block;
height: 100%;
border-radius: 3px;
}
.dim-fill-high { background: #4ade80; }
.dim-fill-mid { background: #facc15; }
.dim-fill-low { background: #f87171; }
.cat-pill {
display: inline-block;
padding: 1px 8px;
border-radius: 9999px;
font-size: 0.65rem;
font-weight: 500;
background: rgba(51, 65, 85, 0.5);
color: #94a3b8;
border: 1px solid rgba(71, 85, 105, 0.4);
white-space: nowrap;
}
.cat-pill-active {
background: rgba(59, 130, 246, 0.2);
color: #60a5fa;
border-color: rgba(59, 130, 246, 0.4);
}
.range-slider {
-webkit-appearance: none;
appearance: none;
height: 4px;
border-radius: 2px;
background: #334155;
outline: none;
}
.range-slider::-webkit-slider-thumb {
-webkit-appearance: none;
appearance: none;
width: 16px;
height: 16px;
border-radius: 50%;
background: #3b82f6;
cursor: pointer;
border: 2px solid #1e293b;
}
.range-slider::-moz-range-thumb {
width: 16px;
height: 16px;
border-radius: 50%;
background: #3b82f6;
cursor: pointer;
border: 2px solid #1e293b;
}
.page-btn {
padding: 6px 12px;
border-radius: 8px;
font-size: 0.8rem;
font-weight: 500;
transition: all 0.15s ease;
}
.page-btn-active {
background: #3b82f6;
color: white;
}
.page-btn-inactive {
background: rgba(30, 41, 59, 0.6);
border: 1px solid #334155;
color: #94a3b8;
}
.page-btn-inactive:hover {
border-color: #475569;
color: #e2e8f0;
}
</style>
{% endblock %}
{% block content %}
<!-- Header -->
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Draft Explorer</h1>
<p class="text-slate-400 text-sm mt-1">Browse, search, and filter {{ result.total }} rated Internet-Drafts on AI and agent topics.</p>
</div>
<!-- Filter Bar -->
<div class="filter-bar rounded-xl border border-slate-800 p-5 mb-6">
<form method="get" action="/drafts" id="filterForm">
<!-- Row 1: Search + Sort + Submit -->
<div class="flex flex-wrap gap-3 items-end">
<!-- Search -->
<div class="flex-1 min-w-[200px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Search</label>
<input type="text" name="q" value="{{ search }}" placeholder="Search by name, title, or summary..."
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-4 py-2 text-sm text-slate-200 placeholder-slate-500 focus:outline-none focus:border-blue-500 focus:ring-1 focus:ring-blue-500/30 transition">
</div>
<!-- Category dropdown -->
<div class="min-w-[180px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Category</label>
<select name="cat"
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
<option value="">All categories</option>
{% for cat, count in categories.items() %}
<option value="{{ cat }}" {% if current_cat == cat %}selected{% endif %}>{{ cat }} ({{ count }})</option>
{% endfor %}
</select>
</div>
<!-- Sort -->
<div class="min-w-[150px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Sort by</label>
<select name="sort"
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
<option value="score" {% if sort == 'score' %}selected{% endif %}>Score</option>
<option value="date" {% if sort == 'date' %}selected{% endif %}>Date</option>
<option value="novelty" {% if sort == 'novelty' %}selected{% endif %}>Novelty</option>
<option value="maturity" {% if sort == 'maturity' %}selected{% endif %}>Maturity</option>
<option value="relevance" {% if sort == 'relevance' %}selected{% endif %}>Relevance</option>
<option value="momentum" {% if sort == 'momentum' %}selected{% endif %}>Momentum</option>
<option value="overlap" {% if sort == 'overlap' %}selected{% endif %}>Overlap</option>
<option value="name" {% if sort == 'name' %}selected{% endif %}>Name</option>
</select>
</div>
<!-- Sort direction -->
<div class="min-w-[110px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Direction</label>
<select name="dir"
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
<option value="desc" {% if sort_dir == 'desc' %}selected{% endif %}>Descending</option>
<option value="asc" {% if sort_dir == 'asc' %}selected{% endif %}>Ascending</option>
</select>
</div>
</div>
<!-- Row 2: Min Score slider -->
<div class="mt-4 flex flex-wrap items-center gap-4">
<div class="flex items-center gap-3">
<label class="text-xs font-medium text-slate-500 whitespace-nowrap">Min Score:</label>
<input type="range" name="min_score" id="scoreSlider" value="{{ min_score }}" step="0.5" min="0" max="5"
class="range-slider w-40" oninput="document.getElementById('scoreVal').textContent = this.value">
<span id="scoreVal" class="text-sm font-mono font-semibold text-blue-400 w-8">{{ min_score }}</span>
</div>
<div class="flex gap-2 ml-auto">
<button type="submit" class="px-5 py-2 bg-blue-600 text-white rounded-lg text-sm font-medium hover:bg-blue-500 transition-colors">
Apply Filters
</button>
<a href="/drafts" class="px-4 py-2 border border-slate-700 text-slate-400 rounded-lg text-sm hover:border-slate-500 hover:text-slate-300 transition-colors">
Reset
</a>
</div>
</div>
<!-- Row 3: Category pills (quick filter) -->
{% if categories %}
<div class="mt-4 pt-3 border-t border-slate-800/50">
<div class="flex flex-wrap gap-1.5">
<a href="/drafts?q={{ search }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="cat-pill {% if not current_cat %}cat-pill-active{% endif %}">All</a>
{% for cat, count in categories.items() %}
<a href="/drafts?cat={{ cat }}&q={{ search }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="cat-pill {% if current_cat == cat %}cat-pill-active{% endif %}">
{{ cat }} <span class="opacity-50">{{ count }}</span>
</a>
{% endfor %}
</div>
</div>
{% endif %}
</form>
</div>
<!-- Results count -->
<div class="flex items-center justify-between mb-4">
<p class="text-sm text-slate-500">
Showing <span class="text-slate-300 font-medium">{{ result.drafts|length }}</span> of
<span class="text-slate-300 font-medium">{{ result.total }}</span> drafts
{% if search %} matching "<span class="text-blue-400">{{ search }}</span>"{% endif %}
{% if current_cat %} in <span class="text-blue-400">{{ current_cat }}</span>{% endif %}
{% if min_score > 0 %} with score >= <span class="text-blue-400">{{ min_score }}</span>{% endif %}
</p>
{% if result.pages > 1 %}
<p class="text-xs text-slate-600">Page {{ result.page }} of {{ result.pages }}</p>
{% endif %}
</div>
<!-- Draft Table -->
<div class="bg-slate-900/60 rounded-xl border border-slate-800 overflow-hidden">
<div class="overflow-x-auto">
<table class="w-full text-sm">
<thead>
<tr class="border-b border-slate-800 bg-slate-900/80">
{% macro sort_header(field, label, extra_class="", title="") %}
{% set is_active = sort == field %}
{% set next_dir = 'asc' if (is_active and sort_dir == 'desc') else 'desc' %}
<th class="px-4 py-3 text-left text-xs font-medium uppercase tracking-wide {{ extra_class }} {{ 'text-blue-400' if is_active else 'text-slate-500' }}">
<a href="/drafts?q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ field }}&dir={{ next_dir }}"
class="hover:text-blue-400 transition inline-flex items-center gap-1"
{% if title %}title="{{ title }}"{% endif %}>
{{ label }}
{% if is_active %}
<svg class="w-3 h-3 {{ 'rotate-180' if sort_dir == 'asc' else '' }}" fill="currentColor" viewBox="0 0 20 20">
<path fill-rule="evenodd" d="M5.293 7.293a1 1 0 011.414 0L10 10.586l3.293-3.293a1 1 0 111.414 1.414l-4 4a1 1 0 01-1.414 0l-4-4a1 1 0 010-1.414z" clip-rule="evenodd"/>
</svg>
{% endif %}
</a>
</th>
{% endmacro %}
{{ sort_header("score", "Score", "w-20") }}
{{ sort_header("name", "Draft") }}
{{ sort_header("date", "Date", "w-24 hidden md:table-cell") }}
{{ sort_header("novelty", "Nov", "w-20 hidden lg:table-cell", "Novelty") }}
{{ sort_header("maturity", "Mat", "w-20 hidden lg:table-cell", "Maturity") }}
{{ sort_header("relevance", "Rel", "w-20 hidden lg:table-cell", "Relevance") }}
{{ sort_header("momentum", "Mom", "w-20 hidden xl:table-cell", "Momentum") }}
{{ sort_header("overlap", "Ovl", "w-20 hidden xl:table-cell", "Overlap") }}
<th class="px-4 py-3 text-left text-xs font-medium text-slate-500 uppercase tracking-wide hidden md:table-cell">Categories</th>
</tr>
</thead>
<tbody class="divide-y divide-slate-800/30">
{% for d in result.drafts %}
<tr class="draft-row">
<!-- Score badge -->
<td class="px-4 py-3">
<span class="score-badge {% if d.score >= 3.5 %}score-high{% elif d.score >= 2.5 %}score-mid{% else %}score-low{% endif %}">
{{ d.score }}
</span>
</td>
<!-- Draft name + title -->
<td class="px-4 py-3">
<a href="/drafts/{{ d.name }}" class="text-blue-400 hover:text-blue-300 font-medium text-sm transition">
{{ d.title }}
</a>
<div class="text-xs text-slate-600 mt-0.5 font-mono">{{ d.name }}</div>
{% if d.summary %}
<div class="text-xs text-slate-500 mt-1 line-clamp-1 max-w-lg">{{ d.summary }}</div>
{% endif %}
</td>
<!-- Date -->
<td class="px-4 py-3 text-xs text-slate-500 hidden md:table-cell whitespace-nowrap">{{ d.date }}</td>
<!-- Dimension bars -->
{% macro dim_cell(value) %}
<td class="px-4 py-3 hidden lg:table-cell">
<div class="flex items-center gap-1.5">
<span class="dim-bar-bg">
<span class="dim-bar-fill {% if value >= 4 %}dim-fill-high{% elif value >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
style="width: {{ (value / 5 * 100)|int }}%"></span>
</span>
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ value }}</span>
</div>
</td>
{% endmacro %}
{{ dim_cell(d.novelty) }}
{{ dim_cell(d.maturity) }}
{{ dim_cell(d.relevance) }}
<td class="px-4 py-3 hidden xl:table-cell">
<div class="flex items-center gap-1.5">
<span class="dim-bar-bg">
<span class="dim-bar-fill {% if d.momentum >= 4 %}dim-fill-high{% elif d.momentum >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
style="width: {{ (d.momentum / 5 * 100)|int }}%"></span>
</span>
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ d.momentum }}</span>
</div>
</td>
<td class="px-4 py-3 hidden xl:table-cell">
<div class="flex items-center gap-1.5">
<span class="dim-bar-bg">
<span class="dim-bar-fill {% if d.overlap >= 4 %}dim-fill-high{% elif d.overlap >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
style="width: {{ (d.overlap / 5 * 100)|int }}%"></span>
</span>
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ d.overlap }}</span>
</div>
</td>
<!-- Categories -->
<td class="px-4 py-3 hidden md:table-cell">
<div class="flex flex-wrap gap-1">
{% for cat in d.categories[:3] %}
<span class="cat-pill">{{ cat }}</span>
{% endfor %}
{% if d.categories|length > 3 %}
<span class="cat-pill opacity-50">+{{ d.categories|length - 3 }}</span>
{% endif %}
</div>
</td>
</tr>
{% endfor %}
{% if not result.drafts %}
<tr>
<td colspan="9" class="px-4 py-12 text-center text-slate-500">
<svg class="w-12 h-12 mx-auto mb-3 opacity-30" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z"/>
</svg>
<p class="text-sm">No drafts match your filters.</p>
<a href="/drafts" class="text-blue-400 text-sm hover:text-blue-300 mt-1 inline-block">Clear all filters</a>
</td>
</tr>
{% endif %}
</tbody>
</table>
</div>
</div>
<!-- Pagination -->
{% if result.pages > 1 %}
<nav class="flex items-center justify-center gap-1.5 mt-6">
{% if result.page > 1 %}
<a href="/drafts?page={{ result.page - 1 }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">
<svg class="w-4 h-4 inline" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7"/></svg>
Prev
</a>
{% endif %}
{% set start_page = [1, result.page - 2]|max %}
{% set end_page = [result.pages, result.page + 2]|min %}
{% if start_page > 1 %}
<a href="/drafts?page=1&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">1</a>
{% if start_page > 2 %}<span class="text-slate-600 px-1">...</span>{% endif %}
{% endif %}
{% for p in range(start_page, end_page + 1) %}
{% if p == result.page %}
<span class="page-btn page-btn-active">{{ p }}</span>
{% else %}
<a href="/drafts?page={{ p }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">{{ p }}</a>
{% endif %}
{% endfor %}
{% if end_page < result.pages %}
{% if end_page < result.pages - 1 %}<span class="text-slate-600 px-1">...</span>{% endif %}
<a href="/drafts?page={{ result.pages }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">{{ result.pages }}</a>
{% endif %}
{% if result.page < result.pages %}
<a href="/drafts?page={{ result.page + 1 }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">
Next
<svg class="w-4 h-4 inline" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"/></svg>
</a>
{% endif %}
</nav>
{% endif %}
{% endblock %}