Idea quality pipeline, web UI features, academic paper

- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing
  1,907 ideas to 468 across 434 drafts (78% reduction)
- Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity
- Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter)
  using Claude to rate ideas 1-5, removing 49 generic building blocks
- Final count: 419 high-quality ideas (avg 1.1/draft)
- Web UI: gap explorer with live draft generation and pre-generated demos
- Web UI: D3.js author collaboration network (498 nodes, 1142 edges,
  68 clusters, org filtering, interactive zoom/pan)
- Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft
  AI agent standards landscape
- Save improvement ideas backlog to data/reports/improvement-ideas.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-06 22:17:57 +01:00
parent 3c3d7e649f
commit 6e3a387778
29 changed files with 6575 additions and 240 deletions

View File

@@ -77,7 +77,7 @@ Abstract: {abstract}
{text_excerpt}
Return 0-8 ideas. Only include CONCRETE, NOVEL technical contributions — not restatements of the abstract or general goals. If the draft has no substantive technical ideas (e.g. it is a problem statement, administrative document, or off-topic), return an empty array [].
Return 1-4 ideas. Extract only TOP-LEVEL novel contributions. Do NOT list sub-features, optimizations, variants, or extensions as separate ideas. If a draft defines one protocol with multiple features, that is ONE idea, not several. Each idea must be independently novel — could it be its own draft? If not, merge it with the parent idea. Only include CONCRETE, NOVEL technical contributions — not restatements of the abstract or general goals. If the draft has no substantive technical ideas (e.g. it is a problem statement, administrative document, or off-topic), return an empty array [].
JSON array only, no fences."""
BATCH_IDEAS_PROMPT = """\
@@ -86,7 +86,7 @@ Per idea: {{"title":"short name","description":"1 sentence","type":"mechanism|pr
{drafts_block}
0-8 ideas per draft. Only include CONCRETE, NOVEL technical contributions. If a draft has no substantive ideas, map it to an empty array. Do not pad with restatements of the abstract.
1-4 ideas per draft. Extract only TOP-LEVEL novel contributions. Do NOT list sub-features, optimizations, variants, or extensions as separate ideas. If a draft defines one protocol with multiple features, that is ONE idea, not several. Each idea must be independently novel — could it be its own draft? If not, merge it with the parent idea. Only include CONCRETE, NOVEL technical contributions. If a draft has no substantive ideas, map it to an empty array. Do not pad with restatements of the abstract.
Return ONLY a JSON object like {{"draft-name":[...], ...}}, no fences."""
GAP_ANALYSIS_PROMPT = """\
@@ -115,6 +115,21 @@ Focus on:
JSON array only, no fences."""
SCORE_NOVELTY_PROMPT = """\
Rate each idea's novelty/originality on a 1-5 scale.
1 = Generic building block anyone would include (e.g. "Agent Gateway", "Certificate Authority")
2 = Obvious extension of existing work, minimal originality
3 = Useful and relevant but expected given the problem space
4 = Interesting contribution with some original thinking
5 = Genuinely novel mechanism, protocol, or architectural insight
Ideas to score:
{ideas_block}
Return ONLY a JSON object mapping idea ID to score, like {{"123": 3, "456": 1, ...}}.
No fences, no explanation."""
def _prompt_hash(text: str) -> str:
return hashlib.sha256(text.encode()).hexdigest()[:16]
@@ -558,3 +573,222 @@ class Analyzer:
return text
except anthropic.APIError as e:
return f"Error: {e}"
def dedup_ideas(self, threshold: float = 0.85, dry_run: bool = True,
draft_name: str | None = None) -> dict:
"""Deduplicate ideas within each draft using embedding similarity.
For each draft, computes pairwise cosine similarity of idea embeddings.
Ideas above the threshold are merged (keeping the one with the longer
description).
Args:
threshold: Cosine similarity threshold for merging (default 0.85).
dry_run: If True, report what would be merged without deleting.
draft_name: If provided, only dedup ideas for this draft.
Returns:
Dict with keys: total_before, total_after, merged_count, examples.
"""
import numpy as np
import ollama as ollama_lib
client = ollama_lib.Client(host=self.config.ollama_url)
# Get list of drafts to process
if draft_name:
draft_names = [draft_name]
else:
rows = self.db.conn.execute(
"SELECT DISTINCT draft_name FROM ideas ORDER BY draft_name"
).fetchall()
draft_names = [r["draft_name"] for r in rows]
total_before = 0
merged_count = 0
examples = []
ids_to_delete = []
for dname in draft_names:
ideas = self.db.get_ideas_for_draft(dname)
if len(ideas) < 2:
total_before += len(ideas)
continue
total_before += len(ideas)
# Embed each idea: "title: description"
texts = [f"{idea['title']}: {idea['description']}" for idea in ideas]
try:
resp = client.embed(
model=self.config.ollama_embed_model, input=texts
)
vectors = [
np.array(v, dtype=np.float32)
for v in resp["embeddings"]
]
except Exception as e:
console.print(f"[red]Failed to embed ideas for {dname}: {e}[/]")
continue
# Track which ideas are already marked for deletion in this draft
deleted_in_draft = set()
# Compare all pairs within this draft
for i in range(len(ideas)):
if ideas[i]["id"] in deleted_in_draft:
continue
for j in range(i + 1, len(ideas)):
if ideas[j]["id"] in deleted_in_draft:
continue
# Cosine similarity
dot = np.dot(vectors[i], vectors[j])
norm = np.linalg.norm(vectors[i]) * np.linalg.norm(vectors[j])
sim = float(dot / norm) if norm > 0 else 0.0
if sim >= threshold:
# Keep the idea with the longer description
keep = ideas[i] if len(ideas[i]["description"]) >= len(ideas[j]["description"]) else ideas[j]
drop = ideas[j] if keep is ideas[i] else ideas[i]
ids_to_delete.append(drop["id"])
deleted_in_draft.add(drop["id"])
merged_count += 1
if len(examples) < 20:
examples.append({
"draft": dname,
"keep": keep["title"],
"drop": drop["title"],
"similarity": round(sim, 3),
})
if not dry_run:
for idea_id in ids_to_delete:
self.db.delete_idea(idea_id)
total_after = total_before - merged_count
return {
"total_before": total_before,
"total_after": total_after,
"merged_count": merged_count,
"examples": examples,
}
def score_idea_novelty(self, batch_size: int = 20, cheap: bool = True) -> dict:
"""Score all unscored ideas for novelty (1-5) using Claude.
Args:
batch_size: Number of ideas per API call (default 20).
cheap: Use Haiku model for lower cost (default True).
Returns:
Dict with keys: scored_count, avg_score, distribution.
"""
unscored = self.db.ideas_with_drafts(unscored_only=True)
if not unscored:
console.print("All ideas already scored.")
return {"scored_count": 0, "avg_score": 0.0, "distribution": {}}
model_label = "Haiku" if cheap else "Sonnet"
console.print(
f"Scoring [bold]{len(unscored)}[/] ideas for novelty "
f"(batches of {batch_size}, {model_label})..."
)
scored_count = 0
all_scores: list[int] = []
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
MofNCompleteColumn(),
console=console,
) as progress:
task = progress.add_task("Scoring novelty...", total=len(unscored))
for i in range(0, len(unscored), batch_size):
batch = unscored[i:i + batch_size]
progress.update(task, description=f"Batch {i // batch_size + 1}")
# Build ideas block for prompt
ideas_block = ""
for idea in batch:
ideas_block += (
f"\n---\nID: {idea['id']}\n"
f"Draft: {idea['draft_title']}\n"
f"Idea: {idea['title']}\n"
f"Description: {idea['description']}\n"
)
prompt = SCORE_NOVELTY_PROMPT.format(ideas_block=ideas_block)
phash = _prompt_hash(prompt)
# Check cache
cached = self.db.get_cached_response("_novelty_score_", phash)
if cached:
try:
scores = json.loads(cached)
if isinstance(scores, dict):
batch_scores = {int(k): int(v) for k, v in scores.items()}
self.db.update_idea_scores_bulk(batch_scores)
scored_count += len(batch_scores)
all_scores.extend(batch_scores.values())
progress.advance(task, advance=len(batch))
continue
except (json.JSONDecodeError, KeyError, ValueError):
pass
try:
text, in_tok, out_tok = self._call_claude(
prompt, max_tokens=50 * len(batch), cheap=cheap
)
text = self._extract_json(text)
scores = json.loads(text)
if not isinstance(scores, dict):
console.print(f"[red]Batch {i // batch_size + 1}: unexpected response format[/]")
progress.advance(task, advance=len(batch))
continue
# Cache the raw response
self.db.cache_response(
"_novelty_score_", phash,
self.config.claude_model_cheap if cheap else self.config.claude_model,
prompt, text, in_tok, out_tok,
)
# Parse and store scores
batch_scores = {}
for k, v in scores.items():
try:
idea_id = int(k)
score = max(1, min(5, int(v)))
batch_scores[idea_id] = score
except (ValueError, TypeError):
continue
self.db.update_idea_scores_bulk(batch_scores)
scored_count += len(batch_scores)
all_scores.extend(batch_scores.values())
except (json.JSONDecodeError, anthropic.APIError) as e:
console.print(f"[red]Batch {i // batch_size + 1} failed: {e}[/]")
progress.advance(task, advance=len(batch))
# Build distribution
distribution: dict[int, int] = {}
for s in all_scores:
distribution[s] = distribution.get(s, 0) + 1
avg = sum(all_scores) / len(all_scores) if all_scores else 0.0
in_tok, out_tok = self.db.total_tokens_used()
console.print(
f"Scored [bold green]{scored_count}[/] ideas "
f"(avg: {avg:.1f}) | Tokens: {in_tok:,} in + {out_tok:,} out"
)
return {"scored_count": scored_count, "avg_score": round(avg, 2), "distribution": distribution}

View File

@@ -256,6 +256,60 @@ def embed():
db.close()
# ── embed-ideas ──────────────────────────────────────────────────────────────
@main.command("embed-ideas")
@click.option("--limit", default=0, help="Max ideas to embed (0=all)")
@click.option("--batch-size", default=50, help="Batch size for Ollama")
def embed_ideas(limit: int, batch_size: int):
"""Generate embeddings for extracted ideas via Ollama."""
import ollama as ollama_lib
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, MofNCompleteColumn
cfg = _get_config()
db = Database(cfg)
client = ollama_lib.Client(host=cfg.ollama_url)
try:
missing = db.ideas_without_embeddings(limit=limit if limit > 0 else 10000)
if not missing:
console.print("All ideas already have embeddings.")
return
total = len(missing)
console.print(f"Embedding [bold]{total}[/] ideas in batches of {batch_size}...")
count = 0
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
MofNCompleteColumn(),
console=console,
) as progress:
task = progress.add_task("Embedding ideas...", total=total)
for start in range(0, total, batch_size):
batch = missing[start:start + batch_size]
texts = [f"{idea['title']}. {idea['description']}" for idea in batch]
try:
resp = client.embed(model=cfg.ollama_embed_model, input=texts)
for i, idea in enumerate(batch):
import numpy as np
vec = np.array(resp["embeddings"][i], dtype=np.float32)
db.store_idea_embedding(idea["id"], cfg.ollama_embed_model, vec)
count += 1
progress.advance(task)
except Exception as e:
console.print(f"[red]Batch failed: {e}[/]")
for _ in batch:
progress.advance(task)
console.print(f"Embedded [bold green]{count}[/] ideas")
finally:
db.close()
# ── similar ──────────────────────────────────────────────────────────────────
@@ -531,6 +585,261 @@ def co_occurrence_report():
db.close()
@report.command("wg")
def wg_report():
"""Working group analysis report — overlaps, alignment, submission targets."""
from .reports import Reporter
cfg = _get_config()
db = Database(cfg)
reporter = Reporter(cfg, db)
try:
path = reporter.wg_report()
console.print(f"Report saved: [bold]{path}[/]")
finally:
db.close()
# ── wg (working group analysis) ─────────────────────────────────────────
@main.group()
def wg():
"""Working group analysis — overlaps, alignment opportunities, submission targets."""
pass
@wg.command("list")
@click.option("--min-drafts", default=1, help="Minimum drafts to show a WG")
def wg_list(min_drafts: int):
"""List working groups with draft counts and average scores."""
cfg = _get_config()
db = Database(cfg)
try:
summaries = db.wg_summary()
if not summaries:
console.print("[yellow]No WG data. Run: python scripts/backfill-wg-names.py[/]")
return
summaries = [s for s in summaries if s["draft_count"] >= min_drafts]
table = Table(title=f"Working Groups ({len(summaries)} with >= {min_drafts} drafts)")
table.add_column("WG", style="cyan", width=12)
table.add_column("#", justify="right", width=4)
table.add_column("Ideas", justify="right", width=5)
table.add_column("Nov", justify="center", width=4)
table.add_column("Mat", justify="center", width=4)
table.add_column("Ovl", justify="center", width=4)
table.add_column("Mom", justify="center", width=4)
table.add_column("Rel", justify="center", width=4)
table.add_column("Top Categories")
for s in summaries:
top_cats = sorted(s["categories"].items(), key=lambda x: x[1], reverse=True)[:3]
cats_str = ", ".join(f"{c}({n})" for c, n in top_cats) if top_cats else "-"
table.add_row(
s["wg"], str(s["draft_count"]), str(s["idea_count"]),
str(s["avg_novelty"]), str(s["avg_maturity"]),
str(s["avg_overlap"]), str(s["avg_momentum"]),
str(s["avg_relevance"]), cats_str,
)
console.print(table)
# Also show individual submission count
indiv = db.conn.execute(
'SELECT COUNT(*) FROM drafts WHERE "group" = \'none\' OR "group" IS NULL'
).fetchone()[0]
console.print(f"\n[dim]Individual submissions (no WG): {indiv}[/]")
finally:
db.close()
@wg.command("show")
@click.argument("name")
def wg_show(name: str):
"""Show details for a specific working group."""
cfg = _get_config()
db = Database(cfg)
try:
drafts = db.wg_drafts(name)
if not drafts:
console.print(f"[red]No drafts found for WG: {name}[/]")
return
console.print(f"\n[bold]Working Group: {name}[/] ({len(drafts)} drafts)\n")
table = Table()
table.add_column("Date", style="dim", width=10)
table.add_column("Name", style="cyan")
table.add_column("Title", max_width=50)
table.add_column("Score", justify="right", width=6)
for d in drafts:
rating = db.get_rating(d.name)
score = f"{rating.composite_score:.1f}" if rating else "-"
table.add_row(d.date, d.name, d.title[:50], score)
console.print(table)
# Show ideas for this WG
ideas = []
for d in drafts:
ideas.extend(db.get_ideas_for_draft(d.name))
if ideas:
console.print(f"\n[bold]Ideas ({len(ideas)}):[/]")
for idea in ideas[:15]:
console.print(f" - [cyan]{idea['title']}[/]: {idea['description'][:80]}")
if len(ideas) > 15:
console.print(f" [dim]... and {len(ideas) - 15} more[/]")
finally:
db.close()
@wg.command("overlaps")
@click.option("--min-wgs", default=2, help="Minimum WGs sharing a category to show")
def wg_overlaps(min_wgs: int):
"""Find categories and ideas that span multiple WGs — alignment opportunities."""
cfg = _get_config()
db = Database(cfg)
try:
# Category spread across WGs
spread = db.category_wg_spread()
multi = [s for s in spread if s["wg_count"] >= min_wgs
and not all(w["wg"] == "none" for w in s["wgs"])]
if multi:
console.print(f"\n[bold]Categories spanning {min_wgs}+ WGs[/]\n")
for s in multi:
wg_strs = [f"{w['wg']}({w['count']})" for w in s["wgs"] if w["wg"] != "none"]
if wg_strs:
console.print(f" [cyan]{s['category']}[/] — {s['total_drafts']} drafts across {s['wg_count']} WGs")
console.print(f" WGs: {', '.join(wg_strs)}")
# Idea overlap across WGs
idea_overlaps = db.wg_idea_overlap()
cross_wg = [o for o in idea_overlaps
if not all(w == "none" for w in o["wg_names"])]
if cross_wg:
console.print(f"\n[bold]Ideas appearing in {min_wgs}+ WGs ({len(cross_wg)} found)[/]\n")
for o in cross_wg[:20]:
real_wgs = [w for w in o["wg_names"] if w != "none"]
console.print(f" [cyan]{o['idea_title']}[/] — WGs: {', '.join(real_wgs)}")
for entry in o["wgs"]:
if entry["wg"] != "none":
console.print(f" - [{entry['wg']}] {entry['draft_name']}")
if len(cross_wg) > 20:
console.print(f"\n [dim]... and {len(cross_wg) - 20} more[/]")
if not multi and not cross_wg:
console.print("[yellow]No cross-WG overlaps found.[/]")
finally:
db.close()
@wg.command("alignment")
def wg_alignment():
"""Identify where individual drafts should be consolidated into WG standards."""
cfg = _get_config()
db = Database(cfg)
try:
# Compare individual vs WG category distribution
dist = db.individual_vs_wg_categories()
indiv = dist["individual"]
adopted = dist["wg_adopted"]
console.print("\n[bold]Individual vs WG-Adopted Category Distribution[/]\n")
table = Table()
table.add_column("Category", width=25)
table.add_column("Individual", justify="right", width=10)
table.add_column("WG-Adopted", justify="right", width=10)
table.add_column("Signal", width=40)
all_cats = sorted(set(list(indiv.keys()) + list(adopted.keys())))
for cat in all_cats:
i_count = indiv.get(cat, 0)
w_count = adopted.get(cat, 0)
signal = ""
if i_count >= 5 and w_count == 0:
signal = "[yellow]High individual activity, no WG — needs WG?[/]"
elif i_count >= 3 and w_count >= 1:
signal = "[green]WG exists, individual drafts could target it[/]"
elif w_count > i_count and i_count > 0:
signal = "[dim]WG leading, some individual work[/]"
table.add_row(cat, str(i_count), str(w_count), signal)
console.print(table)
# Find overlap clusters within individual submissions that might warrant a WG
console.print("\n[bold]Consolidation Candidates[/]")
console.print("[dim]Categories with many individual drafts but no WG adoption — "
"potential for new WG or BoF[/]\n")
candidates = []
for cat in all_cats:
i_count = indiv.get(cat, 0)
w_count = adopted.get(cat, 0)
if i_count >= 5 and w_count == 0:
candidates.append((cat, i_count))
if candidates:
for cat, count in sorted(candidates, key=lambda x: x[1], reverse=True):
console.print(f" [yellow]{cat}[/]: {count} individual drafts, no WG home")
# Show sample drafts
rows = db.conn.execute("""
SELECT d.name, d.title FROM drafts d
JOIN ratings r ON d.name = r.draft_name
WHERE (d."group" = 'none' OR d."group" IS NULL)
AND r.categories LIKE ?
ORDER BY (r.novelty * 0.30 + r.relevance * 0.25 + r.maturity * 0.20
+ r.momentum * 0.15 + (6 - r.overlap) * 0.10) DESC
LIMIT 5
""", (f"%{cat}%",)).fetchall()
for row in rows:
console.print(f" - {row['name']}: {row['title'][:60]}")
console.print()
else:
console.print(" [green]All active categories have WG representation.[/]")
finally:
db.close()
@wg.command("targets")
def wg_targets():
"""Suggest best WGs for submitting new work in each category."""
cfg = _get_config()
db = Database(cfg)
try:
spread = db.category_wg_spread()
summaries = {s["wg"]: s for s in db.wg_summary()}
console.print("\n[bold]Recommended Submission Targets by Category[/]\n")
for s in spread:
cat = s["category"]
# Filter to real WGs (not 'none')
real_wgs = [w for w in s["wgs"] if w["wg"] != "none"]
if not real_wgs:
console.print(f" [cyan]{cat}[/]: [yellow]No active WG — individual submission[/]")
continue
best = real_wgs[0]
wg_info = summaries.get(best["wg"], {})
console.print(
f" [cyan]{cat}[/]: [bold green]{best['wg']}[/] "
f"({best['count']} drafts"
f"{', avg relevance ' + str(wg_info.get('avg_relevance', '?')) if wg_info else ''})"
)
if len(real_wgs) > 1:
alts = ", ".join(f"{w['wg']}({w['count']})" for w in real_wgs[1:3])
console.print(f" Also: {alts}")
console.print()
finally:
db.close()
# ── visualize ────────────────────────────────────────────────────────────
@@ -808,14 +1117,21 @@ def network(top: int):
# ── ideas ───────────────────────────────────────────────────────────────
@main.command()
@click.argument("name", required=False)
@main.group(invoke_without_command=True)
@click.option("--name", default=None, help="Extract ideas from a specific draft")
@click.option("--all", "extract_all", is_flag=True, help="Extract ideas from all drafts")
@click.option("--limit", "-n", default=50, help="Max drafts to extract (with --all)")
@click.option("--batch", "-b", default=5, help="Drafts per API call (default 5, set 1 for individual)")
@click.option("--cheap/--quality", default=True, help="Use Haiku (cheap) vs Sonnet (quality)")
def ideas(name: str | None, extract_all: bool, limit: int, batch: int, cheap: bool):
"""Extract technical ideas from drafts using Claude."""
@click.option("--reextract", is_flag=True, help="Clear existing ideas and re-extract with current prompt")
@click.option("--draft", "reextract_draft", default=None, help="Specific draft to re-extract (with --reextract)")
@click.pass_context
def ideas(ctx, name: str | None, extract_all: bool, limit: int, batch: int, cheap: bool,
reextract: bool, reextract_draft: str | None):
"""Extract, score, and filter technical ideas from drafts."""
if ctx.invoked_subcommand is not None:
return
from .analyzer import Analyzer
cfg = _get_config()
@@ -823,7 +1139,24 @@ def ideas(name: str | None, extract_all: bool, limit: int, batch: int, cheap: bo
analyzer = Analyzer(cfg, db)
try:
if extract_all:
if reextract:
# Clear existing ideas, then re-extract
deleted = db.delete_ideas(draft_name=reextract_draft)
if reextract_draft:
console.print(f"Cleared [bold]{deleted}[/] ideas for {reextract_draft}")
idea_list = analyzer.extract_ideas(reextract_draft, use_cache=True)
if idea_list:
console.print(f"Re-extracted [bold green]{len(idea_list)}[/] ideas:")
for idea in idea_list:
console.print(f" [{idea.get('type', '?')}] [bold]{idea['title']}[/]")
console.print(f" {idea['description']}\n")
else:
console.print("[red]Re-extraction failed or no ideas found[/]")
else:
console.print(f"Cleared [bold]{deleted}[/] ideas from all drafts")
count = analyzer.extract_all_ideas(limit=limit, batch_size=batch, cheap=cheap)
console.print(f"Re-extracted ideas from [bold green]{count}[/] drafts")
elif extract_all:
count = analyzer.extract_all_ideas(limit=limit, batch_size=batch, cheap=cheap)
console.print(f"Extracted ideas from [bold green]{count}[/] drafts")
elif name:
@@ -836,7 +1169,166 @@ def ideas(name: str | None, extract_all: bool, limit: int, batch: int, cheap: bo
else:
console.print("[red]Extraction failed or no ideas found[/]")
else:
console.print("Provide a draft name or use --all")
console.print("Use --name DRAFT, --all, or a subcommand: ideas score / ideas filter")
finally:
db.close()
@ideas.command("score")
@click.option("--cheap/--quality", default=True, help="Use Haiku (cheap) vs Sonnet (quality)")
@click.option("--batch", "-b", default=20, help="Ideas per API call (default 20)")
def ideas_score(cheap: bool, batch: int):
"""Score ideas for novelty (1=generic, 5=genuinely novel)."""
from .analyzer import Analyzer
cfg = _get_config()
db = Database(cfg)
analyzer = Analyzer(cfg, db)
try:
stats = analyzer.score_idea_novelty(batch_size=batch, cheap=cheap)
if stats["scored_count"] == 0:
return
# Show distribution table
dist = db.idea_score_distribution()
table = Table(title="Novelty Score Distribution")
table.add_column("Score", style="bold", justify="center")
table.add_column("Label", style="dim")
table.add_column("Count", justify="right")
table.add_column("Bar", min_width=30)
labels = {
1: "Generic building block",
2: "Obvious extension",
3: "Useful but expected",
4: "Interesting contribution",
5: "Genuinely novel",
}
max_count = max(dist.values()) if dist else 1
for score in range(1, 6):
count = dist.get(score, 0)
bar_len = int(30 * count / max_count) if max_count > 0 else 0
table.add_row(
str(score), labels[score], str(count),
"[green]" + "#" * bar_len + "[/]"
)
total = sum(dist.values())
unscored = db.idea_count() - total
console.print(table)
console.print(f"\nTotal scored: [bold]{total}[/] | Unscored: {unscored} | Avg: [bold]{stats['avg_score']:.1f}[/]")
finally:
db.close()
@ideas.command("filter")
@click.option("--min-score", "-m", default=2, help="Remove ideas below this score (default 2)")
@click.option("--dry-run/--execute", default=True, help="Preview (default) or actually delete")
def ideas_filter(min_score: int, dry_run: bool):
"""Filter out low-novelty ideas by score threshold."""
cfg = _get_config()
db = Database(cfg)
try:
candidates = db.ideas_below_score(min_score)
if not candidates:
console.print(f"No ideas with novelty_score < {min_score}.")
return
# Show what would be removed
table = Table(
title=f"Ideas with novelty_score < {min_score} "
f"({'DRY RUN' if dry_run else 'WILL DELETE'})"
)
table.add_column("Score", style="bold", justify="center")
table.add_column("Idea", style="cyan", max_width=40)
table.add_column("Draft", max_width=50)
table.add_column("Description", max_width=60)
for idea in candidates[:50]: # Show first 50
table.add_row(
str(idea["novelty_score"]),
idea["title"],
idea["draft_title"],
idea["description"][:60] + ("..." if len(idea["description"]) > 60 else ""),
)
console.print(table)
if len(candidates) > 50:
console.print(f" ... and {len(candidates) - 50} more")
console.print(f"\nTotal to remove: [bold red]{len(candidates)}[/] / {db.idea_count()} ideas")
if not dry_run:
deleted = db.delete_low_score_ideas(min_score)
console.print(f"[bold red]Deleted {deleted} low-novelty ideas.[/]")
console.print(f"Remaining ideas: [bold green]{db.idea_count()}[/]")
else:
console.print("[dim]Use --execute to actually delete.[/]")
finally:
db.close()
# ── dedup-ideas ─────────────────────────────────────────────────────────
@main.command("dedup-ideas")
@click.option("--threshold", "-t", default=0.85, type=float,
help="Cosine similarity threshold for merging (default 0.85)")
@click.option("--dry-run/--execute", default=True,
help="Preview merges (default) vs actually delete duplicates")
@click.option("--draft", "draft_name", default=None,
help="Limit to a single draft name")
def dedup_ideas(threshold: float, dry_run: bool, draft_name: str | None):
"""Deduplicate similar ideas within each draft using embedding similarity."""
from .analyzer import Analyzer
cfg = _get_config()
db = Database(cfg)
analyzer = Analyzer(cfg, db)
try:
mode = "[bold yellow]DRY RUN[/]" if dry_run else "[bold red]EXECUTE[/]"
console.print(f"\n{mode} — Deduplicating ideas (threshold={threshold})")
if draft_name:
console.print(f"Limiting to draft: [bold]{draft_name}[/]")
console.print()
result = analyzer.dedup_ideas(
threshold=threshold, dry_run=dry_run, draft_name=draft_name
)
if result["examples"]:
table = Table(title="Merge Candidates" if dry_run else "Merged Ideas")
table.add_column("Draft", style="dim", max_width=40)
table.add_column("Keep", style="green")
table.add_column("Drop", style="red")
table.add_column("Similarity", justify="right")
for ex in result["examples"]:
table.add_row(
ex["draft"].split("/")[-1][:40],
ex["keep"],
ex["drop"],
f"{ex['similarity']:.3f}",
)
console.print(table)
console.print()
action = "Would remove" if dry_run else "Removed"
console.print(
f"Ideas before: [bold]{result['total_before']}[/] | "
f"{action}: [bold]{result['merged_count']}[/] | "
f"After: [bold]{result['total_after']}[/]"
)
if dry_run and result["merged_count"] > 0:
console.print(
"\n[dim]Run with --execute to apply these merges.[/]"
)
finally:
db.close()
@@ -2024,3 +2516,163 @@ def observatory_diff(since: str | None):
console.print(f" [{d.get('source', '?')}] {d.get('name', '?')}: {d.get('title', '')[:60]}")
finally:
db.close()
# ── monitor ─────────────────────────────────────────────────────────────
@main.group()
def monitor():
"""Monitor IETF Datatracker for new AI/agent drafts."""
pass
@monitor.command("run")
@click.option("--analyze/--no-analyze", default=True, help="Analyze new drafts")
@click.option("--embed/--no-embed", default=True, help="Generate embeddings")
@click.option("--ideas/--no-ideas", default=True, help="Extract ideas")
def monitor_run(analyze, embed, ideas):
"""Run one monitoring cycle: fetch -> analyze -> embed -> ideas."""
from .analyzer import Analyzer
from .embeddings import Embedder
from .fetcher import Fetcher
cfg = _get_config()
db = Database(cfg)
run_id = db.start_monitor_run()
stats = {
"new_drafts_found": 0,
"drafts_analyzed": 0,
"drafts_embedded": 0,
"ideas_extracted": 0,
}
try:
console.print("[bold]Monitor run started[/]")
# Determine since date from last successful run
last_run = db.get_last_successful_run()
since = last_run["completed_at"][:10] if last_run and last_run.get("completed_at") else cfg.fetch_since
console.print(f" Fetching drafts since: [cyan]{since}[/]")
# Fetch new drafts
fetcher = Fetcher(cfg)
try:
existing_count = db.count_drafts()
drafts = fetcher.search_drafts(keywords=list(cfg.search_keywords), since=since)
for draft in drafts:
db.upsert_draft(draft)
# Download text for any missing
missing_text = db.drafts_without_text()
if missing_text:
console.print(f" Downloading text for [bold]{len(missing_text)}[/] drafts...")
texts = fetcher.download_texts(missing_text)
for name, text in texts.items():
draft = db.get_draft(name)
if draft:
draft.full_text = text
db.upsert_draft(draft)
finally:
fetcher.close()
new_count = db.count_drafts() - existing_count
stats["new_drafts_found"] = max(new_count, 0)
console.print(f" New drafts found: [bold green]{stats['new_drafts_found']}[/]")
# Analyze unrated drafts
if analyze:
unrated = db.unrated_drafts(limit=200)
if unrated:
console.print(f" Analyzing [bold]{len(unrated)}[/] unrated drafts...")
analyzer = Analyzer(cfg, db)
count = analyzer.rate_all_unrated(limit=200)
stats["drafts_analyzed"] = count
console.print(f" Analyzed: [bold green]{count}[/]")
# Embed missing drafts
if embed:
missing_embed = db.drafts_without_embeddings(limit=500)
if missing_embed:
console.print(f" Embedding [bold]{len(missing_embed)}[/] drafts...")
embedder = Embedder(cfg, db)
count = embedder.embed_all_missing()
stats["drafts_embedded"] = count
console.print(f" Embedded: [bold green]{count}[/]")
# Extract ideas
if ideas:
missing_ideas = db.drafts_without_ideas(limit=500)
if missing_ideas:
console.print(f" Extracting ideas from [bold]{len(missing_ideas)}[/] drafts...")
analyzer = Analyzer(cfg, db)
count = analyzer.extract_all_ideas(limit=500, batch_size=5, cheap=True)
stats["ideas_extracted"] = count
console.print(f" Ideas extracted from: [bold green]{count}[/] drafts")
db.complete_monitor_run(run_id, stats)
console.print("\n[bold green]Monitor run completed successfully[/]")
except Exception as e:
db.fail_monitor_run(run_id, str(e))
console.print(f"\n[bold red]Monitor run failed:[/] {e}")
raise
finally:
db.close()
@monitor.command("status")
def monitor_status():
"""Show monitoring status and recent runs."""
cfg = _get_config()
db = Database(cfg)
try:
runs = db.get_monitor_runs(limit=20)
last = db.get_last_successful_run()
# Unprocessed counts
unrated = len(db.unrated_drafts(limit=9999))
unembedded = len(db.drafts_without_embeddings(limit=9999))
no_ideas = len(db.drafts_without_ideas(limit=9999))
console.print("\n[bold]Monitor Status[/]\n")
if last:
console.print(f" Last successful run: [green]{last['completed_at']}[/]")
console.print(f" Duration: {last['duration_seconds']:.1f}s")
console.print(f" New drafts: {last['new_drafts_found']}")
else:
console.print(" [yellow]No successful runs yet[/]")
console.print(f"\n[bold]Unprocessed[/]")
console.print(f" Unrated: [{'yellow' if unrated > 0 else 'green'}]{unrated}[/]")
console.print(f" Unembedded: [{'yellow' if unembedded > 0 else 'green'}]{unembedded}[/]")
console.print(f" No ideas: [{'yellow' if no_ideas > 0 else 'green'}]{no_ideas}[/]")
if runs:
console.print(f"\n[bold]Recent Runs[/] ({len(runs)} total)\n")
table = Table()
table.add_column("#", justify="right", width=4)
table.add_column("Started", width=20)
table.add_column("Duration", justify="right", width=8)
table.add_column("Status", width=10)
table.add_column("New", justify="right", width=5)
table.add_column("Analyzed", justify="right", width=8)
table.add_column("Embedded", justify="right", width=8)
table.add_column("Ideas", justify="right", width=6)
for r in runs:
status_style = {"completed": "green", "failed": "red", "running": "yellow"}.get(r["status"], "dim")
table.add_row(
str(r["id"]),
r["started_at"][:19] if r["started_at"] else "",
f"{r['duration_seconds']:.1f}s" if r["duration_seconds"] else "-",
f"[{status_style}]{r['status']}[/{status_style}]",
str(r["new_drafts_found"]),
str(r["drafts_analyzed"]),
str(r["drafts_embedded"]),
str(r["ideas_extracted"]),
)
console.print(table)
finally:
db.close()

View File

@@ -106,6 +106,14 @@ CREATE TABLE IF NOT EXISTS ideas (
CREATE INDEX IF NOT EXISTS idx_ideas_draft ON ideas(draft_name);
-- Idea embeddings (for clustering)
CREATE TABLE IF NOT EXISTS idea_embeddings (
idea_id INTEGER PRIMARY KEY REFERENCES ideas(id),
model TEXT NOT NULL,
vector BLOB NOT NULL,
created_at TEXT
);
-- Gap analysis results
CREATE TABLE IF NOT EXISTS gaps (
id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -184,6 +192,20 @@ CREATE TABLE IF NOT EXISTS gap_history (
recorded_at TEXT
);
-- Monitor runs
CREATE TABLE IF NOT EXISTS monitor_runs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
started_at TEXT NOT NULL,
completed_at TEXT,
status TEXT DEFAULT 'running',
new_drafts_found INTEGER DEFAULT 0,
drafts_analyzed INTEGER DEFAULT 0,
drafts_embedded INTEGER DEFAULT 0,
ideas_extracted INTEGER DEFAULT 0,
error_message TEXT DEFAULT '',
duration_seconds REAL DEFAULT 0
);
-- Triggers to keep FTS index in sync
CREATE TRIGGER IF NOT EXISTS drafts_ai AFTER INSERT ON drafts BEGIN
INSERT INTO drafts_fts(rowid, name, title, abstract, full_text)
@@ -234,6 +256,12 @@ class Database:
for col, typedef in migrations:
if col not in cols:
self._conn.execute(f"ALTER TABLE drafts ADD COLUMN {col} {typedef}")
# ideas table migrations
idea_cols = {r[1] for r in self._conn.execute("PRAGMA table_info(ideas)").fetchall()}
if "novelty_score" not in idea_cols:
self._conn.execute("ALTER TABLE ideas ADD COLUMN novelty_score INTEGER")
self._conn.commit()
def close(self) -> None:
@@ -501,12 +529,13 @@ class Database:
ORDER BY da.author_order""",
(draft_name,),
).fetchall()
cols = rows[0].keys() if rows else []
return [Author(
person_id=r["person_id"], name=r["name"],
ascii_name=r.get("ascii_name", ""),
affiliation=r.get("affiliation", ""),
resource_uri=r.get("resource_uri", ""),
fetched_at=r.get("fetched_at"),
ascii_name=r["ascii_name"] if "ascii_name" in cols else "",
affiliation=r["affiliation"] if "affiliation" in cols else "",
resource_uri=r["resource_uri"] if "resource_uri" in cols else "",
fetched_at=r["fetched_at"] if "fetched_at" in cols else None,
) for r in rows]
def drafts_without_authors(self, limit: int = 500) -> list[str]:
@@ -624,13 +653,42 @@ class Database:
)
self.conn.commit()
def delete_ideas(self, draft_name: str | None = None) -> int:
"""Delete ideas from the ideas table.
Args:
draft_name: If provided, delete only ideas for this draft.
If None, delete all ideas.
Returns:
Number of rows deleted.
"""
if draft_name:
self.conn.execute(
"DELETE FROM idea_embeddings WHERE idea_id IN (SELECT id FROM ideas WHERE draft_name = ?)", (draft_name,)
)
cursor = self.conn.execute(
"DELETE FROM ideas WHERE draft_name = ?", (draft_name,)
)
else:
self.conn.execute("DELETE FROM idea_embeddings")
cursor = self.conn.execute("DELETE FROM ideas")
self.conn.commit()
return cursor.rowcount
def get_ideas_for_draft(self, draft_name: str) -> list[dict]:
rows = self.conn.execute(
"SELECT * FROM ideas WHERE draft_name = ?", (draft_name,)
).fetchall()
return [{"title": r["title"], "description": r["description"],
return [{"id": r["id"], "title": r["title"], "description": r["description"],
"type": r["idea_type"], "draft_name": r["draft_name"]} for r in rows]
def delete_idea(self, idea_id: int) -> None:
"""Delete a single idea and its embedding by ID."""
self.conn.execute("DELETE FROM idea_embeddings WHERE idea_id = ?", (idea_id,))
self.conn.execute("DELETE FROM ideas WHERE id = ?", (idea_id,))
self.conn.commit()
def drafts_without_ideas(self, limit: int = 500) -> list[str]:
rows = self.conn.execute(
"""SELECT d.name FROM drafts d
@@ -653,6 +711,103 @@ class Database:
def idea_count(self) -> int:
return self.conn.execute("SELECT COUNT(*) FROM ideas").fetchone()[0]
def ideas_with_drafts(self, unscored_only: bool = False, limit: int = 5000) -> list[dict]:
"""Return ideas joined with draft title, optionally only unscored ones."""
where = "WHERE i.novelty_score IS NULL" if unscored_only else ""
rows = self.conn.execute(
f"""SELECT i.id, i.draft_name, i.title, i.description, i.idea_type,
i.novelty_score, d.title AS draft_title
FROM ideas i JOIN drafts d ON i.draft_name = d.name
{where}
ORDER BY i.id LIMIT ?""",
(limit,),
).fetchall()
return [dict(r) for r in rows]
def update_idea_score(self, idea_id: int, score: int) -> None:
"""Set the novelty_score for a single idea."""
self.conn.execute(
"UPDATE ideas SET novelty_score = ? WHERE id = ?",
(score, idea_id),
)
self.conn.commit()
def update_idea_scores_bulk(self, scores: dict[int, int]) -> None:
"""Bulk-update novelty scores. scores maps idea_id -> score."""
self.conn.executemany(
"UPDATE ideas SET novelty_score = ? WHERE id = ?",
[(score, idea_id) for idea_id, score in scores.items()],
)
self.conn.commit()
def delete_low_score_ideas(self, min_score: int) -> int:
"""Delete ideas with novelty_score below min_score. Returns count deleted."""
# Also clean up associated idea embeddings
self.conn.execute(
"""DELETE FROM idea_embeddings WHERE idea_id IN
(SELECT id FROM ideas WHERE novelty_score IS NOT NULL AND novelty_score < ?)""",
(min_score,),
)
cursor = self.conn.execute(
"DELETE FROM ideas WHERE novelty_score IS NOT NULL AND novelty_score < ?",
(min_score,),
)
self.conn.commit()
return cursor.rowcount
def idea_score_distribution(self) -> dict[int, int]:
"""Return {score: count} for scored ideas."""
rows = self.conn.execute(
"SELECT novelty_score, COUNT(*) as cnt FROM ideas "
"WHERE novelty_score IS NOT NULL GROUP BY novelty_score ORDER BY novelty_score"
).fetchall()
return {r["novelty_score"]: r["cnt"] for r in rows}
def ideas_below_score(self, min_score: int) -> list[dict]:
"""Return ideas with novelty_score below min_score."""
rows = self.conn.execute(
"""SELECT i.id, i.draft_name, i.title, i.description, i.novelty_score,
d.title AS draft_title
FROM ideas i JOIN drafts d ON i.draft_name = d.name
WHERE i.novelty_score IS NOT NULL AND i.novelty_score < ?
ORDER BY i.novelty_score, i.title""",
(min_score,),
).fetchall()
return [dict(r) for r in rows]
# --- Idea Embeddings ---
def store_idea_embedding(self, idea_id: int, model: str, vector: np.ndarray) -> None:
self.conn.execute(
"""INSERT INTO idea_embeddings (idea_id, model, vector, created_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(idea_id) DO UPDATE SET
model=excluded.model, vector=excluded.vector, created_at=excluded.created_at
""",
(idea_id, model, vector.astype(np.float32).tobytes(),
datetime.now(timezone.utc).isoformat()),
)
self.conn.commit()
def all_idea_embeddings(self) -> dict[int, np.ndarray]:
rows = self.conn.execute("SELECT idea_id, vector FROM idea_embeddings").fetchall()
return {
r["idea_id"]: np.frombuffer(r["vector"], dtype=np.float32)
for r in rows
}
def ideas_without_embeddings(self, limit: int = 500) -> list[dict]:
rows = self.conn.execute(
"""SELECT i.id, i.title, i.description, i.idea_type, i.draft_name
FROM ideas i
LEFT JOIN idea_embeddings ie ON i.id = ie.idea_id
WHERE ie.idea_id IS NULL
LIMIT ?""",
(limit,),
).fetchall()
return [{"id": r["id"], "title": r["title"], "description": r["description"],
"type": r["idea_type"], "draft_name": r["draft_name"]} for r in rows]
# --- Gaps ---
def insert_gaps(self, gaps: list[dict]) -> None:
@@ -981,6 +1136,250 @@ class Database:
for r in rows
]
# --- Working Groups ---
def wg_summary(self) -> list[dict]:
"""Return per-WG summary: group, draft_count, avg scores, categories, idea_count.
Excludes 'none' (individual submissions) — those are returned separately.
"""
rows = self.conn.execute("""
SELECT d."group" as wg, COUNT(*) as draft_count,
AVG(r.novelty) as avg_novelty, AVG(r.maturity) as avg_maturity,
AVG(r.overlap) as avg_overlap, AVG(r.momentum) as avg_momentum,
AVG(r.relevance) as avg_relevance,
(SELECT COUNT(*) FROM ideas i WHERE i.draft_name IN
(SELECT name FROM drafts WHERE "group" = d."group")) as idea_count
FROM drafts d
LEFT JOIN ratings r ON d.name = r.draft_name
WHERE d."group" IS NOT NULL AND d."group" != '' AND d."group" != 'none'
GROUP BY d."group"
ORDER BY draft_count DESC
""").fetchall()
# Build categories per WG from a separate query
cat_rows = self.conn.execute("""
SELECT d."group" as wg, r.categories
FROM drafts d JOIN ratings r ON d.name = r.draft_name
WHERE d."group" IS NOT NULL AND d."group" != '' AND d."group" != 'none'
""").fetchall()
wg_cats: dict[str, dict[str, int]] = {}
for cr in cat_rows:
wg = cr["wg"]
if wg not in wg_cats:
wg_cats[wg] = {}
try:
for c in json.loads(cr["categories"]):
c = normalize_category(c)
wg_cats[wg][c] = wg_cats[wg].get(c, 0) + 1
except (json.JSONDecodeError, TypeError):
pass
results = []
for r in rows:
results.append({
"wg": r["wg"],
"draft_count": r["draft_count"],
"avg_novelty": round(r["avg_novelty"] or 0, 1),
"avg_maturity": round(r["avg_maturity"] or 0, 1),
"avg_overlap": round(r["avg_overlap"] or 0, 1),
"avg_momentum": round(r["avg_momentum"] or 0, 1),
"avg_relevance": round(r["avg_relevance"] or 0, 1),
"categories": wg_cats.get(r["wg"], {}),
"idea_count": r["idea_count"],
})
return results
def wg_drafts(self, wg: str) -> list[Draft]:
"""Return all drafts for a specific working group."""
rows = self.conn.execute(
'SELECT * FROM drafts WHERE "group" = ? ORDER BY time DESC', (wg,)
).fetchall()
return [self._row_to_draft(r) for r in rows]
def wg_category_matrix(self) -> dict[str, dict[str, int]]:
"""Return {wg: {category: count}} matrix for all WGs (excluding 'none')."""
rows = self.conn.execute("""
SELECT d."group" as wg, r.categories
FROM drafts d
JOIN ratings r ON d.name = r.draft_name
WHERE d."group" IS NOT NULL AND d."group" != '' AND d."group" != 'none'
""").fetchall()
matrix: dict[str, dict[str, int]] = {}
for r in rows:
wg = r["wg"]
if wg not in matrix:
matrix[wg] = {}
try:
for c in json.loads(r["categories"]):
c = normalize_category(c)
matrix[wg][c] = matrix[wg].get(c, 0) + 1
except (json.JSONDecodeError, TypeError):
pass
return matrix
def wg_idea_overlap(self) -> list[dict]:
"""Find ideas that appear across multiple WGs — signals for alignment.
Returns list of {idea_title, wgs: [{wg, draft_name, draft_title}], wg_count}.
"""
rows = self.conn.execute("""
SELECT i.title as idea_title, i.description, d."group" as wg,
d.name as draft_name, d.title as draft_title
FROM ideas i
JOIN drafts d ON i.draft_name = d.name
WHERE d."group" IS NOT NULL AND d."group" != ''
ORDER BY i.title, d."group"
""").fetchall()
# Group by idea title
from collections import defaultdict
idea_groups: dict[str, list[dict]] = defaultdict(list)
for r in rows:
idea_groups[r["idea_title"]].append({
"wg": r["wg"],
"draft_name": r["draft_name"],
"draft_title": r["draft_title"],
})
# Only keep ideas spanning 2+ distinct WGs
results = []
for title, entries in idea_groups.items():
wgs = set(e["wg"] for e in entries)
if len(wgs) >= 2:
results.append({
"idea_title": title,
"wgs": entries,
"wg_count": len(wgs),
"wg_names": sorted(wgs),
})
return sorted(results, key=lambda x: x["wg_count"], reverse=True)
def individual_vs_wg_categories(self) -> dict[str, dict[str, int]]:
"""Compare category distribution: individual submissions vs WG-adopted.
Returns {"individual": {cat: count}, "wg_adopted": {cat: count}}.
"""
rows = self.conn.execute("""
SELECT CASE WHEN d."group" = 'none' OR d."group" IS NULL THEN 'individual'
ELSE 'wg_adopted' END as stream,
r.categories
FROM drafts d
JOIN ratings r ON d.name = r.draft_name
""").fetchall()
result: dict[str, dict[str, int]] = {"individual": {}, "wg_adopted": {}}
for r in rows:
stream = r["stream"]
try:
for c in json.loads(r["categories"]):
c = normalize_category(c)
result[stream][c] = result[stream].get(c, 0) + 1
except (json.JSONDecodeError, TypeError):
pass
return result
def category_wg_spread(self) -> list[dict]:
"""For each category, which WGs contribute drafts? High spread = alignment opportunity.
Returns [{category, wgs: [{wg, count}], wg_count, total_drafts}].
"""
rows = self.conn.execute("""
SELECT d."group" as wg, r.categories
FROM drafts d
JOIN ratings r ON d.name = r.draft_name
WHERE d."group" IS NOT NULL AND d."group" != ''
""").fetchall()
from collections import defaultdict
cat_wgs: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
for r in rows:
wg = r["wg"]
try:
for c in json.loads(r["categories"]):
c = normalize_category(c)
cat_wgs[c][wg] += 1
except (json.JSONDecodeError, TypeError):
pass
results = []
for cat, wg_counts in cat_wgs.items():
wg_list = sorted(wg_counts.items(), key=lambda x: x[1], reverse=True)
results.append({
"category": cat,
"wgs": [{"wg": wg, "count": cnt} for wg, cnt in wg_list],
"wg_count": len(wg_list),
"total_drafts": sum(wg_counts.values()),
})
return sorted(results, key=lambda x: x["wg_count"], reverse=True)
# --- Monitor Runs ---
def start_monitor_run(self) -> int:
now = datetime.now(timezone.utc).isoformat()
cur = self.conn.execute(
"INSERT INTO monitor_runs (started_at, status) VALUES (?, 'running')",
(now,),
)
self.conn.commit()
return cur.lastrowid
def complete_monitor_run(self, run_id: int, stats: dict) -> None:
now = datetime.now(timezone.utc).isoformat()
started = self.conn.execute(
"SELECT started_at FROM monitor_runs WHERE id = ?", (run_id,)
).fetchone()
duration = 0.0
if started:
try:
start_dt = datetime.fromisoformat(started["started_at"])
duration = (datetime.now(timezone.utc) - start_dt).total_seconds()
except (ValueError, TypeError):
pass
self.conn.execute(
"""UPDATE monitor_runs SET
status='completed', completed_at=?,
new_drafts_found=?, drafts_analyzed=?,
drafts_embedded=?, ideas_extracted=?,
duration_seconds=?
WHERE id=?""",
(now, stats.get("new_drafts_found", 0), stats.get("drafts_analyzed", 0),
stats.get("drafts_embedded", 0), stats.get("ideas_extracted", 0),
duration, run_id),
)
self.conn.commit()
def fail_monitor_run(self, run_id: int, error: str) -> None:
now = datetime.now(timezone.utc).isoformat()
started = self.conn.execute(
"SELECT started_at FROM monitor_runs WHERE id = ?", (run_id,)
).fetchone()
duration = 0.0
if started:
try:
start_dt = datetime.fromisoformat(started["started_at"])
duration = (datetime.now(timezone.utc) - start_dt).total_seconds()
except (ValueError, TypeError):
pass
self.conn.execute(
"""UPDATE monitor_runs SET
status='failed', completed_at=?, error_message=?, duration_seconds=?
WHERE id=?""",
(now, error, duration, run_id),
)
self.conn.commit()
def get_monitor_runs(self, limit: int = 20) -> list[dict]:
rows = self.conn.execute(
"SELECT * FROM monitor_runs ORDER BY started_at DESC LIMIT ?", (limit,)
).fetchall()
return [dict(r) for r in rows]
def get_last_successful_run(self) -> dict | None:
row = self.conn.execute(
"SELECT * FROM monitor_runs WHERE status='completed' ORDER BY started_at DESC LIMIT 1"
).fetchone()
return dict(row) if row else None
# --- Helpers ---
@staticmethod

View File

@@ -1673,3 +1673,143 @@ class Reporter:
path = self.output_dir / "co-occurrence.md"
path.write_text(report)
return str(path)
def wg_report(self) -> str:
"""Generate working group analysis report."""
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
summaries = self.db.wg_summary()
spread = self.db.category_wg_spread()
idea_overlaps = self.db.wg_idea_overlap()
indiv_vs_wg = self.db.individual_vs_wg_categories()
total = self.db.count_drafts()
indiv_count = self.db.conn.execute(
'SELECT COUNT(*) FROM drafts WHERE "group" = \'none\' OR "group" IS NULL'
).fetchone()[0]
wg_count = total - indiv_count
lines = [
f"# Working Group Analysis",
f"*Generated {now}{total} drafts ({wg_count} WG-adopted, {indiv_count} individual)*\n",
]
# WG summary table
lines.extend([
"## Working Group Overview\n",
"| WG | Drafts | Ideas | Novelty | Maturity | Overlap | Momentum | Relevance |",
"|:---|-------:|------:|--------:|---------:|--------:|---------:|----------:|",
])
for s in summaries:
lines.append(
f"| **{s['wg']}** | {s['draft_count']} | {s['idea_count']} "
f"| {s['avg_novelty']} | {s['avg_maturity']} | {s['avg_overlap']} "
f"| {s['avg_momentum']} | {s['avg_relevance']} |"
)
# Category spread — where topics live across WGs
multi_wg = [s for s in spread if s["wg_count"] >= 2
and not all(w["wg"] == "none" for w in s["wgs"])]
if multi_wg:
lines.extend([
"\n## Cross-WG Category Spread\n",
"Categories appearing in multiple WGs — potential coordination or alignment needed.\n",
"| Category | WG Count | Total Drafts | WGs |",
"|:---------|:--------:|-------------:|:----|",
])
for s in multi_wg:
real_wgs = [f"{w['wg']}({w['count']})" for w in s["wgs"] if w["wg"] != "none"]
lines.append(
f"| {s['category']} | {s['wg_count']} | {s['total_drafts']} "
f"| {', '.join(real_wgs)} |"
)
# Idea overlap across WGs
cross_wg_ideas = [o for o in idea_overlaps
if not all(w == "none" for w in o["wg_names"])]
if cross_wg_ideas:
lines.extend([
"\n## Cross-WG Idea Overlap\n",
"Same technical ideas appearing in different WGs — strongest signals for alignment.\n",
])
for o in cross_wg_ideas[:30]:
real_wgs = [w for w in o["wg_names"] if w != "none"]
lines.append(f"### {o['idea_title']} ({len(real_wgs)} WGs: {', '.join(real_wgs)})\n")
for entry in o["wgs"]:
if entry["wg"] != "none":
lines.append(f"- **[{entry['wg']}]** [{entry['draft_name']}]"
f"(https://datatracker.ietf.org/doc/{entry['draft_name']}/) — "
f"{entry['draft_title']}")
lines.append("")
# Individual vs WG comparison
indiv = indiv_vs_wg["individual"]
adopted = indiv_vs_wg["wg_adopted"]
all_cats = sorted(set(list(indiv.keys()) + list(adopted.keys())))
lines.extend([
"\n## Individual vs WG-Adopted Distribution\n",
"| Category | Individual | WG-Adopted | Assessment |",
"|:---------|----------:|-----------:|:-----------|",
])
consolidation_candidates = []
for cat in all_cats:
i_count = indiv.get(cat, 0)
w_count = adopted.get(cat, 0)
if i_count >= 5 and w_count == 0:
assessment = "**Needs WG** — high individual activity, no WG home"
consolidation_candidates.append((cat, i_count))
elif i_count >= 3 and w_count >= 1:
assessment = "WG exists — individual drafts could target it"
elif w_count > i_count and i_count > 0:
assessment = "WG leading"
elif w_count == 0 and i_count > 0:
assessment = "Individual only"
else:
assessment = "-"
lines.append(f"| {cat} | {i_count} | {w_count} | {assessment} |")
# Consolidation recommendations
if consolidation_candidates:
lines.extend([
"\n## Consolidation Candidates\n",
"Categories with significant individual draft activity but no WG — "
"candidates for new WG charter or BoF.\n",
])
for cat, count in sorted(consolidation_candidates, key=lambda x: x[1], reverse=True):
lines.append(f"### {cat} ({count} individual drafts)\n")
rows = self.db.conn.execute("""
SELECT d.name, d.title, r.summary FROM drafts d
JOIN ratings r ON d.name = r.draft_name
WHERE (d."group" = 'none' OR d."group" IS NULL)
AND r.categories LIKE ?
ORDER BY (r.novelty * 0.30 + r.relevance * 0.25 + r.maturity * 0.20
+ r.momentum * 0.15 + (6 - r.overlap) * 0.10) DESC
LIMIT 8
""", (f"%{cat}%",)).fetchall()
for row in rows:
lines.append(
f"- [{row['name']}](https://datatracker.ietf.org/doc/{row['name']}/) — "
f"{row['title']}"
)
lines.append("")
# Submission targets
lines.extend([
"\n## Recommended Submission Targets\n",
"For each category, the best WG to submit new work to.\n",
"| Category | Best WG | Alternatives |",
"|:---------|:--------|:-------------|",
])
for s in spread:
real_wgs = [w for w in s["wgs"] if w["wg"] != "none"]
if not real_wgs:
lines.append(f"| {s['category']} | *Individual submission* | - |")
else:
best = real_wgs[0]["wg"]
alts = ", ".join(f"{w['wg']}({w['count']})" for w in real_wgs[1:3]) or "-"
lines.append(f"| {s['category']} | **{best}** | {alts} |")
report = "\n".join(lines)
path = self.output_dir / "wg-analysis.md"
path.write_text(report)
return str(path)

80
src/webui/PLAN.md Normal file
View File

@@ -0,0 +1,80 @@
# IETF Draft Analyzer — Web Dashboard Architecture
## Overview
A read-only Flask dashboard for exploring and visualizing 361+ IETF Internet-Drafts on AI/agent topics. All data comes from the existing SQLite database (`data/drafts.db`) via the `Database` class from `src/ietf_analyzer/db.py`.
## Tech Stack
- **Backend**: Flask (simple routes, no blueprints)
- **Database**: Existing SQLite via `ietf_analyzer.db.Database` (read-only)
- **CSS**: Tailwind CSS via CDN (dark theme: slate/gray palette)
- **Charts**: Plotly.js via CDN (all interactive charts rendered client-side)
- **Fonts**: Inter via Google Fonts CDN
## File Structure
```
src/webui/
__init__.py # Empty package init
app.py # Flask app, all routes
data.py # Data access layer (wraps Database queries, returns JSON-ready dicts)
templates/
base.html # Dark-themed base with sidebar nav, Tailwind, Plotly CDN
overview.html # Dashboard home: key stats, charts
drafts.html # Draft explorer: search, filter, sortable table
draft_detail.html # Single draft detail page
ideas.html # Ideas explorer with type breakdown
gaps.html # Gap analysis display
ratings.html # Rating distributions and comparisons
landscape.html # UMAP/t-SNE scatter (embeddings)
authors.html # Author network and top contributors
about.html # About page with project info
```
## Pages & Routes
| Route | Template | Description |
|-------|----------|-------------|
| `/` | `overview.html` | Dashboard home: total drafts, rated count, author count, idea count, gap count. Charts: category treemap, timeline, score distribution histogram. |
| `/drafts` | `drafts.html` | Searchable, filterable, sortable table of all drafts with ratings. Pagination. Category chip filters. Score range slider. |
| `/drafts/<name>` | `draft_detail.html` | Single draft: all rating dimensions with notes, categories, authors, ideas extracted, references. |
| `/ideas` | `ideas.html` | All extracted ideas grouped by type. Bar chart of idea types. Searchable. |
| `/gaps` | `gaps.html` | Gap analysis results: severity badges, categories, evidence. |
| `/ratings` | `ratings.html` | Rating analytics: dimension distributions (violin/box), category radar profiles, top-scored drafts. |
| `/landscape` | `landscape.html` | Embedding scatter plot (pre-computed coordinates served as JSON). |
| `/authors` | `authors.html` | Top authors table, org contributions bar chart, co-author network graph. |
| `/about` | `about.html` | Project description, data freshness, counts. |
## Data Layer (`data.py`)
Thin wrapper around `Database` that returns plain dicts/lists ready for `jsonify()` or template rendering:
- `get_overview_stats()` — counts for drafts, ratings, authors, ideas, gaps
- `get_drafts_page(page, per_page, search, category, min_score, sort)` — paginated draft list with ratings
- `get_draft_detail(name)` — single draft + rating + authors + ideas + refs
- `get_category_counts()` — {category: count} for filter chips
- `get_rating_distributions()` — arrays for each dimension for Plotly
- `get_timeline_data()` — monthly counts by category for stacked area
- `get_ideas_by_type()` — grouped idea counts
- `get_all_gaps()` — gap list with severity
- `get_top_authors(limit)` — author leaderboard
- `get_org_data(limit)` — organization contributions
- `get_landscape_coords()` — pre-computed 2D coordinates + metadata
## Design System
- **Dark theme**: `bg-slate-900` body, `bg-slate-800` cards, `bg-slate-700` hover states
- **Accent**: Blue-500 (`#3b82f6`) for links, active states, charts
- **Text**: `text-slate-100` primary, `text-slate-400` secondary
- **Cards**: Rounded corners (`rounded-xl`), subtle border (`border-slate-700`)
- **Sidebar**: Fixed left, 240px wide, collapsible on mobile
- **Charts**: Plotly dark theme (`plotly_dark` template), consistent color palette
## Key Decisions
1. **Read-only**: No writes to DB. All data comes from CLI pipeline runs.
2. **Server-side rendering**: Templates with Jinja2, chart data passed as JSON.
3. **No build step**: All CSS/JS from CDN. Zero npm/webpack complexity.
4. **Reuse existing queries**: `data.py` calls `Database` methods directly.
5. **Responsive**: Tailwind responsive utilities, sidebar collapses to hamburger.

1
src/webui/__init__.py Normal file
View File

@@ -0,0 +1 @@
# IETF Draft Analyzer — Web Dashboard

297
src/webui/app.py Normal file
View File

@@ -0,0 +1,297 @@
"""IETF Draft Analyzer — Web Dashboard.
Run with: python src/webui/app.py
"""
from __future__ import annotations
import sys
from pathlib import Path
# Ensure project src is on path
_project_root = Path(__file__).resolve().parent.parent.parent
sys.path.insert(0, str(_project_root / "src"))
from flask import Flask, render_template, request, jsonify, abort, g
from webui.data import (
get_db,
get_overview_stats,
get_category_counts,
get_drafts_page,
get_draft_detail,
get_rating_distributions,
get_timeline_data,
get_ideas_by_type,
get_all_gaps,
get_gap_detail,
get_generated_drafts,
read_generated_draft,
get_top_authors,
get_org_data,
get_category_radar_data,
get_score_histogram,
get_coauthor_network,
get_cross_org_data,
get_landscape_tsne,
get_similarity_graph,
get_timeline_animation_data,
get_idea_clusters,
get_monitor_status,
get_author_network_full,
)
app = Flask(
__name__,
template_folder=str(Path(__file__).parent / "templates"),
)
app.config["SECRET_KEY"] = "ietf-dashboard-dev"
# --- Database lifecycle (per-request to avoid SQLite threading issues) ---
def db():
if "db" not in g:
g.db = get_db()
return g.db
# --- Routes ---
@app.route("/")
def overview():
stats = get_overview_stats(db())
categories = get_category_counts(db())
timeline = get_timeline_data(db())
scores = get_score_histogram(db())
radar = get_category_radar_data(db())
return render_template(
"overview.html",
stats=stats,
categories=categories,
timeline=timeline,
scores=scores,
radar=radar,
)
@app.route("/drafts")
def drafts():
page = request.args.get("page", 1, type=int)
search = request.args.get("q", "")
category = request.args.get("cat", "")
min_score = request.args.get("min_score", 0.0, type=float)
sort = request.args.get("sort", "score")
sort_dir = request.args.get("dir", "desc")
result = get_drafts_page(
db(),
page=page,
search=search,
category=category,
min_score=min_score,
sort=sort,
sort_dir=sort_dir,
)
categories = get_category_counts(db())
return render_template(
"drafts.html",
result=result,
categories=categories,
search=search,
current_cat=category,
min_score=min_score,
sort=sort,
sort_dir=sort_dir,
)
@app.route("/drafts/<path:name>")
def draft_detail(name: str):
detail = get_draft_detail(db(), name)
if not detail:
abort(404)
return render_template("draft_detail.html", draft=detail)
@app.route("/ideas")
def ideas():
data = get_ideas_by_type(db())
return render_template("ideas.html", data=data)
@app.route("/gaps")
def gaps():
gap_list = get_all_gaps(db())
generated = get_generated_drafts()
return render_template("gaps.html", gaps=gap_list, generated_drafts=generated)
@app.route("/gaps/demo")
def gaps_demo():
"""Show a pre-generated example draft so users can see output without API calls."""
generated = get_generated_drafts()
# Default to the first generated draft, or allow selection via query param
selected = request.args.get("file", "")
draft_text = None
draft_info = None
if selected:
draft_text = read_generated_draft(selected)
for g in generated:
if g["filename"] == selected:
draft_info = g
break
elif generated:
draft_info = generated[0]
draft_text = read_generated_draft(draft_info["filename"])
return render_template(
"gap_demo.html",
generated_drafts=generated,
draft_text=draft_text,
draft_info=draft_info,
selected=selected,
)
@app.route("/gaps/<int:gap_id>")
def gap_detail(gap_id: int):
gap = get_gap_detail(db(), gap_id)
if not gap:
abort(404)
generated = get_generated_drafts()
return render_template("gap_detail.html", gap=gap, generated_drafts=generated)
@app.route("/gaps/<int:gap_id>/generate", methods=["POST"])
def gap_generate(gap_id: int):
"""Trigger draft generation for a gap. Returns JSON with the generated text."""
gap = get_gap_detail(db(), gap_id)
if not gap:
return jsonify({"error": "Gap not found"}), 404
try:
from ietf_analyzer.config import Config
from ietf_analyzer.analyzer import Analyzer
from ietf_analyzer.draftgen import DraftGenerator
cfg = Config.load()
database = db()
analyzer = Analyzer(cfg, database)
generator = DraftGenerator(cfg, database, analyzer)
# Generate into a file named after the gap
slug = gap["topic"].lower().replace(" ", "-")[:40]
output_path = str(Path(_project_root) / "data" / "reports" / "generated-drafts" / f"draft-gap-{gap_id}-{slug}.txt")
path = generator.generate(gap["topic"], output_path=output_path)
draft_text = Path(path).read_text(errors="replace")
return jsonify({
"success": True,
"text": draft_text,
"filename": Path(path).name,
"path": path,
})
except Exception as e:
return jsonify({"error": str(e)}), 500
@app.route("/ratings")
def ratings():
distributions = get_rating_distributions(db())
radar = get_category_radar_data(db())
return render_template(
"ratings.html",
dist=distributions,
radar=radar,
)
@app.route("/landscape")
def landscape():
distributions = get_rating_distributions(db())
tsne_data = get_landscape_tsne(db())
return render_template(
"landscape.html",
dist=distributions,
tsne_data=tsne_data,
)
@app.route("/timeline")
def timeline_animation():
data = get_timeline_animation_data(db())
return render_template("timeline.html", animation=data)
@app.route("/idea-clusters")
def idea_clusters():
data = get_idea_clusters(db())
return render_template("idea_clusters.html", clusters=data)
@app.route("/similarity")
def similarity():
network = get_similarity_graph(db())
return render_template("similarity.html", network=network)
@app.route("/authors")
def authors():
top = get_top_authors(db(), limit=50)
orgs = get_org_data(db(), limit=20)
network = get_author_network_full(db())
cross_org = get_cross_org_data(db(), limit=20)
return render_template(
"authors.html",
authors=top,
orgs=orgs,
orgs_data=orgs,
network=network,
cross_org=cross_org,
)
@app.route("/monitor")
def monitor_page():
status = get_monitor_status(db())
return render_template("monitor.html", status=status)
@app.route("/about")
def about():
stats = get_overview_stats(db())
return render_template("about.html", stats=stats)
# --- API endpoints for AJAX (used by client-side charts) ---
@app.route("/api/drafts")
def api_drafts():
page = request.args.get("page", 1, type=int)
search = request.args.get("q", "")
category = request.args.get("cat", "")
min_score = request.args.get("min_score", 0.0, type=float)
sort = request.args.get("sort", "score")
sort_dir = request.args.get("dir", "desc")
return jsonify(
get_drafts_page(db(), page=page, search=search, category=category,
min_score=min_score, sort=sort, sort_dir=sort_dir)
)
@app.route("/api/stats")
def api_stats():
return jsonify(get_overview_stats(db()))
@app.route("/api/authors/network")
def api_author_network():
return jsonify(get_author_network_full(db()))
if __name__ == "__main__":
print("Starting IETF Draft Analyzer Dashboard on http://127.0.0.1:5000")
app.run(debug=True, host="127.0.0.1", port=5000)

767
src/webui/data.py Normal file
View File

@@ -0,0 +1,767 @@
"""Data access layer for the web dashboard.
Thin wrapper around ietf_analyzer.db.Database that returns plain dicts
ready for JSON serialization or Jinja2 template rendering.
"""
from __future__ import annotations
import json
import sys
from collections import Counter, defaultdict
from pathlib import Path
# Add project root to path so we can import ietf_analyzer
_project_root = Path(__file__).resolve().parent.parent.parent
if str(_project_root) not in sys.path:
sys.path.insert(0, str(_project_root / "src"))
from ietf_analyzer.config import Config
from ietf_analyzer.db import Database
def get_db() -> Database:
"""Get a Database instance using default config."""
config = Config.load()
return Database(config)
def get_overview_stats(db: Database) -> dict:
"""Return high-level stats for the dashboard home page."""
total_drafts = db.count_drafts()
rated_pairs = db.drafts_with_ratings(limit=1000)
rated_count = len(rated_pairs)
author_count = db.author_count()
idea_count = db.idea_count()
gaps = db.all_gaps()
input_tok, output_tok = db.total_tokens_used()
return {
"total_drafts": total_drafts,
"rated_count": rated_count,
"author_count": author_count,
"idea_count": idea_count,
"gap_count": len(gaps),
"input_tokens": input_tok,
"output_tokens": output_tok,
}
def get_category_counts(db: Database) -> dict[str, int]:
"""Return {category: draft_count} for all categories."""
pairs = db.drafts_with_ratings(limit=1000)
counts: dict[str, int] = Counter()
for _, rating in pairs:
for cat in rating.categories:
counts[cat] += 1
return dict(counts.most_common())
def get_drafts_page(
db: Database,
page: int = 1,
per_page: int = 50,
search: str = "",
category: str = "",
min_score: float = 0.0,
sort: str = "score",
sort_dir: str = "desc",
) -> dict:
"""Return a paginated, filtered list of drafts with ratings.
Returns dict with keys: drafts, total, page, per_page, pages.
"""
pairs = db.drafts_with_ratings(limit=1000)
# Filter
filtered = []
for draft, rating in pairs:
if min_score > 0 and rating.composite_score < min_score:
continue
if category and category not in rating.categories:
continue
if search:
haystack = f"{draft.name} {draft.title} {rating.summary}".lower()
if not all(w in haystack for w in search.lower().split()):
continue
filtered.append((draft, rating))
# Sort
sort_keys = {
"score": lambda p: p[1].composite_score,
"name": lambda p: p[0].name,
"date": lambda p: p[0].time or "",
"novelty": lambda p: p[1].novelty,
"maturity": lambda p: p[1].maturity,
"relevance": lambda p: p[1].relevance,
"overlap": lambda p: p[1].overlap,
"momentum": lambda p: p[1].momentum,
}
key_fn = sort_keys.get(sort, sort_keys["score"])
reverse = sort_dir == "desc"
filtered.sort(key=key_fn, reverse=reverse)
total = len(filtered)
pages = max(1, (total + per_page - 1) // per_page)
page = max(1, min(page, pages))
start = (page - 1) * per_page
page_items = filtered[start : start + per_page]
drafts = []
for draft, rating in page_items:
drafts.append({
"name": draft.name,
"title": draft.title,
"date": draft.date,
"url": draft.datatracker_url,
"pages": draft.pages or 0,
"group": draft.group or "individual",
"score": round(rating.composite_score, 2),
"novelty": rating.novelty,
"maturity": rating.maturity,
"overlap": rating.overlap,
"momentum": rating.momentum,
"relevance": rating.relevance,
"categories": rating.categories,
"summary": rating.summary,
})
return {
"drafts": drafts,
"total": total,
"page": page,
"per_page": per_page,
"pages": pages,
}
def get_draft_detail(db: Database, name: str) -> dict | None:
"""Return full detail for a single draft."""
draft = db.get_draft(name)
if not draft:
return None
rating = db.get_rating(name)
authors = db.get_authors_for_draft(name)
ideas = db.get_ideas_for_draft(name)
refs = db.get_refs_for_draft(name)
result = {
"name": draft.name,
"title": draft.title,
"rev": draft.rev,
"abstract": draft.abstract,
"date": draft.date,
"time": draft.time,
"url": draft.datatracker_url,
"text_url": draft.text_url,
"pages": draft.pages,
"words": draft.words,
"group": draft.group or "individual",
"categories": draft.categories,
"tags": draft.tags,
"authors": [
{"name": a.name, "affiliation": a.affiliation, "person_id": a.person_id}
for a in authors
],
"ideas": ideas,
"refs": [{"type": t, "id": rid} for t, rid in refs],
}
if rating:
result["rating"] = {
"score": round(rating.composite_score, 2),
"novelty": rating.novelty,
"maturity": rating.maturity,
"overlap": rating.overlap,
"momentum": rating.momentum,
"relevance": rating.relevance,
"summary": rating.summary,
"novelty_note": rating.novelty_note,
"maturity_note": rating.maturity_note,
"overlap_note": rating.overlap_note,
"momentum_note": rating.momentum_note,
"relevance_note": rating.relevance_note,
"categories": rating.categories,
}
return result
def get_rating_distributions(db: Database) -> dict:
"""Return arrays for each rating dimension, suitable for Plotly."""
pairs = db.drafts_with_ratings(limit=1000)
dims = {
"novelty": [],
"maturity": [],
"overlap": [],
"momentum": [],
"relevance": [],
"scores": [],
"categories": [],
"names": [],
}
for draft, rating in pairs:
dims["novelty"].append(rating.novelty)
dims["maturity"].append(rating.maturity)
dims["overlap"].append(rating.overlap)
dims["momentum"].append(rating.momentum)
dims["relevance"].append(rating.relevance)
dims["scores"].append(round(rating.composite_score, 2))
dims["categories"].append(rating.categories[0] if rating.categories else "Other")
dims["names"].append(draft.name)
return dims
def get_timeline_data(db: Database) -> dict:
"""Return monthly counts by category for timeline chart."""
pairs = db.drafts_with_ratings(limit=1000)
all_drafts = db.list_drafts(limit=1000, order_by="time ASC")
rating_map = {d.name: r for d, r in pairs}
month_cat: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
for d in all_drafts:
month = d.time[:7] if d.time else "unknown"
r = rating_map.get(d.name)
if r:
cat = r.categories[0] if r.categories else "Other"
month_cat[month][cat] += 1
months = sorted(month_cat.keys())
cat_totals: Counter = Counter()
for mc in month_cat.values():
for c, cnt in mc.items():
cat_totals[c] += cnt
top_cats = [c for c, _ in cat_totals.most_common(10)]
series = {}
for cat in top_cats:
series[cat] = [month_cat[m].get(cat, 0) for m in months]
return {"months": months, "series": series, "categories": top_cats}
def get_ideas_by_type(db: Database) -> dict:
"""Return ideas grouped by type with counts."""
all_ideas = db.all_ideas()
type_counts = Counter(i.get("type", "other") or "other" for i in all_ideas)
return {
"total": len(all_ideas),
"by_type": dict(type_counts.most_common()),
"ideas": all_ideas,
}
def get_all_gaps(db: Database) -> list[dict]:
"""Return all gap analysis results."""
return db.all_gaps()
def get_gap_detail(db: Database, gap_id: int) -> dict | None:
"""Return a single gap by ID, or None if not found."""
gaps = db.all_gaps()
for g in gaps:
if g["id"] == gap_id:
return g
return None
def get_generated_drafts() -> list[dict]:
"""Return list of pre-generated draft files in data/reports/generated-drafts/."""
drafts_dir = _project_root / "data" / "reports" / "generated-drafts"
if not drafts_dir.exists():
return []
results = []
for f in sorted(drafts_dir.glob("draft-*.txt")):
# Extract title from first non-empty content line after header
title = f.stem
text = f.read_text(errors="replace")
for line in text.splitlines():
stripped = line.strip()
if stripped and not stripped.startswith("Internet-Draft") and \
not stripped.startswith("Intended status") and \
not stripped.startswith("Expires:") and stripped != "":
title = stripped
break
results.append({
"filename": f.name,
"stem": f.stem,
"title": title,
"size": f.stat().st_size,
"path": str(f),
})
return results
def read_generated_draft(filename: str) -> str | None:
"""Read a generated draft file by filename. Returns text or None."""
drafts_dir = _project_root / "data" / "reports" / "generated-drafts"
path = drafts_dir / filename
if not path.exists() or not path.is_file():
return None
# Safety: ensure we're not reading outside the directory
if not str(path.resolve()).startswith(str(drafts_dir.resolve())):
return None
return path.read_text(errors="replace")
def get_top_authors(db: Database, limit: int = 30) -> list[dict]:
"""Return top authors by draft count."""
rows = db.top_authors(limit=limit)
return [
{"name": name, "affiliation": aff, "draft_count": cnt, "drafts": drafts}
for name, aff, cnt, drafts in rows
]
def get_org_data(db: Database, limit: int = 20) -> list[dict]:
"""Return organization contribution data."""
rows = db.top_orgs(limit=limit)
return [
{"org": org, "author_count": authors, "draft_count": drafts}
for org, authors, drafts in rows
]
def get_category_radar_data(db: Database) -> dict:
"""Return average rating profiles per category for radar chart."""
pairs = db.drafts_with_ratings(limit=1000)
cat_ratings: dict[str, list] = defaultdict(list)
for _, r in pairs:
for c in r.categories:
cat_ratings[c].append(r)
top_cats = sorted(cat_ratings.keys(), key=lambda c: len(cat_ratings[c]), reverse=True)[:8]
result = {}
for cat in top_cats:
ratings = cat_ratings[cat]
n = len(ratings)
result[cat] = {
"count": n,
"novelty": round(sum(r.novelty for r in ratings) / n, 2),
"maturity": round(sum(r.maturity for r in ratings) / n, 2),
"relevance": round(sum(r.relevance for r in ratings) / n, 2),
"momentum": round(sum(r.momentum for r in ratings) / n, 2),
"low_overlap": round(sum(6 - r.overlap for r in ratings) / n, 2),
}
return result
def get_score_histogram(db: Database) -> list[float]:
"""Return list of composite scores for histogram."""
pairs = db.drafts_with_ratings(limit=1000)
return [round(r.composite_score, 2) for _, r in pairs]
def get_coauthor_network(db: Database, min_shared: int = 1) -> dict:
"""Return co-authorship network data for force-directed graph.
Returns {nodes: [{id, name, org, draft_count}], edges: [{source, target, weight}]}
"""
pairs = db.coauthor_pairs()
top = db.top_authors(limit=100)
# Build node set from authors who have co-authorships
author_info = {name: {"org": aff, "draft_count": cnt} for name, aff, cnt, _ in top}
node_set = set()
edges = []
for a, b, shared in pairs:
if shared >= min_shared:
node_set.add(a)
node_set.add(b)
edges.append({"source": a, "target": b, "weight": shared})
nodes = []
for name in node_set:
info = author_info.get(name, {"org": "", "draft_count": 1})
nodes.append({
"id": name,
"name": name,
"org": info["org"],
"draft_count": info["draft_count"],
})
return {"nodes": nodes, "edges": edges}
def get_similarity_graph(db: Database, threshold: float = 0.75) -> dict:
"""Return draft similarity network for force-directed graph.
Returns {nodes: [{name, title, category, score}],
edges: [{source, target, similarity}],
stats: {node_count, edge_count, avg_similarity}}
"""
import numpy as np
embeddings = db.all_embeddings()
if len(embeddings) < 2:
return {"nodes": [], "edges": [], "stats": {"node_count": 0, "edge_count": 0, "avg_similarity": 0}}
pairs = db.drafts_with_ratings(limit=1000)
rating_map = {d.name: r for d, r in pairs}
draft_map = {d.name: d for d, _ in pairs}
# Filter to drafts with both embeddings and ratings
names = [n for n in embeddings if n in rating_map]
if len(names) < 2:
return {"nodes": [], "edges": [], "stats": {"node_count": 0, "edge_count": 0, "avg_similarity": 0}}
matrix = np.array([embeddings[n] for n in names])
# L2-normalize and compute cosine similarity
norms = np.linalg.norm(matrix, axis=1, keepdims=True)
norms[norms == 0] = 1.0
normalized = matrix / norms
sim_matrix = normalized @ normalized.T
# Find pairs above threshold (upper triangle only)
edges = []
node_set = set()
for i in range(len(names)):
for j in range(i + 1, len(names)):
sim = float(sim_matrix[i, j])
if sim >= threshold:
edges.append({"source": names[i], "target": names[j], "similarity": round(sim, 4)})
node_set.add(names[i])
node_set.add(names[j])
# Build nodes from connected drafts only
nodes = []
for name in names:
if name not in node_set:
continue
r = rating_map[name]
d = draft_map.get(name)
nodes.append({
"name": name,
"title": d.title if d else name,
"category": r.categories[0] if r.categories else "Other",
"score": round(r.composite_score, 2),
})
avg_sim = round(sum(e["similarity"] for e in edges) / max(len(edges), 1), 4)
return {
"nodes": nodes,
"edges": edges,
"stats": {"node_count": len(nodes), "edge_count": len(edges), "avg_similarity": avg_sim},
}
def get_cross_org_data(db: Database, limit: int = 20) -> list[dict]:
"""Return cross-org collaboration pairs."""
rows = db.cross_org_collaborations(limit=limit)
return [
{"org_a": a, "org_b": b, "shared_drafts": cnt}
for a, b, cnt in rows
]
def get_author_network_full(db: Database) -> dict:
"""Return enriched co-authorship network with avg scores and cluster info.
Returns {
nodes: [{id, name, org, draft_count, avg_score, drafts: [name,...]}],
edges: [{source, target, weight}],
clusters: [{id, members: [name,...], org_mix: {org: count}, size}],
}
"""
pairs = db.coauthor_pairs()
top = db.top_authors(limit=500)
# Build rating lookup for avg scores
rated = db.drafts_with_ratings(limit=2000)
draft_score = {d.name: r.composite_score for d, r in rated}
# Author info map
author_info = {}
for name, aff, cnt, drafts in top:
scores = [draft_score[dn] for dn in drafts if dn in draft_score]
avg = round(sum(scores) / len(scores), 2) if scores else 0
author_info[name] = {
"org": aff, "draft_count": cnt, "drafts": drafts, "avg_score": avg
}
# Build node set: authors with 2+ drafts OR 1+ co-authorship
node_set = set()
edges = []
for a, b, shared in pairs:
if shared >= 1:
node_set.add(a)
node_set.add(b)
edges.append({"source": a, "target": b, "weight": shared})
# Also include authors with 2+ drafts even if no co-authorships
for name, info in author_info.items():
if info["draft_count"] >= 2:
node_set.add(name)
nodes = []
for name in node_set:
info = author_info.get(name, {"org": "", "draft_count": 1, "drafts": [], "avg_score": 0})
nodes.append({
"id": name,
"name": name,
"org": info["org"],
"draft_count": info["draft_count"],
"avg_score": info["avg_score"],
"drafts": info["drafts"][:8], # cap for JSON size
})
# Cluster detection via connected components (BFS)
adjacency: dict[str, set[str]] = defaultdict(set)
for e in edges:
adjacency[e["source"]].add(e["target"])
adjacency[e["target"]].add(e["source"])
visited: set[str] = set()
clusters = []
for node in sorted(node_set):
if node in visited:
continue
component: list[str] = []
queue = [node]
while queue:
current = queue.pop(0)
if current in visited:
continue
visited.add(current)
component.append(current)
for neighbor in adjacency.get(current, []):
if neighbor not in visited:
queue.append(neighbor)
if len(component) >= 2:
org_mix: dict[str, int] = Counter()
for m in component:
org = author_info.get(m, {}).get("org", "")
if org:
org_mix[org] += 1
clusters.append({
"id": len(clusters),
"members": component,
"org_mix": dict(org_mix.most_common()),
"size": len(component),
})
clusters.sort(key=lambda c: c["size"], reverse=True)
return {"nodes": nodes, "edges": edges, "clusters": clusters}
def get_idea_clusters(db: Database) -> dict:
"""Cluster ideas by embedding similarity, return clusters + t-SNE scatter."""
import numpy as np
embeddings = db.all_idea_embeddings()
if not embeddings:
return {"clusters": [], "scatter": [], "stats": {"total": 0, "clustered": 0, "num_clusters": 0}, "empty": True}
# Fetch ideas with IDs for metadata lookup
rows = db.conn.execute("SELECT id, title, description, idea_type, draft_name FROM ideas").fetchall()
idea_map = {r["id"]: {"title": r["title"], "description": r["description"],
"type": r["idea_type"], "draft_name": r["draft_name"]} for r in rows}
# Build matrix from embeddings that have matching ideas
idea_ids = [iid for iid in embeddings if iid in idea_map]
if len(idea_ids) < 5:
return {"clusters": [], "scatter": [], "stats": {"total": len(idea_ids), "clustered": 0, "num_clusters": 0}, "empty": True}
matrix = np.array([embeddings[iid] for iid in idea_ids])
# Agglomerative clustering with cosine distance
try:
from sklearn.cluster import AgglomerativeClustering
clustering = AgglomerativeClustering(
n_clusters=None, distance_threshold=0.5,
metric='cosine', linkage='average',
)
labels = clustering.fit_predict(matrix)
except Exception:
return {"clusters": [], "scatter": [], "stats": {"total": len(idea_ids), "clustered": 0, "num_clusters": 0}, "empty": True}
# Build cluster data
cluster_ideas: dict[int, list] = defaultdict(list)
for idx, iid in enumerate(idea_ids):
cluster_ideas[labels[idx]].append(iid)
# Filter to clusters with 2+ ideas
stop = {"a", "an", "the", "of", "for", "in", "to", "and", "or", "with", "on", "by", "is", "as", "at", "from", "that", "this", "it"}
clusters = []
for cid in sorted(cluster_ideas.keys()):
members = cluster_ideas[cid]
if len(members) < 2:
continue
ideas_in_cluster = [idea_map[iid] for iid in members if iid in idea_map]
# Theme: most common significant words in titles
words = Counter()
for idea in ideas_in_cluster:
for w in idea["title"].lower().split():
w_clean = w.strip("()[].,;:-\"'")
if len(w_clean) > 2 and w_clean not in stop:
words[w_clean] += 1
top_words = [w for w, _ in words.most_common(4)]
theme = " ".join(top_words).title() if top_words else f"Cluster {cid}"
drafts = list({idea["draft_name"] for idea in ideas_in_cluster})
clusters.append({
"id": len(clusters),
"theme": theme,
"size": len(ideas_in_cluster),
"ideas": ideas_in_cluster[:20],
"drafts": drafts,
})
# t-SNE for scatter
scatter = []
try:
from sklearn.manifold import TSNE
perp = min(30, len(idea_ids) - 1)
tsne = TSNE(n_components=2, perplexity=perp, random_state=42, max_iter=500)
coords = tsne.fit_transform(matrix)
for idx, iid in enumerate(idea_ids):
info = idea_map.get(iid, {})
scatter.append({
"x": round(float(coords[idx, 0]), 3),
"y": round(float(coords[idx, 1]), 3),
"cluster_id": int(labels[idx]),
"title": info.get("title", ""),
"draft_name": info.get("draft_name", ""),
})
except Exception:
pass
total = len(idea_ids)
clustered = sum(c["size"] for c in clusters)
return {
"clusters": clusters,
"scatter": scatter,
"stats": {"total": total, "clustered": clustered, "num_clusters": len(clusters)},
"empty": False,
}
def get_timeline_animation_data(db: Database) -> dict:
"""Compute t-SNE on all drafts, return points with month info + category_monthly.
t-SNE is computed once on ALL drafts so coordinates are stable across
animation frames. Each point carries a ``month`` field (YYYY-MM) so the
front-end can build cumulative animation frames.
"""
import numpy as np
embeddings = db.all_embeddings()
if len(embeddings) < 5:
return {"points": [], "months": [], "category_monthly": {}}
pairs = db.drafts_with_ratings(limit=1000)
rating_map = {d.name: r for d, r in pairs}
draft_map = {d.name: d for d, _ in pairs}
# Filter to drafts that have both embeddings and ratings
names = [n for n in embeddings if n in rating_map]
if len(names) < 5:
return {"points": [], "months": [], "category_monthly": {}}
matrix = np.array([embeddings[n] for n in names])
try:
from sklearn.manifold import TSNE
tsne = TSNE(n_components=2, perplexity=min(30, len(names) - 1),
random_state=42, max_iter=500)
coords = tsne.fit_transform(matrix)
except Exception:
return {"points": [], "months": [], "category_monthly": {}}
# Build points with month
points = []
month_set: set[str] = set()
category_monthly: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
for i, name in enumerate(names):
r = rating_map[name]
d = draft_map.get(name)
month = (d.time[:7] if d and d.time else "unknown")
cat = r.categories[0] if r.categories else "Other"
month_set.add(month)
category_monthly[month][cat] += 1
points.append({
"name": name,
"title": d.title if d else name,
"x": round(float(coords[i, 0]), 3),
"y": round(float(coords[i, 1]), 3),
"category": cat,
"score": round(r.composite_score, 2),
"month": month,
})
months = sorted(month_set)
# Convert defaultdict to plain dict for JSON
cat_monthly_plain = {m: dict(cats) for m, cats in category_monthly.items()}
return {
"points": points,
"months": months,
"category_monthly": cat_monthly_plain,
}
def get_monitor_status(db: Database) -> dict:
"""Return monitoring status data for dashboard."""
runs = db.get_monitor_runs(limit=20)
last = runs[0] if runs else None
unrated = len(db.unrated_drafts(limit=9999))
unembedded = len(db.drafts_without_embeddings(limit=9999))
no_ideas = len(db.drafts_without_ideas(limit=9999))
return {
"last_run": last,
"runs": runs,
"unprocessed": {"unrated": unrated, "unembedded": unembedded, "no_ideas": no_ideas},
"total_runs": len(runs),
}
def get_landscape_tsne(db: Database) -> list[dict]:
"""Compute t-SNE from embeddings, return [{name, title, x, y, category, score}].
Uses cached coordinates if available, otherwise computes fresh.
"""
import numpy as np
embeddings = db.all_embeddings()
if len(embeddings) < 5:
return []
pairs = db.drafts_with_ratings(limit=1000)
rating_map = {d.name: r for d, r in pairs}
draft_map = {d.name: d for d, _ in pairs}
# Filter to drafts that have both embeddings and ratings
names = [n for n in embeddings if n in rating_map]
if len(names) < 5:
return []
matrix = np.array([embeddings[n] for n in names])
try:
from sklearn.manifold import TSNE
tsne = TSNE(n_components=2, perplexity=min(30, len(names) - 1),
random_state=42, max_iter=500)
coords = tsne.fit_transform(matrix)
except Exception:
return []
result = []
for i, name in enumerate(names):
r = rating_map[name]
d = draft_map.get(name)
result.append({
"name": name,
"title": d.title if d else name,
"x": round(float(coords[i, 0]), 3),
"y": round(float(coords[i, 1]), 3),
"category": r.categories[0] if r.categories else "Other",
"score": round(r.composite_score, 2),
})
return result

View File

@@ -0,0 +1,65 @@
{% extends "base.html" %}
{% set active_page = "about" %}
{% block title %}About — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="max-w-3xl">
<h1 class="text-2xl font-bold text-white mb-6">About IETF Draft Analyzer</h1>
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6 mb-6">
<h2 class="text-lg font-semibold text-white mb-3">What is this?</h2>
<p class="text-sm text-slate-400 leading-relaxed mb-4">
A tool for tracking, categorizing, rating, and mapping IETF Internet-Drafts
focused on AI and agent-related topics. It uses Claude for analysis and rating,
Ollama for embeddings, and SQLite for storage.
</p>
<p class="text-sm text-slate-400 leading-relaxed">
The dashboard provides interactive visualizations of the draft landscape,
including category breakdowns, rating distributions, author networks,
extracted ideas, and gap analysis.
</p>
</div>
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6 mb-6">
<h2 class="text-lg font-semibold text-white mb-3">Current Data</h2>
<div class="grid grid-cols-2 gap-4 text-sm">
<div>
<div class="text-slate-500">Total Drafts</div>
<div class="text-xl font-bold text-blue-400">{{ stats.total_drafts }}</div>
</div>
<div>
<div class="text-slate-500">Rated Drafts</div>
<div class="text-xl font-bold text-green-400">{{ stats.rated_count }}</div>
</div>
<div>
<div class="text-slate-500">Authors Tracked</div>
<div class="text-xl font-bold text-purple-400">{{ stats.author_count }}</div>
</div>
<div>
<div class="text-slate-500">Ideas Extracted</div>
<div class="text-xl font-bold text-amber-400">{{ stats.idea_count }}</div>
</div>
<div>
<div class="text-slate-500">Gaps Identified</div>
<div class="text-xl font-bold text-red-400">{{ stats.gap_count }}</div>
</div>
<div>
<div class="text-slate-500">API Tokens Used</div>
<div class="text-xl font-bold text-slate-300">{{ "{:,}".format(stats.input_tokens + stats.output_tokens) }}</div>
</div>
</div>
</div>
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6">
<h2 class="text-lg font-semibold text-white mb-3">Tech Stack</h2>
<ul class="text-sm text-slate-400 space-y-2">
<li><span class="text-slate-200 font-medium">Analysis:</span> Claude (Sonnet for analysis, Haiku for bulk)</li>
<li><span class="text-slate-200 font-medium">Embeddings:</span> Ollama (nomic-embed-text)</li>
<li><span class="text-slate-200 font-medium">Storage:</span> SQLite with FTS5 full-text search</li>
<li><span class="text-slate-200 font-medium">Dashboard:</span> Flask, Tailwind CSS, Plotly.js</li>
<li><span class="text-slate-200 font-medium">Data source:</span> IETF Datatracker API</li>
</ul>
</div>
</div>
{% endblock %}

View File

@@ -0,0 +1,598 @@
{% extends "base.html" %}
{% set active_page = "authors" %}
{% block title %}Author Network — IETF Draft Analyzer{% endblock %}
{% block extra_head %}
<script src="https://d3js.org/d3.v7.min.js"></script>
<style>
#networkSvg {
width: 100%;
height: 600px;
cursor: grab;
}
#networkSvg:active { cursor: grabbing; }
#networkSvg .node { cursor: pointer; }
#networkSvg .node circle { stroke: rgba(255,255,255,0.15); stroke-width: 1.5px; transition: r 0.2s; }
#networkSvg .node:hover circle { stroke: #60a5fa; stroke-width: 2.5px; }
#networkSvg .node text { pointer-events: none; }
#networkSvg .link { stroke-opacity: 0.25; }
#networkSvg .link:hover { stroke-opacity: 0.7; }
.tooltip-card {
position: absolute; pointer-events: none; z-index: 50;
background: #1e293b; border: 1px solid #334155; border-radius: 8px;
padding: 10px 14px; font-size: 12px; color: #e2e8f0;
box-shadow: 0 8px 24px rgba(0,0,0,0.4);
max-width: 280px; opacity: 0; transition: opacity 0.15s;
}
.tooltip-card.visible { opacity: 1; }
.legend-swatch { width: 12px; height: 12px; border-radius: 3px; display: inline-block; }
.cluster-card { transition: all 0.2s; }
.cluster-card:hover { border-color: #3b82f6 !important; }
.filter-btn { transition: all 0.15s; }
.filter-btn:hover { background: rgba(59, 130, 246, 0.2); }
.filter-btn.active { background: rgba(59, 130, 246, 0.3); border-color: #3b82f6; color: #60a5fa; }
</style>
{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Author Network</h1>
<p class="text-slate-400 text-sm mt-1">Interactive collaboration graph of {{ network.nodes | length }} authors across {{ orgs | length }} organizations</p>
</div>
<!-- Summary stats -->
<div class="grid grid-cols-2 md:grid-cols-5 gap-4 mb-6">
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Authors Shown</div>
<div class="text-2xl font-bold text-white mt-1">{{ network.nodes | length }}</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Organizations</div>
<div class="text-2xl font-bold text-white mt-1">{{ orgs | length }}</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Co-Author Links</div>
<div class="text-2xl font-bold text-white mt-1">{{ network.edges | length }}</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-amber-500 to-amber-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Clusters</div>
<div class="text-2xl font-bold text-white mt-1">{{ network.clusters | length }}</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-rose-500 to-rose-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Multi-Draft</div>
<div class="text-2xl font-bold text-white mt-1">{{ authors | selectattr('draft_count', 'gt', 1) | list | length }}</div>
</div>
</div>
<!-- D3 Force-directed Network Graph -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6 relative">
<div class="flex flex-wrap items-center justify-between gap-3 mb-3">
<div>
<h2 class="text-sm font-semibold text-slate-300">Co-Authorship Network</h2>
<p class="text-xs text-slate-500 mt-0.5">Node size = draft count. Color = organization. Edge thickness = shared drafts. Drag nodes to rearrange. Scroll to zoom.</p>
</div>
<div class="flex gap-2 items-center">
<button id="resetZoom" class="text-xs px-3 py-1.5 rounded-lg border border-slate-700 text-slate-400 hover:text-white hover:border-slate-500 transition">Reset View</button>
<select id="highlightOrg" class="text-xs px-3 py-1.5 rounded-lg border border-slate-700 bg-slate-800 text-slate-300 focus:outline-none focus:border-blue-500">
<option value="">All Organizations</option>
</select>
</div>
</div>
<div class="relative">
<svg id="networkSvg"></svg>
<div id="tooltip" class="tooltip-card"></div>
</div>
<!-- Legend -->
<div id="legend" class="flex flex-wrap gap-3 mt-3 pt-3 border-t border-slate-800"></div>
</div>
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
<!-- Organization bar chart -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Organizations by Draft Count</h2>
<p class="text-xs text-slate-500 mb-3">Color intensity = number of authors from that org.</p>
<div id="orgChart" style="height: 500px;"></div>
</div>
<!-- Cross-org collaboration -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Cross-Organization Collaboration</h2>
<p class="text-xs text-slate-500 mb-3">Organizations co-authoring drafts together.</p>
<div id="crossOrgChart" style="height: 500px;"></div>
</div>
</div>
<!-- Collaboration Clusters -->
{% if network.clusters %}
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Collaboration Clusters</h2>
<p class="text-xs text-slate-500 mb-4">Connected groups of authors who co-author drafts. Click a cluster to highlight it in the graph.</p>
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4" id="clusterGrid">
{% for c in network.clusters[:12] %}
<div class="cluster-card bg-slate-800/50 rounded-lg border border-slate-700/50 p-4 cursor-pointer" data-cluster-id="{{ c.id }}" onclick="highlightCluster({{ c.id }})">
<div class="flex items-center justify-between mb-2">
<span class="text-sm font-semibold text-white">Cluster #{{ c.id + 1 }}</span>
<span class="text-xs px-2 py-0.5 rounded-full bg-blue-500/20 text-blue-400">{{ c.size }} authors</span>
</div>
<div class="flex flex-wrap gap-1 mb-2">
{% for org, count in c.org_mix.items() %}
<span class="text-xs px-2 py-0.5 rounded-full bg-slate-700 text-slate-300">{{ org }} ({{ count }})</span>
{% endfor %}
</div>
<div class="text-xs text-slate-500 truncate" title="{{ c.members | join(', ') }}">
{{ c.members[:5] | join(', ') }}{% if c.members | length > 5 %} +{{ c.members | length - 5 }} more{% endif %}
</div>
</div>
{% endfor %}
</div>
</div>
{% endif %}
<!-- Top Authors Table and Org Stats side by side -->
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6 mb-6">
<!-- Top authors table -->
<div class="lg:col-span-2 bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
<div class="p-4 border-b border-slate-800 flex items-center justify-between">
<h2 class="text-sm font-semibold text-slate-300">Top Authors</h2>
<span class="text-xs text-slate-500">Showing top {{ authors | length }}</span>
</div>
<div class="overflow-x-auto max-h-[600px] overflow-y-auto">
<table class="w-full text-sm" id="authorsTable">
<thead class="sticky top-0 z-10">
<tr class="border-b border-slate-800 bg-slate-900">
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="index">#</th>
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="name">Author</th>
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="org">Organization</th>
<th class="px-4 py-2.5 text-right text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="drafts">Drafts</th>
</tr>
</thead>
<tbody class="divide-y divide-slate-800/50" id="authorsBody">
{% for a in authors %}
<tr class="hover:bg-slate-800/50 transition author-row" data-name="{{ a.name }}" data-org="{{ a.affiliation }}" data-count="{{ a.draft_count }}">
<td class="px-4 py-2.5 text-slate-500 text-xs">{{ loop.index }}</td>
<td class="px-4 py-2.5">
<a href="/drafts?q={{ a.name | urlencode }}" class="text-blue-400 hover:text-blue-300 transition">{{ a.name }}</a>
</td>
<td class="px-4 py-2.5 text-slate-500 text-xs truncate max-w-[200px]" title="{{ a.affiliation }}">{{ a.affiliation }}</td>
<td class="px-4 py-2.5 text-right">
<span class="px-2 py-0.5 rounded-full text-xs font-medium
{% if a.draft_count >= 5 %}bg-green-500/20 text-green-400
{% elif a.draft_count >= 3 %}bg-blue-500/20 text-blue-400
{% else %}bg-slate-700/50 text-slate-400{% endif %}">
{{ a.draft_count }}
</span>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
<!-- Organization stats cards -->
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
<div class="p-4 border-b border-slate-800">
<h2 class="text-sm font-semibold text-slate-300">Organization Stats</h2>
</div>
<div class="max-h-[600px] overflow-y-auto divide-y divide-slate-800/50">
{% for o in orgs %}
<div class="px-4 py-3 hover:bg-slate-800/30 transition cursor-pointer org-card" data-org="{{ o.org }}" onclick="filterByOrg('{{ o.org | e }}')">
<div class="flex items-center justify-between mb-1">
<span class="text-sm text-slate-200 font-medium truncate max-w-[180px]" title="{{ o.org }}">{{ o.org }}</span>
<span class="text-xs px-2 py-0.5 rounded-full bg-blue-500/20 text-blue-400">{{ o.draft_count }} drafts</span>
</div>
<div class="flex items-center gap-3 text-xs text-slate-500">
<span>{{ o.author_count }} author{{ 's' if o.author_count != 1 }}</span>
<span class="text-slate-700">|</span>
<span>{{ (o.draft_count / o.author_count) | round(1) }} drafts/author</span>
</div>
<div class="mt-1.5 w-full bg-slate-800 rounded-full h-1.5">
<div class="bg-blue-500/60 h-1.5 rounded-full" style="width: {{ (o.draft_count / orgs[0].draft_count * 100) | round }}%"></div>
</div>
</div>
{% endfor %}
</div>
</div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
// --- Shared Plotly config ---
const PLOTLY_LAYOUT = {
paper_bgcolor: 'transparent',
plot_bgcolor: 'transparent',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 20, r: 20, b: 40, l: 50 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const CFG = { responsive: true, displayModeBar: false };
const PALETTE = [
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
'#fb923c', '#34d399', '#c084fc', '#38bdf8', '#fbbf24',
];
// --- Data from server ---
const network = {{ network | tojson }};
// ===========================================================
// D3.js Force-Directed Co-Authorship Network
// ===========================================================
(function() {
if (network.nodes.length === 0) {
document.getElementById('networkSvg').outerHTML =
'<p class="text-slate-500 text-sm text-center py-20">No co-authorship data available</p>';
return;
}
const svg = d3.select('#networkSvg');
const container = svg.node().parentElement;
const width = container.clientWidth;
const height = 600;
svg.attr('viewBox', [0, 0, width, height]);
// Build org color map (top orgs by frequency)
const orgCounts = {};
network.nodes.forEach(n => {
if (n.org) orgCounts[n.org] = (orgCounts[n.org] || 0) + 1;
});
const orgsSorted = Object.entries(orgCounts).sort((a,b) => b[1] - a[1]);
const orgColor = {};
orgsSorted.forEach(([org], i) => {
orgColor[org] = i < PALETTE.length ? PALETTE[i] : '#475569';
});
// Populate org dropdown
const orgSelect = document.getElementById('highlightOrg');
orgsSorted.slice(0, 30).forEach(([org, cnt]) => {
const opt = document.createElement('option');
opt.value = org;
opt.textContent = `${org} (${cnt})`;
orgSelect.appendChild(opt);
});
// Populate legend
const legendEl = document.getElementById('legend');
orgsSorted.slice(0, 12).forEach(([org]) => {
const item = document.createElement('div');
item.className = 'flex items-center gap-1.5 text-xs text-slate-400';
item.innerHTML = `<span class="legend-swatch" style="background:${orgColor[org]}"></span>${org}`;
legendEl.appendChild(item);
});
// Build cluster lookup
const clusterOf = {};
(network.clusters || []).forEach(c => {
c.members.forEach(m => { clusterOf[m] = c.id; });
});
// Prepare simulation data (deep copy to avoid mutating)
const nodes = network.nodes.map(n => ({...n}));
const links = network.edges.map(e => ({
source: e.source,
target: e.target,
weight: e.weight,
}));
// Size scale
const maxDrafts = d3.max(nodes, n => n.draft_count) || 1;
const rScale = d3.scaleSqrt().domain([1, maxDrafts]).range([4, 22]);
// Force simulation
const simulation = d3.forceSimulation(nodes)
.force('link', d3.forceLink(links)
.id(d => d.id)
.distance(d => 80 / Math.sqrt(d.weight))
.strength(d => 0.3 * d.weight)
)
.force('charge', d3.forceManyBody().strength(-120).distanceMax(300))
.force('center', d3.forceCenter(width / 2, height / 2))
.force('collision', d3.forceCollide().radius(d => rScale(d.draft_count) + 3))
.force('x', d3.forceX(width / 2).strength(0.03))
.force('y', d3.forceY(height / 2).strength(0.03));
// Zoom behavior
const g = svg.append('g');
const zoom = d3.zoom()
.scaleExtent([0.2, 5])
.on('zoom', (event) => g.attr('transform', event.transform));
svg.call(zoom);
document.getElementById('resetZoom').addEventListener('click', () => {
svg.transition().duration(500).call(zoom.transform, d3.zoomIdentity);
});
// Draw edges
const linkGroup = g.append('g').attr('class', 'links');
const link = linkGroup.selectAll('line')
.data(links)
.join('line')
.attr('class', 'link')
.attr('stroke', '#475569')
.attr('stroke-width', d => Math.max(1, d.weight * 1.5));
// Draw nodes
const nodeGroup = g.append('g').attr('class', 'nodes');
const node = nodeGroup.selectAll('g')
.data(nodes)
.join('g')
.attr('class', 'node')
.call(d3.drag()
.on('start', dragStarted)
.on('drag', dragged)
.on('end', dragEnded)
);
node.append('circle')
.attr('r', d => rScale(d.draft_count))
.attr('fill', d => orgColor[d.org] || '#475569')
.attr('opacity', 0.85);
// Labels for nodes with 3+ drafts
node.filter(d => d.draft_count >= 3)
.append('text')
.text(d => {
const parts = d.name.split(' ');
return parts[parts.length - 1];
})
.attr('dy', d => -(rScale(d.draft_count) + 4))
.attr('text-anchor', 'middle')
.attr('fill', '#94a3b8')
.attr('font-size', '9px')
.attr('font-family', 'Inter, system-ui, sans-serif');
// Tooltip
const tooltip = document.getElementById('tooltip');
node.on('mouseover', function(event, d) {
const draftList = (d.drafts || []).slice(0, 5).map(dn => {
const short = dn.replace(/^draft-/, '');
return `<div class="truncate text-slate-400">${short}</div>`;
}).join('');
const moreCount = (d.drafts || []).length > 5 ? `<div class="text-slate-500 mt-1">+${d.drafts.length - 5} more</div>` : '';
tooltip.innerHTML = `
<div class="font-semibold text-white mb-1">${d.name}</div>
<div class="text-slate-400 text-xs mb-2">${d.org || 'Unknown org'}</div>
<div class="flex gap-4 text-xs mb-2">
<span><span class="text-blue-400 font-medium">${d.draft_count}</span> drafts</span>
<span>avg <span class="text-emerald-400 font-medium">${d.avg_score}</span></span>
</div>
<div class="text-xs">${draftList}${moreCount}</div>
`;
tooltip.classList.add('visible');
// Highlight connected
const connected = new Set();
links.forEach(l => {
const sid = typeof l.source === 'object' ? l.source.id : l.source;
const tid = typeof l.target === 'object' ? l.target.id : l.target;
if (sid === d.id) connected.add(tid);
if (tid === d.id) connected.add(sid);
});
connected.add(d.id);
node.select('circle')
.attr('opacity', n => connected.has(n.id) ? 1 : 0.15);
node.selectAll('text')
.attr('opacity', n => connected.has(n.id) ? 1 : 0.15);
link
.attr('stroke-opacity', l => {
const sid = typeof l.source === 'object' ? l.source.id : l.source;
const tid = typeof l.target === 'object' ? l.target.id : l.target;
return (sid === d.id || tid === d.id) ? 0.7 : 0.03;
});
})
.on('mousemove', function(event) {
const rect = container.getBoundingClientRect();
tooltip.style.left = (event.clientX - rect.left + 15) + 'px';
tooltip.style.top = (event.clientY - rect.top - 10) + 'px';
})
.on('mouseout', function() {
tooltip.classList.remove('visible');
node.select('circle').attr('opacity', 0.85);
node.selectAll('text').attr('opacity', 1);
link.attr('stroke-opacity', 0.25);
})
.on('click', function(event, d) {
// Navigate to drafts search for this author
window.open(`/drafts?q=${encodeURIComponent(d.name)}`, '_blank');
});
// Tick handler
simulation.on('tick', () => {
link
.attr('x1', d => d.source.x)
.attr('y1', d => d.source.y)
.attr('x2', d => d.target.x)
.attr('y2', d => d.target.y);
node.attr('transform', d => `translate(${d.x},${d.y})`);
});
// Drag handlers
function dragStarted(event, d) {
if (!event.active) simulation.alphaTarget(0.3).restart();
d.fx = d.x; d.fy = d.y;
}
function dragged(event, d) {
d.fx = event.x; d.fy = event.y;
}
function dragEnded(event, d) {
if (!event.active) simulation.alphaTarget(0);
d.fx = null; d.fy = null;
}
// Org filter dropdown
orgSelect.addEventListener('change', function() {
const org = this.value;
if (!org) {
node.select('circle').attr('opacity', 0.85);
node.selectAll('text').attr('opacity', 1);
link.attr('stroke-opacity', 0.25);
return;
}
const inOrg = new Set(nodes.filter(n => n.org === org).map(n => n.id));
node.select('circle')
.attr('opacity', n => inOrg.has(n.id) ? 1 : 0.08);
node.selectAll('text')
.attr('opacity', n => inOrg.has(n.id) ? 1 : 0.08);
link.attr('stroke-opacity', l => {
const sid = typeof l.source === 'object' ? l.source.id : l.source;
const tid = typeof l.target === 'object' ? l.target.id : l.target;
return (inOrg.has(sid) && inOrg.has(tid)) ? 0.6 : 0.02;
});
});
// Expose cluster highlighting globally
window.highlightCluster = function(clusterId) {
const cluster = (network.clusters || []).find(c => c.id === clusterId);
if (!cluster) return;
const members = new Set(cluster.members);
// Reset org dropdown
orgSelect.value = '';
node.select('circle')
.transition().duration(300)
.attr('opacity', n => members.has(n.id) ? 1 : 0.08);
node.selectAll('text')
.transition().duration(300)
.attr('opacity', n => members.has(n.id) ? 1 : 0.08);
link.transition().duration(300)
.attr('stroke-opacity', l => {
const sid = typeof l.source === 'object' ? l.source.id : l.source;
const tid = typeof l.target === 'object' ? l.target.id : l.target;
return (members.has(sid) && members.has(tid)) ? 0.7 : 0.02;
});
// Highlight cluster card
document.querySelectorAll('.cluster-card').forEach(c => {
c.classList.toggle('border-blue-500', c.dataset.clusterId == clusterId);
});
// Zoom to fit cluster members
const clusterNodes = nodes.filter(n => members.has(n.id));
if (clusterNodes.length > 0) {
const xs = clusterNodes.map(n => n.x);
const ys = clusterNodes.map(n => n.y);
const x0 = Math.min(...xs) - 50, x1 = Math.max(...xs) + 50;
const y0 = Math.min(...ys) - 50, y1 = Math.max(...ys) + 50;
const scale = Math.min(width / (x1 - x0), height / (y1 - y0), 3);
const cx = (x0 + x1) / 2, cy = (y0 + y1) / 2;
svg.transition().duration(500).call(
zoom.transform,
d3.zoomIdentity.translate(width/2, height/2).scale(scale).translate(-cx, -cy)
);
}
};
// Filter by org (called from org stats cards)
window.filterByOrg = function(org) {
orgSelect.value = org;
orgSelect.dispatchEvent(new Event('change'));
};
})();
// ===========================================================
// Organization Bar Chart (Plotly)
// ===========================================================
const orgsData = {{ orgs_data | tojson }};
const orgNames = orgsData.map(o => o.org).reverse();
const orgDrafts = orgsData.map(o => o.draft_count).reverse();
const orgAuthors = orgsData.map(o => o.author_count).reverse();
Plotly.newPlot('orgChart', [{
y: orgNames, x: orgDrafts,
type: 'bar', orientation: 'h',
marker: {
color: orgAuthors,
colorscale: [[0, '#1e3a5f'], [0.5, '#3b82f6'], [1, '#60a5fa']],
showscale: true,
colorbar: {
title: { text: 'Authors', font: { color: '#94a3b8', size: 10 } },
tickfont: { color: '#94a3b8', size: 10 },
thickness: 12, len: 0.5,
},
},
text: orgDrafts.map((d, i) => `${d} drafts, ${orgAuthors[i]} authors`),
textposition: 'none',
hovertemplate: '<b>%{y}</b><br>%{text}<extra></extra>',
}], {
...PLOTLY_LAYOUT,
margin: { t: 10, r: 80, b: 40, l: 180 },
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Draft Count', font: { size: 11 } } },
}, CFG);
// ===========================================================
// Cross-Org Collaboration Chart (Plotly)
// ===========================================================
const crossOrg = {{ cross_org | tojson }};
if (crossOrg.length > 0) {
const coLabels = crossOrg.map(c => `${c.org_a} + ${c.org_b}`).reverse();
const coValues = crossOrg.map(c => c.shared_drafts).reverse();
Plotly.newPlot('crossOrgChart', [{
y: coLabels, x: coValues,
type: 'bar', orientation: 'h',
marker: {
color: coValues.map((v, i) => {
const pct = i / Math.max(coValues.length - 1, 1);
return `hsl(${160 + pct * 60}, 65%, 50%)`;
}),
},
hovertemplate: '<b>%{y}</b><br>%{x} shared draft(s)<extra></extra>',
}], {
...PLOTLY_LAYOUT,
margin: { t: 10, r: 40, b: 40, l: 240 },
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Shared Drafts', font: { size: 11 } }, dtick: 1 },
yaxis: { ...PLOTLY_LAYOUT.yaxis, automargin: true, tickfont: { size: 10 } },
}, CFG);
} else {
document.getElementById('crossOrgChart').innerHTML =
'<p class="text-slate-500 text-sm text-center mt-20">No cross-org data available</p>';
}
// ===========================================================
// Sortable Authors Table
// ===========================================================
(function() {
const table = document.getElementById('authorsTable');
const tbody = document.getElementById('authorsBody');
let sortCol = null, sortAsc = true;
table.querySelectorAll('th[data-sort]').forEach(th => {
th.addEventListener('click', () => {
const col = th.dataset.sort;
if (sortCol === col) { sortAsc = !sortAsc; } else { sortCol = col; sortAsc = true; }
table.querySelectorAll('th[data-sort]').forEach(h =>
h.textContent = h.textContent.replace(/ [▲▼]/, ''));
th.textContent += sortAsc ? ' ▲' : ' ▼';
const rows = Array.from(tbody.querySelectorAll('tr'));
rows.sort((a, b) => {
let va, vb;
if (col === 'name') { va = a.dataset.name.toLowerCase(); vb = b.dataset.name.toLowerCase(); }
else if (col === 'org') { va = a.dataset.org.toLowerCase(); vb = b.dataset.org.toLowerCase(); }
else if (col === 'drafts') { va = parseInt(a.dataset.count); vb = parseInt(b.dataset.count); }
else { va = parseInt(a.cells[0].textContent); vb = parseInt(b.cells[0].textContent); }
if (typeof va === 'number') return sortAsc ? va - vb : vb - va;
return sortAsc ? va.localeCompare(vb) : vb.localeCompare(va);
});
rows.forEach(r => tbody.appendChild(r));
});
});
})();
</script>
{% endblock %}

View File

@@ -0,0 +1,165 @@
<!DOCTYPE html>
<html lang="en" class="dark">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% block title %}IETF Draft Analyzer{% endblock %}</title>
<script src="https://cdn.tailwindcss.com"></script>
<script src="https://cdn.plot.ly/plotly-2.35.0.min.js"></script>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<script>
tailwind.config = {
darkMode: 'class',
theme: {
extend: {
fontFamily: {
sans: ['Inter', 'system-ui', 'sans-serif'],
mono: ['JetBrains Mono', 'monospace'],
},
}
}
}
</script>
<style>
body { font-family: 'Inter', system-ui, sans-serif; }
.sidebar-link {
transition: all 0.15s ease;
}
.sidebar-link:hover, .sidebar-link.active {
background: rgba(59, 130, 246, 0.15);
color: #60a5fa;
}
.sidebar-link.active {
border-right: 3px solid #3b82f6;
}
.stat-card {
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
backdrop-filter: blur(10px);
}
.score-badge {
display: inline-block;
padding: 2px 8px;
border-radius: 9999px;
font-weight: 600;
font-size: 0.8rem;
}
.score-high { background: rgba(34, 197, 94, 0.2); color: #4ade80; }
.score-mid { background: rgba(234, 179, 8, 0.2); color: #facc15; }
.score-low { background: rgba(239, 68, 68, 0.2); color: #f87171; }
.dim-bar {
display: inline-block;
height: 8px;
border-radius: 4px;
background: #3b82f6;
vertical-align: middle;
}
/* Plotly dark overrides */
.js-plotly-plot .plotly .modebar { right: 8px !important; }
/* Scrollbar */
::-webkit-scrollbar { width: 6px; height: 6px; }
::-webkit-scrollbar-track { background: #1e293b; }
::-webkit-scrollbar-thumb { background: #475569; border-radius: 3px; }
::-webkit-scrollbar-thumb:hover { background: #64748b; }
/* Mobile sidebar */
@media (max-width: 768px) {
.sidebar { transform: translateX(-100%); }
.sidebar.open { transform: translateX(0); }
}
</style>
{% block extra_head %}{% endblock %}
</head>
<body class="bg-slate-950 text-slate-100 min-h-screen">
<!-- Mobile menu button -->
<button id="menuBtn" class="md:hidden fixed top-4 left-4 z-50 p-2 bg-slate-800 rounded-lg border border-slate-700">
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h16"/>
</svg>
</button>
<!-- Sidebar -->
<aside id="sidebar" class="sidebar fixed top-0 left-0 h-full w-60 bg-slate-900 border-r border-slate-800 z-40 flex flex-col transition-transform md:translate-x-0">
<div class="p-5 border-b border-slate-800">
<h1 class="text-lg font-bold text-white tracking-tight">IETF Draft Analyzer</h1>
<p class="text-xs text-slate-500 mt-1">AI/Agent Standards Tracker</p>
</div>
<nav class="flex-1 py-4 overflow-y-auto">
<a href="/" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'overview' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2V6zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2V6zM4 16a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2v-2zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2v-2z"/></svg>
Overview
</a>
<a href="/drafts" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'drafts' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
Draft Explorer
</a>
<a href="/ratings" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'ratings' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M11.049 2.927c.3-.921 1.603-.921 1.902 0l1.519 4.674a1 1 0 00.95.69h4.915c.969 0 1.371 1.24.588 1.81l-3.976 2.888a1 1 0 00-.363 1.118l1.518 4.674c.3.922-.755 1.688-1.538 1.118l-3.976-2.888a1 1 0 00-1.176 0l-3.976 2.888c-.783.57-1.838-.197-1.538-1.118l1.518-4.674a1 1 0 00-.363-1.118l-3.976-2.888c-.784-.57-.38-1.81.588-1.81h4.914a1 1 0 00.951-.69l1.519-4.674z"/></svg>
Ratings
</a>
<a href="/ideas" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'ideas' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9.663 17h4.673M12 3v1m6.364 1.636l-.707.707M21 12h-1M4 12H3m3.343-5.657l-.707-.707m2.828 9.9a5 5 0 117.072 0l-.548.547A3.374 3.374 0 0014 18.469V19a2 2 0 11-4 0v-.531c0-.895-.356-1.754-.988-2.386l-.548-.547z"/></svg>
Ideas
</a>
<a href="/idea-clusters" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'idea_clusters' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2V6zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2V6zM4 16a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2v-2z"/><circle cx="19" cy="19" r="3" stroke="currentColor" stroke-width="2" fill="none"/></svg>
Idea Clusters
</a>
<a href="/gaps" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'gaps' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-2.5L13.732 4.5c-.77-.833-2.694-.833-3.464 0L3.34 16.5c-.77.833.192 2.5 1.732 2.5z"/></svg>
Gap Explorer
</a>
<a href="/timeline" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'timeline' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 8v4l3 3m6-3a9 9 0 11-18 0 9 9 0 0118 0z"/></svg>
Timeline
</a>
<a href="/landscape" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'landscape' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 20l-5.447-2.724A1 1 0 013 16.382V5.618a1 1 0 011.447-.894L9 7m0 13l6-3m-6 3V7m6 10l4.553 2.276A1 1 0 0021 18.382V7.618a1 1 0 00-.553-.894L15 4m0 13V4m0 0L9 7"/></svg>
Landscape
</a>
<a href="/similarity" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'similarity' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/><circle cx="5" cy="6" r="1.5" fill="currentColor" stroke="none"/><circle cx="19" cy="18" r="1.5" fill="currentColor" stroke="none"/><circle cx="18" cy="6" r="1.5" fill="currentColor" stroke="none"/></svg>
Similarity
</a>
<a href="/authors" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'authors' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M17 20h5v-2a3 3 0 00-5.356-1.857M17 20H7m10 0v-2c0-.656-.126-1.283-.356-1.857M7 20H2v-2a3 3 0 015.356-1.857M7 20v-2c0-.656.126-1.283.356-1.857m0 0a5.002 5.002 0 019.288 0M15 7a3 3 0 11-6 0 3 3 0 016 0zm6 3a2 2 0 11-4 0 2 2 0 014 0zM7 10a2 2 0 11-4 0 2 2 0 014 0z"/></svg>
Authors
</a>
<a href="/monitor" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'monitor' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M5.636 18.364a9 9 0 010-12.728m12.728 0a9 9 0 010 12.728m-9.9-2.829a5 5 0 010-7.07m7.072 0a5 5 0 010 7.07M13 12a1 1 0 11-2 0 1 1 0 012 0z"/></svg>
Monitor
</a>
<div class="border-t border-slate-800 mt-4 pt-4">
<a href="/about" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'about' }}">
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z"/></svg>
About
</a>
</div>
</nav>
<div class="p-4 border-t border-slate-800 text-xs text-slate-600">
IETF Draft Analyzer v0.3
</div>
</aside>
<!-- Main content -->
<main class="md:ml-60 min-h-screen">
<div class="p-6 md:p-8 max-w-7xl mx-auto">
{% block content %}{% endblock %}
</div>
</main>
<script>
// Mobile sidebar toggle
const menuBtn = document.getElementById('menuBtn');
const sidebar = document.getElementById('sidebar');
menuBtn?.addEventListener('click', () => sidebar.classList.toggle('open'));
// Close on click outside
document.addEventListener('click', (e) => {
if (window.innerWidth < 768 && sidebar.classList.contains('open') &&
!sidebar.contains(e.target) && !menuBtn.contains(e.target)) {
sidebar.classList.remove('open');
}
});
</script>
{% block extra_scripts %}{% endblock %}
</body>
</html>

View File

@@ -0,0 +1,298 @@
{% extends "base.html" %}
{% set active_page = "drafts" %}
{% block title %}{{ draft.name }} — IETF Draft Analyzer{% endblock %}
{% block extra_head %}
<style>
.detail-card {
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
backdrop-filter: blur(10px);
}
.score-ring {
width: 100px;
height: 100px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
margin: 0 auto;
position: relative;
}
.score-ring::before {
content: '';
position: absolute;
inset: 0;
border-radius: 50%;
padding: 4px;
mask: linear-gradient(#fff 0 0) content-box, linear-gradient(#fff 0 0);
mask-composite: exclude;
-webkit-mask: linear-gradient(#fff 0 0) content-box, linear-gradient(#fff 0 0);
-webkit-mask-composite: xor;
}
.score-ring-high::before { background: linear-gradient(135deg, #22c55e, #4ade80); }
.score-ring-mid::before { background: linear-gradient(135deg, #eab308, #facc15); }
.score-ring-low::before { background: linear-gradient(135deg, #ef4444, #f87171); }
.dim-progress {
height: 8px;
border-radius: 4px;
background: rgba(51, 65, 85, 0.5);
overflow: hidden;
}
.dim-progress-fill {
height: 100%;
border-radius: 4px;
transition: width 0.6s ease;
}
.dim-high { background: linear-gradient(90deg, #22c55e, #4ade80); }
.dim-mid { background: linear-gradient(90deg, #eab308, #facc15); }
.dim-low { background: linear-gradient(90deg, #ef4444, #f87171); }
.idea-type-protocol { background: rgba(59, 130, 246, 0.15); color: #60a5fa; border-color: rgba(59, 130, 246, 0.3); }
.idea-type-mechanism { background: rgba(168, 85, 247, 0.15); color: #c084fc; border-color: rgba(168, 85, 247, 0.3); }
.idea-type-framework { background: rgba(34, 197, 94, 0.15); color: #4ade80; border-color: rgba(34, 197, 94, 0.3); }
.idea-type-architecture { background: rgba(234, 179, 8, 0.15); color: #facc15; border-color: rgba(234, 179, 8, 0.3); }
.idea-type-default { background: rgba(100, 116, 139, 0.15); color: #94a3b8; border-color: rgba(100, 116, 139, 0.3); }
.ref-rfc { background: rgba(34, 197, 94, 0.15); color: #4ade80; }
.ref-draft { background: rgba(59, 130, 246, 0.15); color: #60a5fa; }
.ref-other { background: rgba(234, 179, 8, 0.15); color: #facc15; }
</style>
{% endblock %}
{% block content %}
<!-- Breadcrumb + Header -->
<div class="mb-6">
<a href="/drafts" class="inline-flex items-center gap-1.5 text-sm text-slate-500 hover:text-slate-300 transition group">
<svg class="w-4 h-4 group-hover:-translate-x-0.5 transition-transform" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7"/>
</svg>
Back to Explorer
</a>
<h1 class="text-xl font-bold text-white mt-3 leading-snug">{{ draft.title }}</h1>
<div class="flex flex-wrap items-center gap-3 mt-2">
<span class="text-sm text-slate-400 font-mono">{{ draft.name }}</span>
{% if draft.rev %}
<span class="text-xs px-2 py-0.5 rounded bg-slate-800 text-slate-500 border border-slate-700">rev {{ draft.rev }}</span>
{% endif %}
<span class="text-xs text-slate-600">{{ draft.date }}</span>
{% if draft.rating %}
<span class="score-badge {% if draft.rating.score >= 3.5 %}score-high{% elif draft.rating.score >= 2.5 %}score-mid{% else %}score-low{% endif %}">
{{ draft.rating.score }}
</span>
{% endif %}
</div>
</div>
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6">
<!-- Left column: Main content -->
<div class="lg:col-span-2 space-y-6">
<!-- Abstract -->
<div class="detail-card rounded-xl border border-slate-800 p-6">
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h7"/></svg>
Abstract
</h2>
<p class="text-sm text-slate-400 leading-relaxed">{{ draft.abstract or "No abstract available." }}</p>
</div>
<!-- Rating Analysis -->
{% if draft.rating %}
<div class="detail-card rounded-xl border border-slate-800 p-6">
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 19v-6a2 2 0 00-2-2H5a2 2 0 00-2 2v6a2 2 0 002 2h2a2 2 0 002-2zm0 0V9a2 2 0 012-2h2a2 2 0 012 2v10m-6 0a2 2 0 002 2h2a2 2 0 002-2m0 0V5a2 2 0 012-2h2a2 2 0 012 2v14a2 2 0 01-2 2h-2a2 2 0 01-2-2z"/></svg>
AI Rating Analysis
</h2>
{% if draft.rating.summary %}
<p class="text-sm text-slate-400 mb-5 leading-relaxed">{{ draft.rating.summary }}</p>
{% endif %}
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
{% for dim, label, icon in [
("novelty", "Novelty", "M13 10V3L4 14h7v7l9-11h-7z"),
("maturity", "Maturity", "M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z"),
("overlap", "Overlap", "M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z"),
("momentum", "Momentum", "M13 7h8m0 0v8m0-8l-8 8-4-4-6 6"),
("relevance", "Relevance", "M5 3v4M3 5h4M6 17v4m-2-2h4m5-16l2.286 6.857L21 12l-5.714 2.143L13 21l-2.286-6.857L5 12l5.714-2.143L13 3z")
] %}
{% set val = draft.rating[dim] %}
<div class="bg-slate-800/30 rounded-lg p-4 border border-slate-800/50">
<div class="flex items-center justify-between mb-2">
<div class="flex items-center gap-2">
<svg class="w-3.5 h-3.5 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="{{ icon }}"/></svg>
<span class="text-xs font-semibold text-slate-300 uppercase tracking-wide">{{ label }}</span>
</div>
<span class="text-lg font-bold {% if val >= 4 %}text-green-400{% elif val >= 3 %}text-amber-400{% else %}text-red-400{% endif %}">{{ val }}<span class="text-xs text-slate-600 font-normal">/5</span></span>
</div>
<div class="dim-progress mb-2">
<div class="dim-progress-fill {% if val >= 4 %}dim-high{% elif val >= 3 %}dim-mid{% else %}dim-low{% endif %}" style="width: {{ val * 20 }}%"></div>
</div>
{% if draft.rating[dim + '_note'] %}
<p class="text-xs text-slate-500 leading-relaxed">{{ draft.rating[dim + '_note'] }}</p>
{% endif %}
</div>
{% endfor %}
</div>
</div>
{% endif %}
<!-- Ideas -->
{% if draft.ideas %}
<div class="detail-card rounded-xl border border-slate-800 p-6">
<h2 class="text-sm font-semibold text-slate-300 mb-4 flex items-center gap-2">
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9.663 17h4.673M12 3v1m6.364 1.636l-.707.707M21 12h-1M4 12H3m3.343-5.657l-.707-.707m2.828 9.9a5 5 0 117.072 0l-.548.547A3.374 3.374 0 0014 18.469V19a2 2 0 11-4 0v-.531c0-.895-.356-1.754-.988-2.386l-.548-.547z"/></svg>
Extracted Ideas <span class="text-slate-600 font-normal">({{ draft.ideas|length }})</span>
</h2>
<div class="space-y-3">
{% for idea in draft.ideas %}
<div class="bg-slate-800/30 rounded-lg p-4 border border-slate-800/50">
<div class="flex items-start gap-2 mb-1">
<span class="text-sm font-medium text-slate-200 leading-snug">{{ idea.title }}</span>
{% if idea.type %}
{% set type_lower = idea.type|lower %}
<span class="flex-shrink-0 px-2 py-0.5 rounded-full text-[10px] font-medium border
{% if type_lower == 'protocol' %}idea-type-protocol
{% elif type_lower == 'mechanism' %}idea-type-mechanism
{% elif type_lower == 'framework' %}idea-type-framework
{% elif type_lower == 'architecture' %}idea-type-architecture
{% else %}idea-type-default{% endif %}">
{{ idea.type }}
</span>
{% endif %}
</div>
{% if idea.description %}
<p class="text-xs text-slate-500 leading-relaxed mt-1">{{ idea.description }}</p>
{% endif %}
</div>
{% endfor %}
</div>
</div>
{% endif %}
</div>
<!-- Right column: Sidebar -->
<div class="space-y-6">
<!-- Score Card -->
{% if draft.rating %}
<div class="detail-card rounded-xl border border-slate-800 p-6 text-center">
<div class="score-ring {% if draft.rating.score >= 3.5 %}score-ring-high{% elif draft.rating.score >= 2.5 %}score-ring-mid{% else %}score-ring-low{% endif %}">
<div>
<div class="text-3xl font-bold {% if draft.rating.score >= 3.5 %}text-green-400{% elif draft.rating.score >= 2.5 %}text-amber-400{% else %}text-red-400{% endif %}">
{{ draft.rating.score }}
</div>
<div class="text-[10px] text-slate-500 uppercase tracking-wider">Score</div>
</div>
</div>
<!-- Mini dimension summary -->
<div class="mt-4 grid grid-cols-5 gap-1 text-center">
{% for dim, abbr in [("novelty","N"), ("maturity","M"), ("overlap","O"), ("momentum","Mo"), ("relevance","R")] %}
{% set v = draft.rating[dim] %}
<div>
<div class="text-xs font-bold {% if v >= 4 %}text-green-400{% elif v >= 3 %}text-amber-400{% else %}text-red-400{% endif %}">{{ v }}</div>
<div class="text-[9px] text-slate-600 uppercase">{{ abbr }}</div>
</div>
{% endfor %}
</div>
</div>
{% endif %}
<!-- Metadata -->
<div class="detail-card rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z"/></svg>
Metadata
</h2>
<dl class="space-y-2.5 text-sm">
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Date</dt><dd class="text-slate-300">{{ draft.date }}</dd></div>
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Revision</dt><dd class="text-slate-300">{{ draft.rev or 'N/A' }}</dd></div>
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Pages</dt><dd class="text-slate-300">{{ draft.pages or 'N/A' }}</dd></div>
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Words</dt><dd class="text-slate-300">{{ '{:,}'.format(draft.words) if draft.words else 'N/A' }}</dd></div>
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Working Group</dt><dd class="text-slate-300">{{ draft.group }}</dd></div>
</dl>
<div class="mt-4 space-y-2">
<a href="{{ draft.url }}" target="_blank" rel="noopener"
class="flex items-center justify-center gap-2 px-3 py-2 bg-blue-600 text-white rounded-lg text-xs font-medium hover:bg-blue-500 transition">
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10 6H6a2 2 0 00-2 2v10a2 2 0 002 2h10a2 2 0 002-2v-4M14 4h6m0 0v6m0-6L10 14"/></svg>
View on Datatracker
</a>
{% if draft.text_url %}
<a href="{{ draft.text_url }}" target="_blank" rel="noopener"
class="flex items-center justify-center gap-2 px-3 py-2 border border-slate-700 text-slate-300 rounded-lg text-xs font-medium hover:border-slate-500 hover:text-white transition">
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 10v6m0 0l-3-3m3 3l3-3m2 8H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
Read Full Text
</a>
{% endif %}
</div>
</div>
<!-- Authors -->
{% if draft.authors %}
<div class="detail-card rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M17 20h5v-2a3 3 0 00-5.356-1.857M17 20H7m10 0v-2c0-.656-.126-1.283-.356-1.857M7 20H2v-2a3 3 0 015.356-1.857M7 20v-2c0-.656.126-1.283.356-1.857m0 0a5.002 5.002 0 019.288 0M15 7a3 3 0 11-6 0 3 3 0 016 0z"/></svg>
Authors <span class="text-slate-600 font-normal">({{ draft.authors|length }})</span>
</h2>
<ul class="space-y-2.5">
{% for a in draft.authors %}
<li class="flex items-start gap-2">
<div class="w-7 h-7 rounded-full bg-slate-800 border border-slate-700 flex items-center justify-center flex-shrink-0 mt-0.5">
<span class="text-xs font-semibold text-slate-400">{{ a.name[0]|upper if a.name else '?' }}</span>
</div>
<div>
<a href="/drafts?q={{ a.name | urlencode }}" class="text-sm text-blue-400 hover:text-blue-300 transition">{{ a.name }}</a>
{% if a.affiliation %}
<div class="text-xs text-slate-500">{{ a.affiliation }}</div>
{% endif %}
</div>
</li>
{% endfor %}
</ul>
</div>
{% endif %}
<!-- Categories -->
{% if draft.rating and draft.rating.categories %}
<div class="detail-card rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M7 7h.01M7 3h5c.512 0 1.024.195 1.414.586l7 7a2 2 0 010 2.828l-7 7a2 2 0 01-2.828 0l-7-7A1.994 1.994 0 013 12V7a4 4 0 014-4z"/></svg>
Categories
</h2>
<div class="flex flex-wrap gap-1.5">
{% for cat in draft.rating.categories %}
<a href="/drafts?cat={{ cat }}"
class="px-2.5 py-1 rounded-full text-xs bg-slate-800/60 text-slate-400 border border-slate-700 hover:border-blue-500 hover:text-blue-400 transition">
{{ cat }}
</a>
{% endfor %}
</div>
</div>
{% endif %}
<!-- References -->
{% if draft.refs %}
<div class="detail-card rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13.828 10.172a4 4 0 00-5.656 0l-4 4a4 4 0 105.656 5.656l1.102-1.101m-.758-4.899a4 4 0 005.656 0l4-4a4 4 0 00-5.656-5.656l-1.1 1.1"/></svg>
References <span class="text-slate-600 font-normal">({{ draft.refs|length }})</span>
</h2>
<div class="flex flex-wrap gap-1.5 max-h-48 overflow-y-auto">
{% for ref in draft.refs %}
{% if ref.type == 'rfc' %}
<a href="https://www.rfc-editor.org/rfc/{{ ref.id }}" target="_blank" rel="noopener"
class="px-2 py-0.5 rounded text-[10px] font-medium ref-rfc hover:opacity-80 transition">
RFC {{ ref.id.replace('rfc', '') }}
</a>
{% elif ref.type == 'draft' %}
<a href="/drafts/{{ ref.id }}"
class="px-2 py-0.5 rounded text-[10px] font-medium ref-draft hover:opacity-80 transition">
{{ ref.id }}
</a>
{% else %}
<span class="px-2 py-0.5 rounded text-[10px] font-medium ref-other">
{{ ref.type|upper }} {{ ref.id }}
</span>
{% endif %}
{% endfor %}
</div>
</div>
{% endif %}
</div>
</div>
{% endblock %}

View File

@@ -0,0 +1,369 @@
{% extends "base.html" %}
{% set active_page = "drafts" %}
{% block title %}Draft Explorer — IETF Draft Analyzer{% endblock %}
{% block extra_head %}
<style>
.filter-bar {
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
backdrop-filter: blur(10px);
}
.draft-row {
transition: all 0.15s ease;
}
.draft-row:hover {
background: rgba(59, 130, 246, 0.05);
}
.dim-bar-bg {
display: inline-block;
width: 40px;
height: 6px;
border-radius: 3px;
background: rgba(51, 65, 85, 0.6);
vertical-align: middle;
position: relative;
overflow: hidden;
}
.dim-bar-fill {
display: block;
height: 100%;
border-radius: 3px;
}
.dim-fill-high { background: #4ade80; }
.dim-fill-mid { background: #facc15; }
.dim-fill-low { background: #f87171; }
.cat-pill {
display: inline-block;
padding: 1px 8px;
border-radius: 9999px;
font-size: 0.65rem;
font-weight: 500;
background: rgba(51, 65, 85, 0.5);
color: #94a3b8;
border: 1px solid rgba(71, 85, 105, 0.4);
white-space: nowrap;
}
.cat-pill-active {
background: rgba(59, 130, 246, 0.2);
color: #60a5fa;
border-color: rgba(59, 130, 246, 0.4);
}
.range-slider {
-webkit-appearance: none;
appearance: none;
height: 4px;
border-radius: 2px;
background: #334155;
outline: none;
}
.range-slider::-webkit-slider-thumb {
-webkit-appearance: none;
appearance: none;
width: 16px;
height: 16px;
border-radius: 50%;
background: #3b82f6;
cursor: pointer;
border: 2px solid #1e293b;
}
.range-slider::-moz-range-thumb {
width: 16px;
height: 16px;
border-radius: 50%;
background: #3b82f6;
cursor: pointer;
border: 2px solid #1e293b;
}
.page-btn {
padding: 6px 12px;
border-radius: 8px;
font-size: 0.8rem;
font-weight: 500;
transition: all 0.15s ease;
}
.page-btn-active {
background: #3b82f6;
color: white;
}
.page-btn-inactive {
background: rgba(30, 41, 59, 0.6);
border: 1px solid #334155;
color: #94a3b8;
}
.page-btn-inactive:hover {
border-color: #475569;
color: #e2e8f0;
}
</style>
{% endblock %}
{% block content %}
<!-- Header -->
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Draft Explorer</h1>
<p class="text-slate-400 text-sm mt-1">Browse, search, and filter {{ result.total }} rated Internet-Drafts on AI and agent topics.</p>
</div>
<!-- Filter Bar -->
<div class="filter-bar rounded-xl border border-slate-800 p-5 mb-6">
<form method="get" action="/drafts" id="filterForm">
<!-- Row 1: Search + Sort + Submit -->
<div class="flex flex-wrap gap-3 items-end">
<!-- Search -->
<div class="flex-1 min-w-[200px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Search</label>
<input type="text" name="q" value="{{ search }}" placeholder="Search by name, title, or summary..."
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-4 py-2 text-sm text-slate-200 placeholder-slate-500 focus:outline-none focus:border-blue-500 focus:ring-1 focus:ring-blue-500/30 transition">
</div>
<!-- Category dropdown -->
<div class="min-w-[180px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Category</label>
<select name="cat"
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
<option value="">All categories</option>
{% for cat, count in categories.items() %}
<option value="{{ cat }}" {% if current_cat == cat %}selected{% endif %}>{{ cat }} ({{ count }})</option>
{% endfor %}
</select>
</div>
<!-- Sort -->
<div class="min-w-[150px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Sort by</label>
<select name="sort"
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
<option value="score" {% if sort == 'score' %}selected{% endif %}>Score</option>
<option value="date" {% if sort == 'date' %}selected{% endif %}>Date</option>
<option value="novelty" {% if sort == 'novelty' %}selected{% endif %}>Novelty</option>
<option value="maturity" {% if sort == 'maturity' %}selected{% endif %}>Maturity</option>
<option value="relevance" {% if sort == 'relevance' %}selected{% endif %}>Relevance</option>
<option value="momentum" {% if sort == 'momentum' %}selected{% endif %}>Momentum</option>
<option value="overlap" {% if sort == 'overlap' %}selected{% endif %}>Overlap</option>
<option value="name" {% if sort == 'name' %}selected{% endif %}>Name</option>
</select>
</div>
<!-- Sort direction -->
<div class="min-w-[110px]">
<label class="block text-xs font-medium text-slate-500 mb-1.5">Direction</label>
<select name="dir"
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
<option value="desc" {% if sort_dir == 'desc' %}selected{% endif %}>Descending</option>
<option value="asc" {% if sort_dir == 'asc' %}selected{% endif %}>Ascending</option>
</select>
</div>
</div>
<!-- Row 2: Min Score slider -->
<div class="mt-4 flex flex-wrap items-center gap-4">
<div class="flex items-center gap-3">
<label class="text-xs font-medium text-slate-500 whitespace-nowrap">Min Score:</label>
<input type="range" name="min_score" id="scoreSlider" value="{{ min_score }}" step="0.5" min="0" max="5"
class="range-slider w-40" oninput="document.getElementById('scoreVal').textContent = this.value">
<span id="scoreVal" class="text-sm font-mono font-semibold text-blue-400 w-8">{{ min_score }}</span>
</div>
<div class="flex gap-2 ml-auto">
<button type="submit" class="px-5 py-2 bg-blue-600 text-white rounded-lg text-sm font-medium hover:bg-blue-500 transition-colors">
Apply Filters
</button>
<a href="/drafts" class="px-4 py-2 border border-slate-700 text-slate-400 rounded-lg text-sm hover:border-slate-500 hover:text-slate-300 transition-colors">
Reset
</a>
</div>
</div>
<!-- Row 3: Category pills (quick filter) -->
{% if categories %}
<div class="mt-4 pt-3 border-t border-slate-800/50">
<div class="flex flex-wrap gap-1.5">
<a href="/drafts?q={{ search }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="cat-pill {% if not current_cat %}cat-pill-active{% endif %}">All</a>
{% for cat, count in categories.items() %}
<a href="/drafts?cat={{ cat }}&q={{ search }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="cat-pill {% if current_cat == cat %}cat-pill-active{% endif %}">
{{ cat }} <span class="opacity-50">{{ count }}</span>
</a>
{% endfor %}
</div>
</div>
{% endif %}
</form>
</div>
<!-- Results count -->
<div class="flex items-center justify-between mb-4">
<p class="text-sm text-slate-500">
Showing <span class="text-slate-300 font-medium">{{ result.drafts|length }}</span> of
<span class="text-slate-300 font-medium">{{ result.total }}</span> drafts
{% if search %} matching "<span class="text-blue-400">{{ search }}</span>"{% endif %}
{% if current_cat %} in <span class="text-blue-400">{{ current_cat }}</span>{% endif %}
{% if min_score > 0 %} with score >= <span class="text-blue-400">{{ min_score }}</span>{% endif %}
</p>
{% if result.pages > 1 %}
<p class="text-xs text-slate-600">Page {{ result.page }} of {{ result.pages }}</p>
{% endif %}
</div>
<!-- Draft Table -->
<div class="bg-slate-900/60 rounded-xl border border-slate-800 overflow-hidden">
<div class="overflow-x-auto">
<table class="w-full text-sm">
<thead>
<tr class="border-b border-slate-800 bg-slate-900/80">
{% macro sort_header(field, label, extra_class="", title="") %}
{% set is_active = sort == field %}
{% set next_dir = 'asc' if (is_active and sort_dir == 'desc') else 'desc' %}
<th class="px-4 py-3 text-left text-xs font-medium uppercase tracking-wide {{ extra_class }} {{ 'text-blue-400' if is_active else 'text-slate-500' }}">
<a href="/drafts?q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ field }}&dir={{ next_dir }}"
class="hover:text-blue-400 transition inline-flex items-center gap-1"
{% if title %}title="{{ title }}"{% endif %}>
{{ label }}
{% if is_active %}
<svg class="w-3 h-3 {{ 'rotate-180' if sort_dir == 'asc' else '' }}" fill="currentColor" viewBox="0 0 20 20">
<path fill-rule="evenodd" d="M5.293 7.293a1 1 0 011.414 0L10 10.586l3.293-3.293a1 1 0 111.414 1.414l-4 4a1 1 0 01-1.414 0l-4-4a1 1 0 010-1.414z" clip-rule="evenodd"/>
</svg>
{% endif %}
</a>
</th>
{% endmacro %}
{{ sort_header("score", "Score", "w-20") }}
{{ sort_header("name", "Draft") }}
{{ sort_header("date", "Date", "w-24 hidden md:table-cell") }}
{{ sort_header("novelty", "Nov", "w-20 hidden lg:table-cell", "Novelty") }}
{{ sort_header("maturity", "Mat", "w-20 hidden lg:table-cell", "Maturity") }}
{{ sort_header("relevance", "Rel", "w-20 hidden lg:table-cell", "Relevance") }}
{{ sort_header("momentum", "Mom", "w-20 hidden xl:table-cell", "Momentum") }}
{{ sort_header("overlap", "Ovl", "w-20 hidden xl:table-cell", "Overlap") }}
<th class="px-4 py-3 text-left text-xs font-medium text-slate-500 uppercase tracking-wide hidden md:table-cell">Categories</th>
</tr>
</thead>
<tbody class="divide-y divide-slate-800/30">
{% for d in result.drafts %}
<tr class="draft-row">
<!-- Score badge -->
<td class="px-4 py-3">
<span class="score-badge {% if d.score >= 3.5 %}score-high{% elif d.score >= 2.5 %}score-mid{% else %}score-low{% endif %}">
{{ d.score }}
</span>
</td>
<!-- Draft name + title -->
<td class="px-4 py-3">
<a href="/drafts/{{ d.name }}" class="text-blue-400 hover:text-blue-300 font-medium text-sm transition">
{{ d.title }}
</a>
<div class="text-xs text-slate-600 mt-0.5 font-mono">{{ d.name }}</div>
{% if d.summary %}
<div class="text-xs text-slate-500 mt-1 line-clamp-1 max-w-lg">{{ d.summary }}</div>
{% endif %}
</td>
<!-- Date -->
<td class="px-4 py-3 text-xs text-slate-500 hidden md:table-cell whitespace-nowrap">{{ d.date }}</td>
<!-- Dimension bars -->
{% macro dim_cell(value) %}
<td class="px-4 py-3 hidden lg:table-cell">
<div class="flex items-center gap-1.5">
<span class="dim-bar-bg">
<span class="dim-bar-fill {% if value >= 4 %}dim-fill-high{% elif value >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
style="width: {{ (value / 5 * 100)|int }}%"></span>
</span>
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ value }}</span>
</div>
</td>
{% endmacro %}
{{ dim_cell(d.novelty) }}
{{ dim_cell(d.maturity) }}
{{ dim_cell(d.relevance) }}
<td class="px-4 py-3 hidden xl:table-cell">
<div class="flex items-center gap-1.5">
<span class="dim-bar-bg">
<span class="dim-bar-fill {% if d.momentum >= 4 %}dim-fill-high{% elif d.momentum >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
style="width: {{ (d.momentum / 5 * 100)|int }}%"></span>
</span>
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ d.momentum }}</span>
</div>
</td>
<td class="px-4 py-3 hidden xl:table-cell">
<div class="flex items-center gap-1.5">
<span class="dim-bar-bg">
<span class="dim-bar-fill {% if d.overlap >= 4 %}dim-fill-high{% elif d.overlap >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
style="width: {{ (d.overlap / 5 * 100)|int }}%"></span>
</span>
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ d.overlap }}</span>
</div>
</td>
<!-- Categories -->
<td class="px-4 py-3 hidden md:table-cell">
<div class="flex flex-wrap gap-1">
{% for cat in d.categories[:3] %}
<span class="cat-pill">{{ cat }}</span>
{% endfor %}
{% if d.categories|length > 3 %}
<span class="cat-pill opacity-50">+{{ d.categories|length - 3 }}</span>
{% endif %}
</div>
</td>
</tr>
{% endfor %}
{% if not result.drafts %}
<tr>
<td colspan="9" class="px-4 py-12 text-center text-slate-500">
<svg class="w-12 h-12 mx-auto mb-3 opacity-30" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z"/>
</svg>
<p class="text-sm">No drafts match your filters.</p>
<a href="/drafts" class="text-blue-400 text-sm hover:text-blue-300 mt-1 inline-block">Clear all filters</a>
</td>
</tr>
{% endif %}
</tbody>
</table>
</div>
</div>
<!-- Pagination -->
{% if result.pages > 1 %}
<nav class="flex items-center justify-center gap-1.5 mt-6">
{% if result.page > 1 %}
<a href="/drafts?page={{ result.page - 1 }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">
<svg class="w-4 h-4 inline" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7"/></svg>
Prev
</a>
{% endif %}
{% set start_page = [1, result.page - 2]|max %}
{% set end_page = [result.pages, result.page + 2]|min %}
{% if start_page > 1 %}
<a href="/drafts?page=1&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">1</a>
{% if start_page > 2 %}<span class="text-slate-600 px-1">...</span>{% endif %}
{% endif %}
{% for p in range(start_page, end_page + 1) %}
{% if p == result.page %}
<span class="page-btn page-btn-active">{{ p }}</span>
{% else %}
<a href="/drafts?page={{ p }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">{{ p }}</a>
{% endif %}
{% endfor %}
{% if end_page < result.pages %}
{% if end_page < result.pages - 1 %}<span class="text-slate-600 px-1">...</span>{% endif %}
<a href="/drafts?page={{ result.pages }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">{{ result.pages }}</a>
{% endif %}
{% if result.page < result.pages %}
<a href="/drafts?page={{ result.page + 1 }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
class="page-btn page-btn-inactive">
Next
<svg class="w-4 h-4 inline" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"/></svg>
</a>
{% endif %}
</nav>
{% endif %}
{% endblock %}

View File

@@ -0,0 +1,118 @@
{% extends "base.html" %}
{% set active_page = "gaps" %}
{% block title %}Draft Demo — Gap Explorer{% endblock %}
{% block extra_head %}
<style>
.draft-viewer {
max-height: 75vh;
overflow-y: auto;
}
.draft-viewer::-webkit-scrollbar { width: 6px; }
.draft-viewer::-webkit-scrollbar-track { background: #0f172a; }
.draft-viewer::-webkit-scrollbar-thumb { background: #334155; border-radius: 3px; }
.draft-tab {
transition: all 0.15s ease;
}
.draft-tab:hover { background: rgba(59, 130, 246, 0.1); }
.draft-tab.active {
background: rgba(59, 130, 246, 0.15);
border-color: #3b82f6;
color: #60a5fa;
}
</style>
{% endblock %}
{% block content %}
<!-- Breadcrumb -->
<nav class="mb-6 text-sm">
<a href="/gaps" class="text-blue-400 hover:text-blue-300 transition">Gap Explorer</a>
<span class="text-slate-600 mx-2">/</span>
<span class="text-slate-400">Demo Drafts</span>
</nav>
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Generated Draft Demo</h1>
<p class="text-slate-400 text-sm mt-1">
Pre-generated Internet-Drafts addressing identified gaps.
These were generated by the gap-to-draft pipeline using Claude to write each section.
</p>
</div>
{% if not generated_drafts %}
<div class="bg-slate-900 rounded-xl border border-slate-800 p-8 text-center">
<svg class="w-12 h-12 text-slate-700 mx-auto mb-3" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
<p class="text-slate-500">No generated drafts found yet.</p>
<p class="text-slate-600 text-sm mt-1">Use the gap detail page to generate one, or run <code class="text-blue-400">ietf draft-gen</code> from the CLI.</p>
</div>
{% else %}
<div class="grid grid-cols-1 lg:grid-cols-4 gap-6">
<!-- Draft selector sidebar -->
<div class="lg:col-span-1">
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
<div class="px-4 py-3 border-b border-slate-800">
<h3 class="text-sm font-semibold text-slate-300">{{ generated_drafts | length }} Generated Draft{{ 's' if generated_drafts | length != 1 }}</h3>
</div>
<div class="divide-y divide-slate-800/50">
{% for gd in generated_drafts %}
<a href="/gaps/demo?file={{ gd.filename | urlencode }}"
class="draft-tab block px-4 py-3 border-l-2
{% if (selected and gd.filename == selected) or (not selected and loop.first) %}active border-blue-500
{% else %}border-transparent{% endif %}">
<div class="text-xs font-medium text-slate-300 truncate">{{ gd.title }}</div>
<div class="text-[10px] text-slate-600 mt-0.5 font-mono">{{ gd.stem }}</div>
<div class="text-[10px] text-slate-600 mt-0.5">{{ (gd.size / 1024) | round(1) }} KB</div>
</a>
{% endfor %}
</div>
</div>
</div>
<!-- Draft viewer -->
<div class="lg:col-span-3">
{% if draft_text %}
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
<div class="flex items-center justify-between px-4 py-3 border-b border-slate-800">
<div>
<h3 class="text-sm font-semibold text-white">{{ draft_info.title if draft_info else 'Draft' }}</h3>
<span class="text-[10px] text-slate-600 font-mono">{{ draft_info.filename if draft_info }}</span>
</div>
<button onclick="downloadCurrentDraft()" class="inline-flex items-center gap-1.5 px-3 py-1.5 bg-slate-800 hover:bg-slate-700 text-slate-300 text-xs font-medium rounded-lg transition">
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 10v6m0 0l-3-3m3 3l3-3m2 8H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
Download .txt
</button>
</div>
<div class="draft-viewer p-4">
<pre class="text-xs text-slate-300 font-mono leading-relaxed whitespace-pre-wrap">{{ draft_text }}</pre>
</div>
</div>
{% else %}
<div class="bg-slate-900 rounded-xl border border-slate-800 p-8 text-center">
<p class="text-slate-500">Select a draft from the list to view it.</p>
</div>
{% endif %}
</div>
</div>
{% endif %}
{% endblock %}
{% block extra_scripts %}
<script>
function downloadCurrentDraft() {
const text = {{ draft_text | tojson if draft_text else '""' }};
const filename = {{ (draft_info.filename if draft_info else 'draft.txt') | tojson }};
if (!text) return;
const blob = new Blob([text], { type: 'text/plain' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = filename;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
}
</script>
{% endblock %}

View File

@@ -0,0 +1,211 @@
{% extends "base.html" %}
{% set active_page = "gaps" %}
{% block title %}{{ gap.topic }} — Gap Explorer{% endblock %}
{% block extra_head %}
<style>
.draft-output {
max-height: 70vh;
overflow-y: auto;
}
.draft-output::-webkit-scrollbar { width: 6px; }
.draft-output::-webkit-scrollbar-track { background: #0f172a; }
.draft-output::-webkit-scrollbar-thumb { background: #334155; border-radius: 3px; }
.generating-spinner {
display: inline-block;
width: 1rem;
height: 1rem;
border: 2px solid rgba(255,255,255,0.3);
border-top-color: white;
border-radius: 50%;
animation: spin 0.8s linear infinite;
}
@keyframes spin { to { transform: rotate(360deg); } }
</style>
{% endblock %}
{% block content %}
<!-- Breadcrumb -->
<nav class="mb-6 text-sm">
<a href="/gaps" class="text-blue-400 hover:text-blue-300 transition">Gap Explorer</a>
<span class="text-slate-600 mx-2">/</span>
<span class="text-slate-400">{{ gap.topic }}</span>
</nav>
<!-- Gap header -->
<div class="bg-slate-900 rounded-xl border
{% if gap.severity == 'critical' %}border-red-500/40
{% elif gap.severity == 'high' %}border-orange-500/30
{% elif gap.severity == 'medium' %}border-yellow-500/20
{% else %}border-slate-800{% endif %}
p-6 mb-6">
<div class="flex items-start justify-between gap-4 mb-4">
<h1 class="text-2xl font-bold text-white">{{ gap.topic }}</h1>
<span class="px-3 py-1 rounded-full text-xs font-bold whitespace-nowrap
{% if gap.severity == 'critical' %}bg-red-500/20 text-red-400 ring-1 ring-red-500/30
{% elif gap.severity == 'high' %}bg-orange-500/20 text-orange-400 ring-1 ring-orange-500/30
{% elif gap.severity == 'medium' %}bg-yellow-500/20 text-yellow-400 ring-1 ring-yellow-500/30
{% else %}bg-green-500/20 text-green-400 ring-1 ring-green-500/30{% endif %}">
{{ gap.severity | upper }}
</span>
</div>
{% if gap.category %}
<span class="inline-block px-2.5 py-0.5 rounded text-xs bg-slate-800 text-slate-400 mb-4 font-medium">{{ gap.category }}</span>
{% endif %}
<div class="space-y-4">
<div>
<h3 class="text-xs font-semibold text-slate-500 uppercase tracking-wider mb-2">Description</h3>
<p class="text-sm text-slate-300 leading-relaxed">{{ gap.description }}</p>
</div>
{% if gap.evidence %}
<div class="bg-slate-800/50 rounded-lg p-4">
<h3 class="text-xs font-semibold text-slate-500 uppercase tracking-wider mb-2">Evidence</h3>
<p class="text-sm text-slate-400 leading-relaxed">{{ gap.evidence }}</p>
</div>
{% endif %}
</div>
<!-- Links -->
<div class="mt-4 pt-4 border-t border-slate-800/50 flex flex-wrap gap-3">
<a href="/drafts?q={{ gap.topic | urlencode }}" class="inline-flex items-center gap-1.5 text-xs text-blue-400/70 hover:text-blue-400 transition">
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z"/></svg>
Search related drafts
</a>
{% if gap.category %}
<span class="text-slate-700">|</span>
<a href="/drafts?cat={{ gap.category | urlencode }}" class="inline-flex items-center gap-1.5 text-xs text-blue-400/70 hover:text-blue-400 transition">
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
Browse {{ gap.category }} drafts
</a>
{% endif %}
</div>
</div>
<!-- Draft Generation Section -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6">
<div class="flex items-center justify-between mb-4">
<div>
<h2 class="text-lg font-semibold text-white">Generate Internet-Draft</h2>
<p class="text-xs text-slate-500 mt-1">Use AI to generate a full Internet-Draft addressing this gap</p>
</div>
<button id="generateBtn" onclick="generateDraft({{ gap.id }})"
class="inline-flex items-center gap-2 px-4 py-2 bg-blue-600 hover:bg-blue-500 disabled:bg-slate-700 disabled:text-slate-500 text-white text-sm font-medium rounded-lg transition">
<svg id="genIcon" class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/></svg>
<span id="genText">Generate Draft</span>
</button>
</div>
<!-- Status area -->
<div id="genStatus" class="hidden mb-4 p-3 rounded-lg bg-blue-500/10 border border-blue-500/20">
<div class="flex items-center gap-2 text-sm text-blue-400">
<span class="generating-spinner"></span>
<span id="statusText">Generating draft... This may take 1-2 minutes.</span>
</div>
</div>
<!-- Error area -->
<div id="genError" class="hidden mb-4 p-3 rounded-lg bg-red-500/10 border border-red-500/20">
<p class="text-sm text-red-400" id="errorText"></p>
</div>
<!-- Generated draft output -->
<div id="draftOutput" class="hidden">
<div class="flex items-center justify-between mb-3">
<h3 class="text-sm font-semibold text-slate-300">Generated Draft</h3>
<button onclick="downloadDraft()" class="inline-flex items-center gap-1.5 px-3 py-1.5 bg-slate-800 hover:bg-slate-700 text-slate-300 text-xs font-medium rounded-lg transition">
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 10v6m0 0l-3-3m3 3l3-3m2 8H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
Download .txt
</button>
</div>
<div class="draft-output bg-slate-950 rounded-lg border border-slate-800 p-4">
<pre id="draftText" class="text-xs text-slate-300 font-mono leading-relaxed whitespace-pre-wrap"></pre>
</div>
</div>
<!-- Hint to demo -->
<div id="demoHint" class="mt-4 text-xs text-slate-600">
Want to see what generated drafts look like without waiting?
<a href="/gaps/demo" class="text-blue-500 hover:text-blue-400 transition">View the demo page</a>
with {{ generated_drafts | length }} pre-generated examples.
</div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
let generatedText = '';
let generatedFilename = '';
function generateDraft(gapId) {
const btn = document.getElementById('generateBtn');
const genIcon = document.getElementById('genIcon');
const genText = document.getElementById('genText');
const status = document.getElementById('genStatus');
const error = document.getElementById('genError');
const output = document.getElementById('draftOutput');
const hint = document.getElementById('demoHint');
// Disable button, show spinner
btn.disabled = true;
genIcon.innerHTML = '';
genIcon.classList.add('generating-spinner');
genText.textContent = 'Generating...';
status.classList.remove('hidden');
error.classList.add('hidden');
output.classList.add('hidden');
hint.classList.add('hidden');
fetch(`/gaps/${gapId}/generate`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
})
.then(r => r.json())
.then(data => {
status.classList.add('hidden');
if (data.error) {
error.classList.remove('hidden');
document.getElementById('errorText').textContent = data.error;
btn.disabled = false;
genIcon.classList.remove('generating-spinner');
genIcon.innerHTML = '<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/>';
genText.textContent = 'Retry';
} else {
generatedText = data.text;
generatedFilename = data.filename || 'generated-draft.txt';
document.getElementById('draftText').textContent = data.text;
output.classList.remove('hidden');
genIcon.classList.remove('generating-spinner');
genIcon.innerHTML = '<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M5 13l4 4L19 7"/>';
genText.textContent = 'Done';
}
})
.catch(err => {
status.classList.add('hidden');
error.classList.remove('hidden');
document.getElementById('errorText').textContent = 'Network error: ' + err.message;
btn.disabled = false;
genIcon.classList.remove('generating-spinner');
genIcon.innerHTML = '<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/>';
genText.textContent = 'Retry';
});
}
function downloadDraft() {
if (!generatedText) return;
const blob = new Blob([generatedText], { type: 'text/plain' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = generatedFilename;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
}
</script>
{% endblock %}

View File

@@ -0,0 +1,89 @@
{% extends "base.html" %}
{% set active_page = "gaps" %}
{% block title %}Gap Explorer — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Gap Explorer</h1>
<p class="text-slate-400 text-sm mt-1">{{ gaps | length }} identified gaps in AI/agent standards coverage — click any gap to explore details or generate a draft</p>
</div>
<!-- Action bar -->
<div class="mb-6 flex flex-wrap gap-3">
<a href="/gaps/demo" class="inline-flex items-center gap-2 px-4 py-2 bg-blue-600 hover:bg-blue-500 text-white text-sm font-medium rounded-lg transition">
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
View Demo Draft
</a>
{% if generated_drafts %}
<span class="inline-flex items-center px-3 py-2 bg-slate-800 text-slate-400 text-sm rounded-lg">
{{ generated_drafts | length }} draft{{ 's' if generated_drafts | length != 1 }} already generated
</span>
{% endif %}
</div>
<!-- Severity overview -->
{% set ns = namespace(critical=0, high=0, medium=0, low=0) %}
{% for gap in gaps %}
{% if gap.severity == 'critical' %}{% set ns.critical = ns.critical + 1 %}
{% elif gap.severity == 'high' %}{% set ns.high = ns.high + 1 %}
{% elif gap.severity == 'medium' %}{% set ns.medium = ns.medium + 1 %}
{% else %}{% set ns.low = ns.low + 1 %}
{% endif %}
{% endfor %}
<div class="grid grid-cols-2 md:grid-cols-5 gap-4 mb-6">
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-3xl font-bold text-slate-200">{{ gaps | length }}</div>
<div class="text-xs text-slate-400 mt-1">Total Gaps</div>
</div>
<div class="stat-card rounded-xl border border-red-500/30 p-4">
<div class="text-3xl font-bold text-red-400">{{ ns.critical }}</div>
<div class="text-xs text-red-400/70 mt-1">Critical</div>
</div>
<div class="stat-card rounded-xl border border-orange-500/30 p-4">
<div class="text-3xl font-bold text-orange-400">{{ ns.high }}</div>
<div class="text-xs text-orange-400/70 mt-1">High</div>
</div>
<div class="stat-card rounded-xl border border-yellow-500/30 p-4">
<div class="text-3xl font-bold text-yellow-400">{{ ns.medium }}</div>
<div class="text-xs text-yellow-400/70 mt-1">Medium</div>
</div>
<div class="stat-card rounded-xl border border-green-500/30 p-4">
<div class="text-3xl font-bold text-green-400">{{ ns.low }}</div>
<div class="text-xs text-green-400/70 mt-1">Low</div>
</div>
</div>
<!-- Gap cards sorted by severity -->
<div class="space-y-4">
{% for gap in gaps | sort(attribute='severity') %}
<a href="/gaps/{{ gap.id }}" class="block bg-slate-900 rounded-xl border
{% if gap.severity == 'critical' %}border-red-500/40 hover:border-red-500/60
{% elif gap.severity == 'high' %}border-orange-500/30 hover:border-orange-500/50
{% elif gap.severity == 'medium' %}border-yellow-500/20 hover:border-yellow-500/40
{% else %}border-slate-800 hover:border-slate-700{% endif %}
p-5 transition group">
<div class="flex items-start justify-between gap-3 mb-3">
<h2 class="text-base font-semibold text-white group-hover:text-blue-400 transition">{{ gap.topic }}</h2>
<div class="flex items-center gap-2 shrink-0">
<span class="px-2.5 py-0.5 rounded-full text-xs font-semibold whitespace-nowrap
{% if gap.severity == 'critical' %}bg-red-500/20 text-red-400 ring-1 ring-red-500/30
{% elif gap.severity == 'high' %}bg-orange-500/20 text-orange-400 ring-1 ring-orange-500/30
{% elif gap.severity == 'medium' %}bg-yellow-500/20 text-yellow-400 ring-1 ring-yellow-500/30
{% else %}bg-green-500/20 text-green-400 ring-1 ring-green-500/30{% endif %}">
{{ gap.severity | upper }}
</span>
<svg class="w-4 h-4 text-slate-600 group-hover:text-blue-400 transition" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"/></svg>
</div>
</div>
{% if gap.category %}
<span class="inline-block px-2 py-0.5 rounded text-[10px] bg-slate-800 text-slate-400 mb-3 font-medium">{{ gap.category }}</span>
{% endif %}
<p class="text-sm text-slate-400 leading-relaxed">{{ gap.description }}</p>
</a>
{% endfor %}
</div>
{% endblock %}

View File

@@ -0,0 +1,200 @@
{% extends "base.html" %}
{% set active_page = "idea_clusters" %}
{% block title %}Idea Clusters — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Idea Clusters</h1>
<p class="text-slate-400 text-sm mt-1">Extracted ideas grouped by semantic similarity using embedding-based clustering</p>
</div>
<div id="emptyState" class="hidden">
<div class="bg-slate-900 rounded-xl border border-slate-800 p-12 text-center">
<svg class="w-16 h-16 mx-auto text-slate-600 mb-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M4 6a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2V6zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2V6zM4 16a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2v-2z"/>
</svg>
<h2 class="text-lg font-semibold text-slate-300 mb-2">No idea embeddings found</h2>
<p class="text-slate-500">Run <code class="bg-slate-800 px-2 py-1 rounded text-sm font-mono text-blue-400">ietf embed-ideas</code> to generate embeddings first.</p>
</div>
</div>
<div id="clusterContent" class="hidden">
<!-- Stat cards -->
<div class="grid grid-cols-1 md:grid-cols-3 gap-4 mb-6">
<div class="stat-card rounded-xl border border-slate-800 p-5">
<p class="text-xs text-slate-500 uppercase tracking-wide">Total Ideas Embedded</p>
<p class="text-2xl font-bold text-white mt-1" id="statTotal">0</p>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-5">
<p class="text-xs text-slate-500 uppercase tracking-wide">Clusters Found</p>
<p class="text-2xl font-bold text-white mt-1" id="statClusters">0</p>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-5">
<p class="text-xs text-slate-500 uppercase tracking-wide">Avg Cluster Size</p>
<p class="text-2xl font-bold text-white mt-1" id="statAvgSize">0</p>
</div>
</div>
<!-- t-SNE Scatter -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Idea Embedding Space (t-SNE)</h2>
<p class="text-xs text-slate-500 mb-3">Each dot is an extracted idea, colored by cluster. Hover for details, click to view the source draft.</p>
<div id="scatterPlot" style="height: 560px;"></div>
</div>
<!-- Treemap -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Cluster Sizes</h2>
<p class="text-xs text-slate-500 mb-3">Treemap showing relative sizes of each idea cluster.</p>
<div id="treemapPlot" style="height: 450px;"></div>
</div>
<!-- Cluster cards grid -->
<h2 class="text-lg font-semibold text-white mb-4">Cluster Details</h2>
<div id="clusterGrid" class="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-4 mb-6">
</div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
const PLOTLY_LAYOUT = {
paper_bgcolor: 'transparent', plot_bgcolor: 'rgba(15,23,42,0.5)',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 20, r: 20, b: 50, l: 50 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const CFG = { responsive: true, displayModeBar: false };
const PALETTE = [
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
];
const data = {{ clusters | tojson }};
if (data.empty) {
document.getElementById('emptyState').classList.remove('hidden');
} else {
document.getElementById('clusterContent').classList.remove('hidden');
// Stats
const stats = data.stats;
document.getElementById('statTotal').textContent = stats.total.toLocaleString();
document.getElementById('statClusters').textContent = stats.num_clusters.toLocaleString();
document.getElementById('statAvgSize').textContent = stats.num_clusters > 0
? (stats.clustered / stats.num_clusters).toFixed(1) : '0';
// --- t-SNE Scatter ---
if (data.scatter.length > 0) {
// Group by cluster_id
const groups = {};
data.scatter.forEach(pt => {
if (!groups[pt.cluster_id]) groups[pt.cluster_id] = { x: [], y: [], text: [], names: [] };
groups[pt.cluster_id].x.push(pt.x);
groups[pt.cluster_id].y.push(pt.y);
groups[pt.cluster_id].text.push(pt.title);
groups[pt.cluster_id].names.push(pt.draft_name);
});
// Map cluster_id to cluster theme
const clusterThemes = {};
data.clusters.forEach((c, i) => {
// Find the original cluster_id by matching scatter points
});
const clusterIds = Object.keys(groups).sort((a, b) => (groups[b].x.length - groups[a].x.length));
const traces = clusterIds.map((cid, i) => {
const g = groups[cid];
const theme = data.clusters[i] ? data.clusters[i].theme : `Cluster ${cid}`;
return {
x: g.x, y: g.y, text: g.text, name: theme,
customdata: g.names,
mode: 'markers', type: 'scatter',
marker: {
size: 6,
color: PALETTE[i % PALETTE.length],
opacity: 0.8,
line: { width: 0.5, color: 'rgba(255,255,255,0.15)' },
},
hovertemplate: '<b>%{text}</b><extra>%{customdata}</extra>',
};
});
Plotly.newPlot('scatterPlot', traces, {
...PLOTLY_LAYOUT,
xaxis: { visible: false, showgrid: false, zeroline: false },
yaxis: { visible: false, showgrid: false, zeroline: false },
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
hovermode: 'closest',
margin: { t: 10, r: 20, b: 10, l: 20 },
}, CFG);
document.getElementById('scatterPlot').on('plotly_click', function(ev) {
const pt = ev.points[0];
if (pt.customdata) {
window.location.href = '/drafts/' + pt.customdata;
}
});
}
// --- Treemap ---
if (data.clusters.length > 0) {
const labels = data.clusters.map(c => c.theme);
const values = data.clusters.map(c => c.size);
const colors = data.clusters.map((_, i) => PALETTE[i % PALETTE.length]);
Plotly.newPlot('treemapPlot', [{
type: 'treemap',
labels: labels,
parents: labels.map(() => ''),
values: values,
textinfo: 'label+value',
marker: { colors: colors },
hovertemplate: '<b>%{label}</b><br>%{value} ideas<extra></extra>',
}], {
...PLOTLY_LAYOUT,
margin: { t: 10, r: 10, b: 10, l: 10 },
}, CFG);
}
// --- Cluster Cards ---
const grid = document.getElementById('clusterGrid');
data.clusters.forEach((cluster, i) => {
const color = PALETTE[i % PALETTE.length];
const topIdeas = cluster.ideas.slice(0, 3);
const ideaListHtml = topIdeas.map(idea =>
`<li class="text-xs text-slate-400 truncate" title="${idea.title}">${idea.title}</li>`
).join('');
const extraCount = cluster.size - topIdeas.length;
const extraHtml = extraCount > 0
? `<li class="text-xs text-slate-600">+${extraCount} more</li>` : '';
const draftBadges = cluster.drafts.slice(0, 4).map(d =>
`<a href="/drafts/${d}" class="inline-block bg-slate-800 text-slate-400 text-xs px-2 py-0.5 rounded hover:text-blue-400 truncate max-w-[140px]" title="${d}">${d.replace('draft-', '').substring(0, 20)}</a>`
).join(' ');
const extraDrafts = cluster.drafts.length > 4
? `<span class="text-xs text-slate-600">+${cluster.drafts.length - 4}</span>` : '';
const card = document.createElement('div');
card.className = 'bg-slate-900 rounded-xl border border-slate-800 p-5';
card.innerHTML = `
<div class="flex items-center gap-2 mb-3">
<div class="w-3 h-3 rounded-full" style="background: ${color}"></div>
<h3 class="text-sm font-semibold text-white">${cluster.theme}</h3>
<span class="ml-auto text-xs text-slate-500">${cluster.size} ideas</span>
</div>
<ul class="space-y-1 mb-3">${ideaListHtml}${extraHtml}</ul>
<div class="border-t border-slate-800 pt-3">
<p class="text-xs text-slate-500 mb-1">${cluster.drafts.length} source draft${cluster.drafts.length !== 1 ? 's' : ''}</p>
<div class="flex flex-wrap gap-1">${draftBadges}${extraDrafts}</div>
</div>
`;
grid.appendChild(card);
});
}
</script>
{% endblock %}

View File

@@ -0,0 +1,124 @@
{% extends "base.html" %}
{% set active_page = "ideas" %}
{% block title %}Ideas — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Extracted Ideas</h1>
<p class="text-slate-400 text-sm mt-1">{{ data.total }} technical ideas extracted from rated drafts</p>
</div>
<!-- Stats header -->
<div class="grid grid-cols-2 md:grid-cols-4 gap-4 mb-6">
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-3xl font-bold text-blue-400">{{ data.total }}</div>
<div class="text-xs text-slate-400 mt-1">Total Ideas</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-3xl font-bold text-purple-400">{{ data.by_type | length }}</div>
<div class="text-xs text-slate-400 mt-1">Idea Types</div>
</div>
{% set top_type = data.by_type.keys() | list %}
{% if top_type %}
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-lg font-bold text-green-400 truncate">{{ top_type[0] }}</div>
<div class="text-xs text-slate-400 mt-1">Most Common Type</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-3xl font-bold text-amber-400">{{ data.by_type[top_type[0]] }}</div>
<div class="text-xs text-slate-400 mt-1">{{ top_type[0] }} Count</div>
</div>
{% endif %}
</div>
<!-- Chart -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Ideas by Type</h2>
<div id="ideasChart" style="height: 350px;"></div>
</div>
<!-- Ideas list -->
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
<div class="p-4 border-b border-slate-800 flex flex-col sm:flex-row gap-3">
<input type="text" id="ideaSearch" placeholder="Search ideas by title, description, or draft name..."
class="flex-1 bg-slate-800 border border-slate-700 rounded-lg px-4 py-2 text-sm text-slate-200 placeholder-slate-500 focus:outline-none focus:border-blue-500">
<select id="typeFilter"
class="bg-slate-800 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500">
<option value="">All Types</option>
{% for t in data.by_type %}
<option value="{{ t }}">{{ t }} ({{ data.by_type[t] }})</option>
{% endfor %}
</select>
</div>
<div class="px-4 py-2 border-b border-slate-800/50 text-xs text-slate-500">
<span id="visibleCount">{{ data.ideas | length }}</span> ideas shown
</div>
<div class="divide-y divide-slate-800/50 max-h-[600px] overflow-y-auto" id="ideaList">
{% for idea in data.ideas %}
<div class="idea-item px-4 py-3 hover:bg-slate-800/50 transition"
data-search="{{ idea.title|lower }} {{ idea.description|lower }} {{ idea.draft_name|lower }}"
data-type="{{ idea.type|default('other', true)|lower }}">
<div class="flex items-center gap-2 mb-1 flex-wrap">
<span class="text-sm font-medium text-slate-200">{{ idea.title }}</span>
{% if idea.type %}
<span class="px-2 py-0.5 rounded text-[10px] font-medium bg-blue-500/20 text-blue-400 whitespace-nowrap">{{ idea.type }}</span>
{% endif %}
</div>
<p class="text-xs text-slate-500 leading-relaxed">{{ idea.description }}</p>
<a href="/drafts/{{ idea.draft_name }}" class="text-[10px] text-slate-600 hover:text-blue-400 transition mt-1 inline-block font-mono">{{ idea.draft_name }}</a>
</div>
{% endfor %}
</div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
const PLOTLY_LAYOUT = {
paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 10, r: 20, b: 40, l: 140 },
xaxis: { gridcolor: '#1e293b', title: 'Count' },
yaxis: { gridcolor: '#1e293b' },
};
const byType = {{ data.by_type | tojson }};
const types = Object.keys(byType).reverse();
const counts = types.map(t => byType[t]);
// Color gradient from blue to purple
const barColors = types.map((_, i) => {
const ratio = i / Math.max(types.length - 1, 1);
const r = Math.round(59 + ratio * (168 - 59));
const g = Math.round(130 + ratio * (85 - 130));
const b = Math.round(246 + ratio * (247 - 246));
return `rgb(${r},${g},${b})`;
});
Plotly.newPlot('ideasChart', [{
y: types, x: counts,
type: 'bar', orientation: 'h',
marker: { color: barColors },
hovertemplate: '<b>%{y}</b>: %{x} ideas<extra></extra>',
}], PLOTLY_LAYOUT, { responsive: true, displayModeBar: false });
// Search and filter
function filterIdeas() {
const q = document.getElementById('ideaSearch').value.toLowerCase().trim();
const typeFilter = document.getElementById('typeFilter').value.toLowerCase();
let visible = 0;
document.querySelectorAll('.idea-item').forEach(el => {
const matchesSearch = !q || el.dataset.search.includes(q);
const matchesType = !typeFilter || el.dataset.type === typeFilter;
const show = matchesSearch && matchesType;
el.style.display = show ? '' : 'none';
if (show) visible++;
});
document.getElementById('visibleCount').textContent = visible;
}
document.getElementById('ideaSearch').addEventListener('input', filterIdeas);
document.getElementById('typeFilter').addEventListener('change', filterIdeas);
</script>
{% endblock %}

View File

@@ -0,0 +1,232 @@
{% extends "base.html" %}
{% set active_page = "landscape" %}
{% block title %}Landscape — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Draft Landscape</h1>
<p class="text-slate-400 text-sm mt-1">Multi-dimensional visualization of the AI/agent draft space</p>
</div>
<!-- Embedding-based t-SNE map -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6" id="tsneSection">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Embedding Landscape (t-SNE)</h2>
<p class="text-xs text-slate-500 mb-3">768-dim embeddings projected to 2D. Color = category, size = composite score. Click for draft detail.</p>
<div id="tsneMap" style="height: 560px;"></div>
</div>
<!-- Main scatter: Novelty vs Maturity -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Novelty vs Maturity</h2>
<p class="text-xs text-slate-500 mb-3">Bubble size = composite score, color = category. Hover for details.</p>
<div id="mainScatter" style="height: 560px;"></div>
</div>
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
<!-- Novelty vs Overlap quadrant -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Innovation-Uniqueness Quadrant</h2>
<p class="text-xs text-slate-500 mb-3">Novelty vs Overlap — find the novel and unique drafts.</p>
<div id="quadrantChart" style="height: 450px;"></div>
</div>
<!-- Score distributions -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Score Distributions</h2>
<p class="text-xs text-slate-500 mb-3">Violin plots for each rating dimension.</p>
<div id="violinChart" style="height: 450px;"></div>
</div>
</div>
<!-- Category distribution -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Category Distribution</h2>
<p class="text-xs text-slate-500 mb-3">Number of rated drafts per primary category.</p>
<div id="categoryBar" style="height: 400px;"></div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
const PLOTLY_LAYOUT = {
paper_bgcolor: 'transparent', plot_bgcolor: 'rgba(15,23,42,0.5)',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 20, r: 20, b: 50, l: 50 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const CFG = { responsive: true, displayModeBar: false };
const PALETTE = [
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
];
const dist = {{ dist | tojson }};
const tsneData = {{ tsne_data | tojson }};
// --- 0. t-SNE Embedding Map ---
if (tsneData.length > 0) {
const tsneCatGroups = {};
tsneData.forEach(d => {
if (!tsneCatGroups[d.category]) tsneCatGroups[d.category] = { x: [], y: [], size: [], text: [], names: [] };
tsneCatGroups[d.category].x.push(d.x);
tsneCatGroups[d.category].y.push(d.y);
tsneCatGroups[d.category].size.push(Math.max(d.score * 4, 6));
tsneCatGroups[d.category].text.push(d.title);
tsneCatGroups[d.category].names.push(d.name);
});
const catList = Object.keys(tsneCatGroups).sort((a, b) =>
tsneCatGroups[b].x.length - tsneCatGroups[a].x.length
);
const tsneTraces = catList.map((cat, i) => {
const g = tsneCatGroups[cat];
return {
x: g.x, y: g.y, text: g.text, name: cat,
customdata: g.names,
mode: 'markers', type: 'scatter',
marker: {
size: g.size,
color: PALETTE[i % PALETTE.length],
opacity: 0.8,
line: { width: 0.5, color: 'rgba(255,255,255,0.15)' },
},
hovertemplate: '<b>%{text}</b><extra>' + cat + '</extra>',
};
});
const tsnePlot = Plotly.newPlot('tsneMap', tsneTraces, {
...PLOTLY_LAYOUT,
xaxis: { visible: false, showgrid: false, zeroline: false },
yaxis: { visible: false, showgrid: false, zeroline: false },
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
hovermode: 'closest',
margin: { t: 10, r: 20, b: 10, l: 20 },
}, CFG);
// Click to navigate to draft detail
document.getElementById('tsneMap').on('plotly_click', function(data) {
const pt = data.points[0];
if (pt.customdata) {
window.location.href = '/drafts/' + pt.customdata;
}
});
} else {
document.getElementById('tsneSection').style.display = 'none';
}
// --- Group by category for rating-based charts ---
const catGroups = {};
dist.names.forEach((name, i) => {
const cat = dist.categories[i];
if (!catGroups[cat]) catGroups[cat] = { x: [], y: [], nov: [], ovl: [], size: [], text: [], scores: [] };
catGroups[cat].x.push(dist.novelty[i] + (Math.random() - 0.5) * 0.25);
catGroups[cat].y.push(dist.maturity[i] + (Math.random() - 0.5) * 0.25);
catGroups[cat].nov.push(dist.novelty[i]);
catGroups[cat].ovl.push(dist.overlap[i]);
catGroups[cat].size.push(Math.max(dist.scores[i] * 4, 5));
catGroups[cat].text.push(name);
catGroups[cat].scores.push(dist.scores[i]);
});
// --- 1. Main Scatter: Novelty vs Maturity ---
const mainTraces = Object.entries(catGroups).map(([cat, d]) => ({
x: d.x, y: d.y, text: d.text, name: cat,
customdata: d.scores.map((s, i) => [s, d.nov[i], d.ovl[i]]),
mode: 'markers', type: 'scatter',
marker: { size: d.size, opacity: 0.75, line: { width: 0.5, color: 'rgba(255,255,255,0.15)' } },
hovertemplate: '<b>%{text}</b><br>Novelty: %{customdata[1]}<br>Maturity: %{y:.0f}<br>Score: %{customdata[0]:.2f}<br>Overlap: %{customdata[2]}<extra>' + cat + '</extra>',
}));
Plotly.newPlot('mainScatter', mainTraces, {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Novelty', range: [0.3, 5.7], dtick: 1 },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Maturity', range: [0.3, 5.7], dtick: 1 },
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
}, CFG);
// Click to navigate to draft detail
document.getElementById('mainScatter').on('plotly_click', function(data) {
const pt = data.points[0];
if (pt.text) {
window.location.href = '/drafts/' + pt.text;
}
});
// --- 2. Novelty vs Overlap Quadrant ---
const quadTraces = Object.entries(catGroups).map(([cat, d]) => ({
x: d.nov.map(v => v + (Math.random() - 0.5) * 0.25),
y: d.ovl.map(v => v + (Math.random() - 0.5) * 0.25),
text: d.text, name: cat,
customdata: d.scores,
mode: 'markers', type: 'scatter',
marker: { size: 7, opacity: 0.7 },
hovertemplate: '<b>%{text}</b><br>Novelty: %{x:.0f}<br>Overlap: %{y:.0f}<br>Score: %{customdata:.2f}<extra>' + cat + '</extra>',
showlegend: false,
}));
Plotly.newPlot('quadrantChart', quadTraces, {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Novelty', range: [0.3, 5.7], dtick: 1 },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Overlap', range: [0.3, 5.7], dtick: 1 },
shapes: [
{ type: 'line', x0: 3, x1: 3, y0: 0, y1: 6, line: { color: '#334155', width: 1, dash: 'dash' } },
{ type: 'line', x0: 0, x1: 6, y0: 3, y1: 3, line: { color: '#334155', width: 1, dash: 'dash' } },
],
annotations: [
{ x: 4.5, y: 1.2, text: 'Novel & Unique', showarrow: false, font: { size: 11, color: '#4ade80' } },
{ x: 4.5, y: 5.0, text: 'Novel & Overlapping', showarrow: false, font: { size: 11, color: '#facc15' } },
{ x: 1.5, y: 1.2, text: 'Mature & Unique', showarrow: false, font: { size: 11, color: '#60a5fa' } },
{ x: 1.5, y: 5.0, text: 'Crowded', showarrow: false, font: { size: 11, color: '#f87171' } },
],
}, CFG);
// --- 3. Violin / Box Plots ---
const dims = ['novelty', 'maturity', 'overlap', 'momentum', 'relevance'];
const dimColors = ['#3b82f6', '#22c55e', '#ef4444', '#f59e0b', '#a855f7'];
const violinTraces = dims.map((d, i) => ({
y: dist[d],
name: d.charAt(0).toUpperCase() + d.slice(1),
type: 'violin',
box: { visible: true },
meanline: { visible: true },
line: { color: dimColors[i] },
fillcolor: dimColors[i] + '30',
opacity: 0.85,
}));
Plotly.newPlot('violinChart', violinTraces, {
...PLOTLY_LAYOUT,
showlegend: false,
yaxis: { ...PLOTLY_LAYOUT.yaxis, range: [0.3, 5.7], dtick: 1, title: 'Score' },
}, CFG);
// --- 4. Category Distribution ---
const catCounts = {};
dist.categories.forEach(c => { catCounts[c] = (catCounts[c] || 0) + 1; });
const sorted = Object.entries(catCounts).sort((a, b) => b[1] - a[1]);
const catNames = sorted.map(s => s[0]).reverse();
const catValues = sorted.map(s => s[1]).reverse();
Plotly.newPlot('categoryBar', [{
y: catNames, x: catValues,
type: 'bar', orientation: 'h',
marker: {
color: catValues.map((_, i) => {
const pct = i / Math.max(catValues.length - 1, 1);
return `hsl(${210 + pct * 120}, 70%, 55%)`;
}),
},
text: catValues.map(v => v.toString()),
textposition: 'outside',
textfont: { color: '#94a3b8', size: 11 },
hovertemplate: '<b>%{y}</b><br>%{x} drafts<extra></extra>',
}], {
...PLOTLY_LAYOUT,
margin: { t: 10, r: 60, b: 40, l: 220 },
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Number of Drafts' },
yaxis: { ...PLOTLY_LAYOUT.yaxis, automargin: true },
}, CFG);
</script>
{% endblock %}

View File

@@ -0,0 +1,191 @@
{% extends "base.html" %}
{% set active_page = "monitor" %}
{% block title %}Monitor — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Live Monitor</h1>
<p class="text-slate-400 text-sm mt-1">Track automated monitoring runs and pipeline status</p>
</div>
<div id="monitor-app"></div>
{% endblock %}
{% block extra_scripts %}
<script>
const PLOTLY_LAYOUT = {
paper_bgcolor: 'transparent',
plot_bgcolor: 'transparent',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 30, r: 20, b: 40, l: 40 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const PLOTLY_CONFIG = { responsive: true, displayModeBar: false };
const PALETTE = [
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
];
const data = {{ status | tojson }};
const app = document.getElementById('monitor-app');
// Status banner
const lastRun = data.last_run;
let bannerColor, bannerBorder, bannerText;
if (!lastRun) {
bannerColor = 'text-slate-400';
bannerBorder = 'border-slate-700';
bannerText = 'No monitoring runs recorded yet. Run <code class="text-slate-300">ietf monitor run</code> to start.';
} else if (lastRun.status === 'completed') {
bannerColor = 'text-green-400';
bannerBorder = 'border-green-500/30';
bannerText = 'Last run completed successfully';
} else if (lastRun.status === 'failed') {
bannerColor = 'text-red-400';
bannerBorder = 'border-red-500/30';
bannerText = 'Last run failed: ' + (lastRun.error_message || 'unknown error');
} else {
bannerColor = 'text-yellow-400';
bannerBorder = 'border-yellow-500/30';
bannerText = 'A monitoring run is currently in progress...';
}
let html = `
<div class="stat-card rounded-xl border ${bannerBorder} p-4 mb-6">
<div class="flex items-center gap-3">
<div class="w-3 h-3 rounded-full ${lastRun && lastRun.status === 'completed' ? 'bg-green-500' : lastRun && lastRun.status === 'failed' ? 'bg-red-500' : lastRun && lastRun.status === 'running' ? 'bg-yellow-500 animate-pulse' : 'bg-slate-600'}"></div>
<span class="${bannerColor} font-medium">${bannerText}</span>
</div>
</div>
`;
// Stat cards row
const lastTime = lastRun ? (lastRun.started_at || '').replace('T', ' ').slice(0, 19) : '-';
const lastDuration = lastRun && lastRun.duration_seconds ? lastRun.duration_seconds.toFixed(1) + 's' : '-';
const lastNew = lastRun ? lastRun.new_drafts_found : 0;
const totalRuns = data.total_runs;
html += `
<div class="grid grid-cols-2 md:grid-cols-4 gap-4 mb-6">
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-2xl font-bold text-slate-200">${totalRuns}</div>
<div class="text-xs text-slate-400 mt-1">Total Runs</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-sm font-mono font-bold text-slate-200 truncate">${lastTime}</div>
<div class="text-xs text-slate-400 mt-1">Last Run</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-2xl font-bold text-slate-200">${lastDuration}</div>
<div class="text-xs text-slate-400 mt-1">Duration</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4">
<div class="text-2xl font-bold text-blue-400">${lastNew}</div>
<div class="text-xs text-slate-400 mt-1">New Drafts (Last Run)</div>
</div>
</div>
`;
// Unprocessed counts
const up = data.unprocessed;
function warnColor(n) { return n > 0 ? 'text-yellow-400 border-yellow-500/30' : 'text-green-400 border-green-500/30'; }
html += `
<h2 class="text-lg font-semibold text-white mb-3">Unprocessed Drafts</h2>
<div class="grid grid-cols-3 gap-4 mb-8">
<div class="stat-card rounded-xl border ${up.unrated > 0 ? 'border-yellow-500/30' : 'border-green-500/30'} p-4">
<div class="text-2xl font-bold ${up.unrated > 0 ? 'text-yellow-400' : 'text-green-400'}">${up.unrated}</div>
<div class="text-xs text-slate-400 mt-1">Unrated</div>
</div>
<div class="stat-card rounded-xl border ${up.unembedded > 0 ? 'border-yellow-500/30' : 'border-green-500/30'} p-4">
<div class="text-2xl font-bold ${up.unembedded > 0 ? 'text-yellow-400' : 'text-green-400'}">${up.unembedded}</div>
<div class="text-xs text-slate-400 mt-1">Un-embedded</div>
</div>
<div class="stat-card rounded-xl border ${up.no_ideas > 0 ? 'border-yellow-500/30' : 'border-green-500/30'} p-4">
<div class="text-2xl font-bold ${up.no_ideas > 0 ? 'text-yellow-400' : 'text-green-400'}">${up.no_ideas}</div>
<div class="text-xs text-slate-400 mt-1">No Ideas</div>
</div>
</div>
`;
// New drafts over time chart
const runs = data.runs.slice().reverse(); // chronological order
if (runs.length > 1) {
html += `
<h2 class="text-lg font-semibold text-white mb-3">New Drafts Found Over Time</h2>
<div id="monitor-chart" class="bg-slate-900/50 rounded-xl border border-slate-800 p-4 mb-8" style="height:300px"></div>
`;
}
// Run history table
if (data.runs.length > 0) {
html += `
<h2 class="text-lg font-semibold text-white mb-3">Run History</h2>
<div class="bg-slate-900/50 rounded-xl border border-slate-800 overflow-hidden">
<table class="w-full text-sm">
<thead>
<tr class="border-b border-slate-800 text-slate-400 text-xs uppercase">
<th class="px-4 py-3 text-left">#</th>
<th class="px-4 py-3 text-left">Started</th>
<th class="px-4 py-3 text-right">Duration</th>
<th class="px-4 py-3 text-center">Status</th>
<th class="px-4 py-3 text-right">New Drafts</th>
<th class="px-4 py-3 text-right">Analyzed</th>
<th class="px-4 py-3 text-right">Embedded</th>
<th class="px-4 py-3 text-right">Ideas</th>
</tr>
</thead>
<tbody>`;
for (const r of data.runs) {
const statusBadge = r.status === 'completed'
? '<span class="px-2 py-0.5 rounded-full text-xs font-semibold bg-green-500/20 text-green-400">completed</span>'
: r.status === 'failed'
? '<span class="px-2 py-0.5 rounded-full text-xs font-semibold bg-red-500/20 text-red-400">failed</span>'
: '<span class="px-2 py-0.5 rounded-full text-xs font-semibold bg-yellow-500/20 text-yellow-400">running</span>';
const started = (r.started_at || '').replace('T', ' ').slice(0, 19);
const dur = r.duration_seconds ? r.duration_seconds.toFixed(1) + 's' : '-';
html += `
<tr class="border-b border-slate-800/50 hover:bg-slate-800/30">
<td class="px-4 py-2.5 text-slate-500">${r.id}</td>
<td class="px-4 py-2.5 font-mono text-xs text-slate-300">${started}</td>
<td class="px-4 py-2.5 text-right text-slate-400">${dur}</td>
<td class="px-4 py-2.5 text-center">${statusBadge}</td>
<td class="px-4 py-2.5 text-right text-slate-300">${r.new_drafts_found}</td>
<td class="px-4 py-2.5 text-right text-slate-300">${r.drafts_analyzed}</td>
<td class="px-4 py-2.5 text-right text-slate-300">${r.drafts_embedded}</td>
<td class="px-4 py-2.5 text-right text-slate-300">${r.ideas_extracted}</td>
</tr>`;
}
html += `
</tbody>
</table>
</div>`;
}
app.innerHTML = html;
// Render chart
if (runs.length > 1) {
const x = runs.map(r => (r.started_at || '').slice(0, 19));
const y = runs.map(r => r.new_drafts_found || 0);
Plotly.newPlot('monitor-chart', [{
x: x,
y: y,
type: 'scatter',
mode: 'lines+markers',
fill: 'tozeroy',
line: { color: PALETTE[0], width: 2 },
marker: { color: PALETTE[0], size: 6 },
fillcolor: PALETTE[0] + '30',
}], {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Run Date', font: { size: 11 } } },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'New Drafts', font: { size: 11 } } },
}, PLOTLY_CONFIG);
}
</script>
{% endblock %}

View File

@@ -0,0 +1,205 @@
{% extends "base.html" %}
{% set active_page = "overview" %}
{% block title %}Overview — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-8">
<h1 class="text-2xl font-bold text-white">Dashboard Overview</h1>
<p class="text-slate-400 text-sm mt-1">IETF AI/Agent Internet-Drafts at a glance</p>
</div>
<!-- Stat cards -->
<div class="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-5 gap-4 mb-8">
<a href="/drafts" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-blue-500/40 transition group">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
<div class="text-3xl font-bold text-blue-400">{{ stats.total_drafts }}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-blue-400/70 transition">Total Drafts &rarr;</div>
</a>
<a href="/ratings" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-emerald-500/40 transition group">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
<div class="text-3xl font-bold text-emerald-400">{{ stats.rated_count }}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-emerald-400/70 transition">Rated Drafts &rarr;</div>
</a>
<a href="/authors" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-purple-500/40 transition group">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
<div class="text-3xl font-bold text-purple-400">{{ stats.author_count }}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-purple-400/70 transition">Authors &rarr;</div>
</a>
<a href="/ideas" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-amber-500/40 transition group">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-amber-500 to-amber-400"></div>
<div class="text-3xl font-bold text-amber-400">{{ stats.idea_count }}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-amber-400/70 transition">Ideas &rarr;</div>
</a>
<a href="/gaps" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-red-500/40 transition group">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-red-500 to-red-400"></div>
<div class="text-3xl font-bold text-red-400">{{ stats.gap_count }}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-red-400/70 transition">Gaps Found &rarr;</div>
</a>
</div>
<!-- Charts row 1: Score distribution + Category donut -->
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Composite Score Distribution</h2>
<div id="scoreHist" style="height: 300px;"></div>
</div>
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Drafts by Category</h2>
<div id="categoryPie" style="height: 300px;"></div>
</div>
</div>
<!-- Timeline (full width) -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Submissions Over Time</h2>
<div id="timeline" style="height: 350px;"></div>
</div>
<!-- Category radar -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Category Rating Profiles</h2>
<div id="radar" style="height: 420px;"></div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
// Shared Plotly config
const PLOTLY_LAYOUT = {
paper_bgcolor: 'transparent',
plot_bgcolor: 'transparent',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 30, r: 20, b: 40, l: 40 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const PLOTLY_CONFIG = { responsive: true, displayModeBar: false };
const PALETTE = [
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
];
// --- Score histogram ---
const scores = {{ scores | tojson }};
if (scores.length > 0) {
Plotly.newPlot('scoreHist', [{
x: scores,
type: 'histogram',
nbinsx: 20,
marker: {
color: 'rgba(59, 130, 246, 0.7)',
line: { color: '#3b82f6', width: 1 },
},
hovertemplate: 'Score: %{x}<br>Count: %{y}<extra></extra>',
}], {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Composite Score', font: { size: 11 } } },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'Count', font: { size: 11 } } },
}, PLOTLY_CONFIG);
} else {
document.getElementById('scoreHist').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No score data available</p>';
}
// --- Category donut ---
const categories = {{ categories | tojson }};
const catNames = Object.keys(categories);
const catVals = Object.values(categories);
if (catNames.length > 0) {
Plotly.newPlot('categoryPie', [{
labels: catNames,
values: catVals,
type: 'pie',
hole: 0.45,
textinfo: 'label+percent',
textposition: 'outside',
textfont: { size: 10, color: '#94a3b8' },
hovertemplate: '%{label}<br>%{value} drafts (%{percent})<extra></extra>',
marker: { colors: PALETTE },
pull: catVals.map((_, i) => i === 0 ? 0.03 : 0),
}], {
...PLOTLY_LAYOUT,
showlegend: false,
margin: { t: 10, r: 10, b: 10, l: 10 },
}, PLOTLY_CONFIG);
// Click category to filter drafts
document.getElementById('categoryPie').on('plotly_click', function(data) {
const cat = data.points[0].label;
if (cat) window.location.href = '/drafts?cat=' + encodeURIComponent(cat);
});
} else {
document.getElementById('categoryPie').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No category data available</p>';
}
// --- Timeline (stacked area) ---
const timeline = {{ timeline | tojson }};
if (timeline.months && timeline.months.length > 0) {
const timeTraces = timeline.categories.map((cat, i) => ({
x: timeline.months,
y: timeline.series[cat],
name: cat,
type: 'scatter',
mode: 'lines',
stackgroup: 'one',
line: { width: 0.5, color: PALETTE[i % PALETTE.length] },
fillcolor: PALETTE[i % PALETTE.length] + '80',
hovertemplate: '%{x}<br>' + cat + ': %{y}<extra></extra>',
}));
Plotly.newPlot('timeline', timeTraces, {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Month', font: { size: 11 } } },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'Drafts', font: { size: 11 } } },
legend: { font: { size: 10, color: '#94a3b8' }, orientation: 'h', y: -0.2, x: 0.5, xanchor: 'center' },
hovermode: 'x unified',
}, PLOTLY_CONFIG);
} else {
document.getElementById('timeline').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No timeline data available</p>';
}
// --- Category radar ---
const radar = {{ radar | tojson }};
const dims = ['novelty', 'maturity', 'relevance', 'momentum', 'low_overlap'];
const dimLabels = ['Novelty', 'Maturity', 'Relevance', 'Momentum', 'Low Overlap'];
const radarCats = Object.keys(radar);
if (radarCats.length > 0) {
const radarTraces = radarCats.map((cat, i) => {
const vals = radar[cat];
return {
type: 'scatterpolar',
r: dims.map(d => vals[d]).concat([vals[dims[0]]]),
theta: dimLabels.concat([dimLabels[0]]),
fill: 'toself',
fillcolor: PALETTE[i % PALETTE.length] + '20',
line: { color: PALETTE[i % PALETTE.length], width: 2 },
name: cat + ' (' + vals.count + ')',
opacity: 0.85,
hovertemplate: cat + '<br>%{theta}: %{r:.1f}<extra></extra>',
};
});
Plotly.newPlot('radar', radarTraces, {
...PLOTLY_LAYOUT,
polar: {
bgcolor: 'transparent',
radialaxis: {
visible: true,
range: [0, 5],
gridcolor: '#1e293b',
color: '#64748b',
tickfont: { size: 10 },
},
angularaxis: {
gridcolor: '#1e293b',
color: '#94a3b8',
tickfont: { size: 11 },
},
},
legend: { font: { size: 10, color: '#94a3b8' }, x: 1.05, y: 0.5 },
margin: { t: 30, r: 120, b: 30, l: 60 },
}, PLOTLY_CONFIG);
} else {
document.getElementById('radar').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No radar data available</p>';
}
</script>
{% endblock %}

View File

@@ -0,0 +1,211 @@
{% extends "base.html" %}
{% set active_page = "ratings" %}
{% block title %}Ratings — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Rating Analytics</h1>
<p class="text-slate-400 text-sm mt-1">Distribution and analysis of AI-generated ratings</p>
</div>
<!-- Score Distribution -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Composite Score Distribution</h2>
<div id="scoreHist" style="height: 300px;"></div>
</div>
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
<!-- Dimension Box Plots -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Score Distributions by Dimension</h2>
<div id="dimDist" style="height: 350px;"></div>
</div>
<!-- Category Radar -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Category Rating Profiles</h2>
<div id="radar" style="height: 350px;"></div>
</div>
</div>
<!-- Scatter: novelty vs maturity -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-3">Novelty vs Maturity (bubble = relevance)</h2>
<div id="scatter" style="height: 450px;"></div>
</div>
<!-- Top 20 Leaderboard -->
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
<div class="p-4 border-b border-slate-800">
<h2 class="text-sm font-semibold text-slate-300">Top 20 Drafts by Composite Score</h2>
</div>
<div class="overflow-x-auto">
<table class="w-full text-sm">
<thead>
<tr class="border-b border-slate-800 text-left text-xs text-slate-500">
<th class="px-4 py-3 font-medium">#</th>
<th class="px-4 py-3 font-medium">Draft</th>
<th class="px-4 py-3 font-medium text-center">Score</th>
<th class="px-4 py-3 font-medium text-center">Novelty</th>
<th class="px-4 py-3 font-medium text-center">Maturity</th>
<th class="px-4 py-3 font-medium text-center">Relevance</th>
<th class="px-4 py-3 font-medium text-center">Momentum</th>
<th class="px-4 py-3 font-medium text-center">Overlap</th>
<th class="px-4 py-3 font-medium">Category</th>
</tr>
</thead>
<tbody id="leaderboard" class="divide-y divide-slate-800/50">
</tbody>
</table>
</div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
const PLOTLY_LAYOUT = {
paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 20, r: 20, b: 40, l: 50 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const CFG = { responsive: true, displayModeBar: false };
const dist = {{ dist | tojson }};
const radar = {{ radar | tojson }};
// Score Histogram
Plotly.newPlot('scoreHist', [{
x: dist.scores,
type: 'histogram',
nbinsx: 25,
marker: { color: '#3b82f6', line: { color: '#1e40af', width: 1 } },
hovertemplate: 'Score: %{x:.1f}<br>Count: %{y}<extra></extra>',
}], {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Composite Score' },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Count' },
}, CFG);
// Box plots for each dimension
const dims = ['novelty', 'maturity', 'overlap', 'momentum', 'relevance'];
const dimLabelsBox = ['Novelty', 'Maturity', 'Overlap', 'Momentum', 'Relevance'];
const colors = ['#3b82f6', '#22c55e', '#ef4444', '#f59e0b', '#a855f7'];
const boxTraces = dims.map((d, i) => ({
y: dist[d], name: dimLabelsBox[i],
type: 'box', marker: { color: colors[i] }, boxmean: true,
}));
Plotly.newPlot('dimDist', boxTraces, {
...PLOTLY_LAYOUT,
showlegend: false,
yaxis: { ...PLOTLY_LAYOUT.yaxis, range: [0.5, 5.5], dtick: 1 },
}, CFG);
// Radar
const radarDims = ['novelty', 'maturity', 'relevance', 'momentum', 'low_overlap'];
const radarLabels = ['Novelty', 'Maturity', 'Relevance', 'Momentum', 'Low Overlap'];
const radarTraces = Object.entries(radar).map(([cat, vals]) => ({
type: 'scatterpolar',
r: radarDims.map(d => vals[d]).concat([vals[radarDims[0]]]),
theta: radarLabels.concat([radarLabels[0]]),
fill: 'toself', name: `${cat} (${vals.count})`, opacity: 0.4,
}));
Plotly.newPlot('radar', radarTraces, {
...PLOTLY_LAYOUT,
polar: {
bgcolor: 'rgba(0,0,0,0)',
radialaxis: { visible: true, range: [0, 5], gridcolor: '#1e293b', color: '#64748b' },
angularaxis: { gridcolor: '#1e293b', color: '#94a3b8' },
},
legend: { font: { size: 10, color: '#94a3b8' } },
margin: { t: 30, r: 60, b: 30, l: 60 },
}, CFG);
// Scatter: novelty vs maturity
const catGroups = {};
dist.names.forEach((name, i) => {
const cat = dist.categories[i];
if (!catGroups[cat]) catGroups[cat] = { x: [], y: [], size: [], text: [] };
catGroups[cat].x.push(dist.novelty[i] + (Math.random() - 0.5) * 0.3);
catGroups[cat].y.push(dist.maturity[i] + (Math.random() - 0.5) * 0.3);
catGroups[cat].size.push(Math.max(dist.relevance[i] * 4, 6));
catGroups[cat].text.push(name);
});
const scatterTraces = Object.entries(catGroups).map(([cat, d]) => ({
x: d.x, y: d.y, text: d.text, name: cat,
mode: 'markers', type: 'scatter',
marker: { size: d.size, opacity: 0.7 },
hovertemplate: '<b>%{text}</b><br>Novelty: %{x:.1f}<br>Maturity: %{y:.1f}<extra>' + cat + '</extra>',
}));
Plotly.newPlot('scatter', scatterTraces, {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Novelty', range: [0.5, 5.5], dtick: 1 },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Maturity', range: [0.5, 5.5], dtick: 1 },
legend: { font: { size: 10, color: '#94a3b8' } },
hovermode: 'closest',
}, CFG);
// Click scatter points to navigate to draft detail
document.getElementById('scatter').on('plotly_click', function(data) {
const pt = data.points[0];
if (pt.text) {
window.location.href = '/drafts/' + pt.text;
}
});
// Top 20 Leaderboard
(function buildLeaderboard() {
// Combine arrays into objects and sort by score descending
const drafts = dist.names.map((name, i) => ({
name,
score: dist.scores[i],
novelty: dist.novelty[i],
maturity: dist.maturity[i],
relevance: dist.relevance[i],
momentum: dist.momentum[i],
overlap: dist.overlap[i],
category: dist.categories[i],
}));
drafts.sort((a, b) => b.score - a.score);
const tbody = document.getElementById('leaderboard');
const top20 = drafts.slice(0, 20);
function scoreClass(score) {
if (score >= 3.5) return 'score-high';
if (score >= 2.5) return 'score-mid';
return 'score-low';
}
function dimBadge(val) {
const cls = val >= 4 ? 'text-green-400' : val >= 3 ? 'text-yellow-400' : 'text-slate-500';
return `<span class="${cls}">${val}</span>`;
}
top20.forEach((d, i) => {
const shortName = d.name.replace('draft-', '').substring(0, 40);
const row = document.createElement('tr');
row.className = 'hover:bg-slate-800/50 transition';
row.innerHTML = `
<td class="px-4 py-3 text-slate-500 font-mono text-xs">${i + 1}</td>
<td class="px-4 py-3">
<a href="/drafts/${d.name}" class="text-blue-400 hover:text-blue-300 transition text-xs font-mono">${shortName}</a>
</td>
<td class="px-4 py-3 text-center">
<span class="score-badge ${scoreClass(d.score)}">${d.score.toFixed(2)}</span>
</td>
<td class="px-4 py-3 text-center">${dimBadge(d.novelty)}</td>
<td class="px-4 py-3 text-center">${dimBadge(d.maturity)}</td>
<td class="px-4 py-3 text-center">${dimBadge(d.relevance)}</td>
<td class="px-4 py-3 text-center">${dimBadge(d.momentum)}</td>
<td class="px-4 py-3 text-center">${dimBadge(d.overlap)}</td>
<td class="px-4 py-3">
<span class="px-2 py-0.5 rounded text-[10px] bg-slate-800 text-slate-400">${d.category}</span>
</td>
`;
tbody.appendChild(row);
});
})();
</script>
{% endblock %}

View File

@@ -0,0 +1,249 @@
{% extends "base.html" %}
{% set active_page = "similarity" %}
{% block title %}Similarity — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Draft Similarity Graph</h1>
<p class="text-slate-400 text-sm mt-1">Force-directed graph of draft-to-draft semantic similarity based on embeddings</p>
</div>
<!-- Summary stats -->
<div class="grid grid-cols-2 md:grid-cols-3 gap-4 mb-6">
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Connected Drafts</div>
<div class="text-2xl font-bold text-white mt-1" id="statNodes">0</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Similarity Links</div>
<div class="text-2xl font-bold text-white mt-1" id="statEdges">0</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
<div class="text-xs text-slate-500 uppercase tracking-wider">Avg Similarity</div>
<div class="text-2xl font-bold text-white mt-1" id="statAvgSim">0</div>
</div>
</div>
<!-- Threshold slider -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-4 mb-6">
<div class="flex items-center gap-4 flex-wrap">
<label class="text-sm text-slate-300 font-medium">Similarity Threshold:</label>
<input type="range" id="thresholdSlider" min="0.50" max="0.99" step="0.01" value="0.75"
class="w-48 h-2 bg-slate-700 rounded-lg appearance-none cursor-pointer accent-blue-500">
<span class="text-sm font-mono text-blue-400" id="thresholdLabel">0.75</span>
<span class="text-xs text-slate-500 ml-2">(<span id="visibleEdges">0</span> edges visible)</span>
</div>
</div>
<!-- Force-directed graph -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Similarity Network</h2>
<p class="text-xs text-slate-500 mb-3">Node size = composite score, color = category. Edge opacity = similarity strength. Click a node to view draft detail.</p>
<div id="simGraph" style="height: 640px;"></div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
const PLOTLY_LAYOUT = {
paper_bgcolor: 'transparent',
plot_bgcolor: 'transparent',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 20, r: 20, b: 40, l: 50 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const CFG = { responsive: true, displayModeBar: false };
const PALETTE = [
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
];
const fullNetwork = {{ network | tojson }};
// Assign color per category
const catSet = [...new Set(fullNetwork.nodes.map(n => n.category))];
const catColor = {};
catSet.forEach((c, i) => { catColor[c] = PALETTE[i % PALETTE.length]; });
// Update stat cards
document.getElementById('statNodes').textContent = fullNetwork.stats.node_count;
document.getElementById('statEdges').textContent = fullNetwork.stats.edge_count;
document.getElementById('statAvgSim').textContent = fullNetwork.stats.avg_similarity.toFixed(3);
function renderGraph(threshold) {
const edges = fullNetwork.edges.filter(e => e.similarity >= threshold);
// Only show nodes that are connected at current threshold
const connectedNames = new Set();
edges.forEach(e => { connectedNames.add(e.source); connectedNames.add(e.target); });
const nodes = fullNetwork.nodes.filter(n => connectedNames.has(n.name));
document.getElementById('visibleEdges').textContent = edges.length;
if (nodes.length === 0) {
document.getElementById('simGraph').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No connections at this threshold. Try lowering it.</p>';
return;
}
// Build index
const N = nodes.length;
const nodeIndex = {};
const pos = [];
nodes.forEach((n, i) => {
nodeIndex[n.name] = i;
pos.push({
x: Math.cos(i * 2 * Math.PI / N) * 3 + (Math.random() - 0.5),
y: Math.sin(i * 2 * Math.PI / N) * 3 + (Math.random() - 0.5)
});
});
// Force-directed spring layout
const k = Math.sqrt(80.0 / Math.max(N, 1));
for (let iter = 0; iter < 150; iter++) {
const disp = pos.map(() => ({ x: 0, y: 0 }));
const temp = 3.0 * (1 - iter / 150);
// Repulsion between all pairs
for (let i = 0; i < N; i++) {
for (let j = i + 1; j < N; j++) {
let dx = pos[i].x - pos[j].x;
let dy = pos[i].y - pos[j].y;
let dist = Math.sqrt(dx * dx + dy * dy) || 0.01;
let force = k * k / dist;
disp[i].x += (dx / dist) * force;
disp[i].y += (dy / dist) * force;
disp[j].x -= (dx / dist) * force;
disp[j].y -= (dy / dist) * force;
}
}
// Attraction along edges
for (const e of edges) {
const si = nodeIndex[e.source];
const ti = nodeIndex[e.target];
if (si === undefined || ti === undefined) continue;
let dx = pos[si].x - pos[ti].x;
let dy = pos[si].y - pos[ti].y;
let dist = Math.sqrt(dx * dx + dy * dy) || 0.01;
let force = dist * dist / k * e.similarity;
disp[si].x -= (dx / dist) * force;
disp[si].y -= (dy / dist) * force;
disp[ti].x += (dx / dist) * force;
disp[ti].y += (dy / dist) * force;
}
// Apply with temperature
for (let i = 0; i < N; i++) {
let len = Math.sqrt(disp[i].x * disp[i].x + disp[i].y * disp[i].y) || 0.01;
pos[i].x += (disp[i].x / len) * Math.min(len, temp);
pos[i].y += (disp[i].y / len) * Math.min(len, temp);
}
}
// Count connections per node for hover
const connCount = {};
edges.forEach(e => {
connCount[e.source] = (connCount[e.source] || 0) + 1;
connCount[e.target] = (connCount[e.target] || 0) + 1;
});
// Build edge traces — group by opacity bands for performance
const edgeX = [];
const edgeY = [];
for (const e of edges) {
const si = nodeIndex[e.source];
const ti = nodeIndex[e.target];
if (si === undefined || ti === undefined) continue;
edgeX.push(pos[si].x, pos[ti].x, null);
edgeY.push(pos[si].y, pos[ti].y, null);
}
// Compute per-segment opacity based on similarity
// Plotly lines don't support per-segment opacity easily, so we use a base color
const minSim = Math.min(...edges.map(e => e.similarity));
const maxSim = Math.max(...edges.map(e => e.similarity));
const avgOpacity = edges.length > 0 ? 0.15 + 0.35 * ((maxSim + minSim) / 2 - threshold) / Math.max(1 - threshold, 0.01) : 0.2;
const edgeTrace = {
x: edgeX, y: edgeY,
mode: 'lines',
type: 'scatter',
line: { color: `rgba(100, 116, 139, ${Math.min(avgOpacity, 0.4).toFixed(2)})`, width: 0.8 },
hoverinfo: 'skip',
showlegend: false,
};
// Build node trace grouped by category for legend
const catGroups = {};
nodes.forEach((n, i) => {
if (!catGroups[n.category]) catGroups[n.category] = { x: [], y: [], size: [], text: [], names: [] };
catGroups[n.category].x.push(pos[i].x);
catGroups[n.category].y.push(pos[i].y);
catGroups[n.category].size.push(Math.max(n.score * 4, 6));
catGroups[n.category].text.push(
`<b>${n.title}</b><br>Category: ${n.category}<br>Score: ${n.score}<br>Connections: ${connCount[n.name] || 0}`
);
catGroups[n.category].names.push(n.name);
});
const catList = Object.keys(catGroups).sort((a, b) =>
catGroups[b].x.length - catGroups[a].x.length
);
const nodeTraces = catList.map((cat, i) => {
const g = catGroups[cat];
return {
x: g.x, y: g.y,
customdata: g.names,
mode: 'markers',
type: 'scatter',
name: cat,
marker: {
size: g.size,
color: catColor[cat] || '#64748b',
opacity: 0.85,
line: { color: 'rgba(255,255,255,0.15)', width: 1 },
},
hovertext: g.text,
hoverinfo: 'text',
};
});
Plotly.newPlot('simGraph', [edgeTrace, ...nodeTraces], {
...PLOTLY_LAYOUT,
xaxis: { visible: false, showgrid: false, zeroline: false },
yaxis: { visible: false, showgrid: false, zeroline: false },
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent', x: 1.02, y: 0.5 },
margin: { t: 10, r: 140, b: 10, l: 10 },
hovermode: 'closest',
}, CFG);
// Click to navigate to draft detail
document.getElementById('simGraph').on('plotly_click', function(data) {
const pt = data.points[0];
if (pt.customdata) {
window.location.href = '/drafts/' + pt.customdata;
}
});
}
// Initial render
renderGraph(0.75);
// Threshold slider
const slider = document.getElementById('thresholdSlider');
const label = document.getElementById('thresholdLabel');
slider.addEventListener('input', function() {
const val = parseFloat(this.value);
label.textContent = val.toFixed(2);
renderGraph(val);
});
</script>
{% endblock %}

View File

@@ -0,0 +1,241 @@
{% extends "base.html" %}
{% set active_page = "timeline" %}
{% block title %}Timeline — IETF Draft Analyzer{% endblock %}
{% block content %}
<div class="mb-6">
<h1 class="text-2xl font-bold text-white">Timeline Animation</h1>
<p class="text-slate-400 text-sm mt-1">Watch the AI/agent draft landscape evolve month by month</p>
</div>
<!-- Stats summary -->
<div class="grid grid-cols-3 gap-4 mb-6" id="statCards">
</div>
<!-- Animated t-SNE map -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6" id="tsneSection">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Animated Embedding Landscape</h2>
<p class="text-xs text-slate-500 mb-3">t-SNE projection with cumulative drafts per month. Color = category, size = composite score. Press Play to animate.</p>
<div id="monthBadge" class="text-center mb-2">
<span class="inline-block bg-slate-800 border border-slate-700 rounded-lg px-4 py-1.5 text-sm font-mono text-blue-400"></span>
</div>
<div id="tsneAnim" style="height: 560px;"></div>
</div>
<!-- Stacked area chart -->
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
<h2 class="text-sm font-semibold text-slate-300 mb-1">Category Submissions Over Time</h2>
<p class="text-xs text-slate-500 mb-3">Stacked area chart showing draft submissions by category per month.</p>
<div id="stackedArea" style="height: 400px;"></div>
</div>
{% endblock %}
{% block extra_scripts %}
<script>
const PLOTLY_LAYOUT = {
paper_bgcolor: 'transparent', plot_bgcolor: 'rgba(15,23,42,0.5)',
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
margin: { t: 20, r: 20, b: 50, l: 50 },
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
};
const CFG = { responsive: true, displayModeBar: false };
const PALETTE = [
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
];
const animData = {{ animation | tojson }};
const points = animData.points;
const months = animData.months;
const catMonthly = animData.category_monthly;
if (points.length > 0 && months.length > 0) {
// --- Stat cards ---
const firstMonth = months[0];
const lastMonth = months[months.length - 1];
const allCats = [...new Set(points.map(p => p.category))];
document.getElementById('statCards').innerHTML = `
<div class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
<div class="text-3xl font-bold text-blue-400">${months.length}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider">Months Span</div>
<div class="text-xs text-slate-500 mt-0.5">${firstMonth} to ${lastMonth}</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
<div class="text-3xl font-bold text-emerald-400">${points.length}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider">Total Drafts</div>
</div>
<div class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden">
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
<div class="text-3xl font-bold text-purple-400">${allCats.length}</div>
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider">Categories</div>
</div>
`;
// --- Build category list sorted by frequency ---
const catCounts = {};
points.forEach(p => { catCounts[p.category] = (catCounts[p.category] || 0) + 1; });
const catList = Object.keys(catCounts).sort((a, b) => catCounts[b] - catCounts[a]);
const catColor = {};
catList.forEach((c, i) => { catColor[c] = PALETTE[i % PALETTE.length]; });
// --- Helper: build traces for points up to a given month ---
function buildTraces(upToMonth) {
const filtered = points.filter(p => p.month <= upToMonth);
const groups = {};
filtered.forEach(p => {
if (!groups[p.category]) groups[p.category] = { x: [], y: [], size: [], text: [], names: [] };
groups[p.category].x.push(p.x);
groups[p.category].y.push(p.y);
groups[p.category].size.push(Math.max(p.score * 4, 6));
groups[p.category].text.push(p.title);
groups[p.category].names.push(p.name);
});
return catList.map(cat => {
const g = groups[cat] || { x: [], y: [], size: [], text: [], names: [] };
return {
x: g.x, y: g.y, text: g.text, name: cat,
customdata: g.names,
mode: 'markers', type: 'scatter',
marker: {
size: g.size,
color: catColor[cat],
opacity: 0.8,
line: { width: 0.5, color: 'rgba(255,255,255,0.15)' },
},
hovertemplate: '<b>%{text}</b><extra>' + cat + '</extra>',
};
});
}
// --- Build frames ---
const frames = months.map(month => {
const cumCount = points.filter(p => p.month <= month).length;
return {
name: month,
data: buildTraces(month),
};
});
// --- Initial plot (first month) ---
const firstTraces = buildTraces(months[0]);
const firstCount = points.filter(p => p.month <= months[0]).length;
// Slider steps
const sliderSteps = months.map(month => ({
method: 'animate',
label: month,
args: [[month], { frame: { duration: 500, redraw: true }, transition: { duration: 300 }, mode: 'immediate' }],
}));
const layout = {
...PLOTLY_LAYOUT,
xaxis: { visible: false, showgrid: false, zeroline: false },
yaxis: { visible: false, showgrid: false, zeroline: false },
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
hovermode: 'closest',
margin: { t: 40, r: 20, b: 60, l: 20 },
updatemenus: [{
type: 'buttons', showactive: false, x: 0.05, y: 1.08,
buttons: [
{
label: '&#9654; Play',
method: 'animate',
args: [null, { frame: { duration: 500, redraw: true }, transition: { duration: 300 }, fromcurrent: true }]
},
{
label: '&#9724; Pause',
method: 'animate',
args: [[null], { frame: { duration: 0, redraw: true }, mode: 'immediate' }]
}
]
}],
sliders: [{
active: 0,
steps: sliderSteps,
x: 0.05, len: 0.9,
xanchor: 'left',
y: -0.02,
yanchor: 'top',
pad: { t: 30, b: 10 },
currentvalue: { visible: false },
transition: { duration: 300 },
font: { size: 9, color: '#64748b' },
bgcolor: '#1e293b',
activebgcolor: '#3b82f6',
bordercolor: '#334155',
borderwidth: 1,
ticklen: 4,
tickcolor: '#475569',
}],
};
Plotly.newPlot('tsneAnim', firstTraces, layout, CFG).then(() => {
Plotly.addFrames('tsneAnim', frames);
});
// Update badge on animation frame
const badge = document.querySelector('#monthBadge span');
badge.textContent = `Month: ${months[0]} (${firstCount} drafts)`;
document.getElementById('tsneAnim').on('plotly_animatingframe', function(ev) {
const month = ev.name;
const cumCount = points.filter(p => p.month <= month).length;
badge.textContent = `Month: ${month} (${cumCount} drafts)`;
});
// Click to navigate
document.getElementById('tsneAnim').on('plotly_click', function(data) {
const pt = data.points[0];
if (pt.customdata) {
window.location.href = '/drafts/' + pt.customdata;
}
});
// --- Stacked area chart ---
// Collect all categories across all months
const areaCats = {};
Object.values(catMonthly).forEach(mc => {
Object.keys(mc).forEach(c => { areaCats[c] = true; });
});
// Sort by total count
const areaCatList = Object.keys(areaCats).sort((a, b) => {
const totalA = months.reduce((s, m) => s + ((catMonthly[m] || {})[a] || 0), 0);
const totalB = months.reduce((s, m) => s + ((catMonthly[m] || {})[b] || 0), 0);
return totalB - totalA;
});
const areaTraces = areaCatList.map((cat, i) => ({
x: months,
y: months.map(m => (catMonthly[m] || {})[cat] || 0),
name: cat,
type: 'scatter',
mode: 'lines',
stackgroup: 'one',
line: { width: 0.5, color: catColor[cat] || PALETTE[i % PALETTE.length] },
fillcolor: (catColor[cat] || PALETTE[i % PALETTE.length]) + '80',
hovertemplate: '%{x}<br>' + cat + ': %{y}<extra></extra>',
}));
Plotly.newPlot('stackedArea', areaTraces, {
...PLOTLY_LAYOUT,
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Month', font: { size: 11 } } },
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'Drafts', font: { size: 11 } } },
legend: { font: { size: 10, color: '#94a3b8' }, orientation: 'h', y: -0.25, x: 0.5, xanchor: 'center' },
hovermode: 'x unified',
margin: { t: 20, r: 20, b: 80, l: 50 },
}, CFG);
} else {
document.getElementById('tsneSection').innerHTML = '<p class="text-slate-500 text-sm text-center py-20">No timeline animation data available. Run the analysis pipeline first.</p>';
document.getElementById('stackedArea').innerHTML = '<p class="text-slate-500 text-sm text-center py-20">No data available.</p>';
document.getElementById('statCards').style.display = 'none';
}
</script>
{% endblock %}