Idea quality pipeline, web UI features, academic paper
- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing 1,907 ideas to 468 across 434 drafts (78% reduction) - Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity - Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter) using Claude to rate ideas 1-5, removing 49 generic building blocks - Final count: 419 high-quality ideas (avg 1.1/draft) - Web UI: gap explorer with live draft generation and pre-generated demos - Web UI: D3.js author collaboration network (498 nodes, 1142 edges, 68 clusters, org filtering, interactive zoom/pan) - Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft AI agent standards landscape - Save improvement ideas backlog to data/reports/improvement-ideas.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -77,7 +77,7 @@ Abstract: {abstract}
|
||||
|
||||
{text_excerpt}
|
||||
|
||||
Return 0-8 ideas. Only include CONCRETE, NOVEL technical contributions — not restatements of the abstract or general goals. If the draft has no substantive technical ideas (e.g. it is a problem statement, administrative document, or off-topic), return an empty array [].
|
||||
Return 1-4 ideas. Extract only TOP-LEVEL novel contributions. Do NOT list sub-features, optimizations, variants, or extensions as separate ideas. If a draft defines one protocol with multiple features, that is ONE idea, not several. Each idea must be independently novel — could it be its own draft? If not, merge it with the parent idea. Only include CONCRETE, NOVEL technical contributions — not restatements of the abstract or general goals. If the draft has no substantive technical ideas (e.g. it is a problem statement, administrative document, or off-topic), return an empty array [].
|
||||
JSON array only, no fences."""
|
||||
|
||||
BATCH_IDEAS_PROMPT = """\
|
||||
@@ -86,7 +86,7 @@ Per idea: {{"title":"short name","description":"1 sentence","type":"mechanism|pr
|
||||
|
||||
{drafts_block}
|
||||
|
||||
0-8 ideas per draft. Only include CONCRETE, NOVEL technical contributions. If a draft has no substantive ideas, map it to an empty array. Do not pad with restatements of the abstract.
|
||||
1-4 ideas per draft. Extract only TOP-LEVEL novel contributions. Do NOT list sub-features, optimizations, variants, or extensions as separate ideas. If a draft defines one protocol with multiple features, that is ONE idea, not several. Each idea must be independently novel — could it be its own draft? If not, merge it with the parent idea. Only include CONCRETE, NOVEL technical contributions. If a draft has no substantive ideas, map it to an empty array. Do not pad with restatements of the abstract.
|
||||
Return ONLY a JSON object like {{"draft-name":[...], ...}}, no fences."""
|
||||
|
||||
GAP_ANALYSIS_PROMPT = """\
|
||||
@@ -115,6 +115,21 @@ Focus on:
|
||||
|
||||
JSON array only, no fences."""
|
||||
|
||||
SCORE_NOVELTY_PROMPT = """\
|
||||
Rate each idea's novelty/originality on a 1-5 scale.
|
||||
|
||||
1 = Generic building block anyone would include (e.g. "Agent Gateway", "Certificate Authority")
|
||||
2 = Obvious extension of existing work, minimal originality
|
||||
3 = Useful and relevant but expected given the problem space
|
||||
4 = Interesting contribution with some original thinking
|
||||
5 = Genuinely novel mechanism, protocol, or architectural insight
|
||||
|
||||
Ideas to score:
|
||||
{ideas_block}
|
||||
|
||||
Return ONLY a JSON object mapping idea ID to score, like {{"123": 3, "456": 1, ...}}.
|
||||
No fences, no explanation."""
|
||||
|
||||
|
||||
def _prompt_hash(text: str) -> str:
|
||||
return hashlib.sha256(text.encode()).hexdigest()[:16]
|
||||
@@ -558,3 +573,222 @@ class Analyzer:
|
||||
return text
|
||||
except anthropic.APIError as e:
|
||||
return f"Error: {e}"
|
||||
|
||||
def dedup_ideas(self, threshold: float = 0.85, dry_run: bool = True,
|
||||
draft_name: str | None = None) -> dict:
|
||||
"""Deduplicate ideas within each draft using embedding similarity.
|
||||
|
||||
For each draft, computes pairwise cosine similarity of idea embeddings.
|
||||
Ideas above the threshold are merged (keeping the one with the longer
|
||||
description).
|
||||
|
||||
Args:
|
||||
threshold: Cosine similarity threshold for merging (default 0.85).
|
||||
dry_run: If True, report what would be merged without deleting.
|
||||
draft_name: If provided, only dedup ideas for this draft.
|
||||
|
||||
Returns:
|
||||
Dict with keys: total_before, total_after, merged_count, examples.
|
||||
"""
|
||||
import numpy as np
|
||||
import ollama as ollama_lib
|
||||
|
||||
client = ollama_lib.Client(host=self.config.ollama_url)
|
||||
|
||||
# Get list of drafts to process
|
||||
if draft_name:
|
||||
draft_names = [draft_name]
|
||||
else:
|
||||
rows = self.db.conn.execute(
|
||||
"SELECT DISTINCT draft_name FROM ideas ORDER BY draft_name"
|
||||
).fetchall()
|
||||
draft_names = [r["draft_name"] for r in rows]
|
||||
|
||||
total_before = 0
|
||||
merged_count = 0
|
||||
examples = []
|
||||
ids_to_delete = []
|
||||
|
||||
for dname in draft_names:
|
||||
ideas = self.db.get_ideas_for_draft(dname)
|
||||
if len(ideas) < 2:
|
||||
total_before += len(ideas)
|
||||
continue
|
||||
|
||||
total_before += len(ideas)
|
||||
|
||||
# Embed each idea: "title: description"
|
||||
texts = [f"{idea['title']}: {idea['description']}" for idea in ideas]
|
||||
try:
|
||||
resp = client.embed(
|
||||
model=self.config.ollama_embed_model, input=texts
|
||||
)
|
||||
vectors = [
|
||||
np.array(v, dtype=np.float32)
|
||||
for v in resp["embeddings"]
|
||||
]
|
||||
except Exception as e:
|
||||
console.print(f"[red]Failed to embed ideas for {dname}: {e}[/]")
|
||||
continue
|
||||
|
||||
# Track which ideas are already marked for deletion in this draft
|
||||
deleted_in_draft = set()
|
||||
|
||||
# Compare all pairs within this draft
|
||||
for i in range(len(ideas)):
|
||||
if ideas[i]["id"] in deleted_in_draft:
|
||||
continue
|
||||
for j in range(i + 1, len(ideas)):
|
||||
if ideas[j]["id"] in deleted_in_draft:
|
||||
continue
|
||||
|
||||
# Cosine similarity
|
||||
dot = np.dot(vectors[i], vectors[j])
|
||||
norm = np.linalg.norm(vectors[i]) * np.linalg.norm(vectors[j])
|
||||
sim = float(dot / norm) if norm > 0 else 0.0
|
||||
|
||||
if sim >= threshold:
|
||||
# Keep the idea with the longer description
|
||||
keep = ideas[i] if len(ideas[i]["description"]) >= len(ideas[j]["description"]) else ideas[j]
|
||||
drop = ideas[j] if keep is ideas[i] else ideas[i]
|
||||
|
||||
ids_to_delete.append(drop["id"])
|
||||
deleted_in_draft.add(drop["id"])
|
||||
merged_count += 1
|
||||
|
||||
if len(examples) < 20:
|
||||
examples.append({
|
||||
"draft": dname,
|
||||
"keep": keep["title"],
|
||||
"drop": drop["title"],
|
||||
"similarity": round(sim, 3),
|
||||
})
|
||||
|
||||
if not dry_run:
|
||||
for idea_id in ids_to_delete:
|
||||
self.db.delete_idea(idea_id)
|
||||
|
||||
total_after = total_before - merged_count
|
||||
return {
|
||||
"total_before": total_before,
|
||||
"total_after": total_after,
|
||||
"merged_count": merged_count,
|
||||
"examples": examples,
|
||||
}
|
||||
|
||||
def score_idea_novelty(self, batch_size: int = 20, cheap: bool = True) -> dict:
|
||||
"""Score all unscored ideas for novelty (1-5) using Claude.
|
||||
|
||||
Args:
|
||||
batch_size: Number of ideas per API call (default 20).
|
||||
cheap: Use Haiku model for lower cost (default True).
|
||||
|
||||
Returns:
|
||||
Dict with keys: scored_count, avg_score, distribution.
|
||||
"""
|
||||
unscored = self.db.ideas_with_drafts(unscored_only=True)
|
||||
if not unscored:
|
||||
console.print("All ideas already scored.")
|
||||
return {"scored_count": 0, "avg_score": 0.0, "distribution": {}}
|
||||
|
||||
model_label = "Haiku" if cheap else "Sonnet"
|
||||
console.print(
|
||||
f"Scoring [bold]{len(unscored)}[/] ideas for novelty "
|
||||
f"(batches of {batch_size}, {model_label})..."
|
||||
)
|
||||
|
||||
scored_count = 0
|
||||
all_scores: list[int] = []
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
MofNCompleteColumn(),
|
||||
console=console,
|
||||
) as progress:
|
||||
task = progress.add_task("Scoring novelty...", total=len(unscored))
|
||||
|
||||
for i in range(0, len(unscored), batch_size):
|
||||
batch = unscored[i:i + batch_size]
|
||||
progress.update(task, description=f"Batch {i // batch_size + 1}")
|
||||
|
||||
# Build ideas block for prompt
|
||||
ideas_block = ""
|
||||
for idea in batch:
|
||||
ideas_block += (
|
||||
f"\n---\nID: {idea['id']}\n"
|
||||
f"Draft: {idea['draft_title']}\n"
|
||||
f"Idea: {idea['title']}\n"
|
||||
f"Description: {idea['description']}\n"
|
||||
)
|
||||
|
||||
prompt = SCORE_NOVELTY_PROMPT.format(ideas_block=ideas_block)
|
||||
phash = _prompt_hash(prompt)
|
||||
|
||||
# Check cache
|
||||
cached = self.db.get_cached_response("_novelty_score_", phash)
|
||||
if cached:
|
||||
try:
|
||||
scores = json.loads(cached)
|
||||
if isinstance(scores, dict):
|
||||
batch_scores = {int(k): int(v) for k, v in scores.items()}
|
||||
self.db.update_idea_scores_bulk(batch_scores)
|
||||
scored_count += len(batch_scores)
|
||||
all_scores.extend(batch_scores.values())
|
||||
progress.advance(task, advance=len(batch))
|
||||
continue
|
||||
except (json.JSONDecodeError, KeyError, ValueError):
|
||||
pass
|
||||
|
||||
try:
|
||||
text, in_tok, out_tok = self._call_claude(
|
||||
prompt, max_tokens=50 * len(batch), cheap=cheap
|
||||
)
|
||||
text = self._extract_json(text)
|
||||
scores = json.loads(text)
|
||||
|
||||
if not isinstance(scores, dict):
|
||||
console.print(f"[red]Batch {i // batch_size + 1}: unexpected response format[/]")
|
||||
progress.advance(task, advance=len(batch))
|
||||
continue
|
||||
|
||||
# Cache the raw response
|
||||
self.db.cache_response(
|
||||
"_novelty_score_", phash,
|
||||
self.config.claude_model_cheap if cheap else self.config.claude_model,
|
||||
prompt, text, in_tok, out_tok,
|
||||
)
|
||||
|
||||
# Parse and store scores
|
||||
batch_scores = {}
|
||||
for k, v in scores.items():
|
||||
try:
|
||||
idea_id = int(k)
|
||||
score = max(1, min(5, int(v)))
|
||||
batch_scores[idea_id] = score
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
|
||||
self.db.update_idea_scores_bulk(batch_scores)
|
||||
scored_count += len(batch_scores)
|
||||
all_scores.extend(batch_scores.values())
|
||||
|
||||
except (json.JSONDecodeError, anthropic.APIError) as e:
|
||||
console.print(f"[red]Batch {i // batch_size + 1} failed: {e}[/]")
|
||||
|
||||
progress.advance(task, advance=len(batch))
|
||||
|
||||
# Build distribution
|
||||
distribution: dict[int, int] = {}
|
||||
for s in all_scores:
|
||||
distribution[s] = distribution.get(s, 0) + 1
|
||||
|
||||
avg = sum(all_scores) / len(all_scores) if all_scores else 0.0
|
||||
|
||||
in_tok, out_tok = self.db.total_tokens_used()
|
||||
console.print(
|
||||
f"Scored [bold green]{scored_count}[/] ideas "
|
||||
f"(avg: {avg:.1f}) | Tokens: {in_tok:,} in + {out_tok:,} out"
|
||||
)
|
||||
return {"scored_count": scored_count, "avg_score": round(avg, 2), "distribution": distribution}
|
||||
|
||||
@@ -256,6 +256,60 @@ def embed():
|
||||
db.close()
|
||||
|
||||
|
||||
# ── embed-ideas ──────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@main.command("embed-ideas")
|
||||
@click.option("--limit", default=0, help="Max ideas to embed (0=all)")
|
||||
@click.option("--batch-size", default=50, help="Batch size for Ollama")
|
||||
def embed_ideas(limit: int, batch_size: int):
|
||||
"""Generate embeddings for extracted ideas via Ollama."""
|
||||
import ollama as ollama_lib
|
||||
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, MofNCompleteColumn
|
||||
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
client = ollama_lib.Client(host=cfg.ollama_url)
|
||||
|
||||
try:
|
||||
missing = db.ideas_without_embeddings(limit=limit if limit > 0 else 10000)
|
||||
if not missing:
|
||||
console.print("All ideas already have embeddings.")
|
||||
return
|
||||
|
||||
total = len(missing)
|
||||
console.print(f"Embedding [bold]{total}[/] ideas in batches of {batch_size}...")
|
||||
|
||||
count = 0
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
MofNCompleteColumn(),
|
||||
console=console,
|
||||
) as progress:
|
||||
task = progress.add_task("Embedding ideas...", total=total)
|
||||
for start in range(0, total, batch_size):
|
||||
batch = missing[start:start + batch_size]
|
||||
texts = [f"{idea['title']}. {idea['description']}" for idea in batch]
|
||||
try:
|
||||
resp = client.embed(model=cfg.ollama_embed_model, input=texts)
|
||||
for i, idea in enumerate(batch):
|
||||
import numpy as np
|
||||
vec = np.array(resp["embeddings"][i], dtype=np.float32)
|
||||
db.store_idea_embedding(idea["id"], cfg.ollama_embed_model, vec)
|
||||
count += 1
|
||||
progress.advance(task)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Batch failed: {e}[/]")
|
||||
for _ in batch:
|
||||
progress.advance(task)
|
||||
|
||||
console.print(f"Embedded [bold green]{count}[/] ideas")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ── similar ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -531,6 +585,261 @@ def co_occurrence_report():
|
||||
db.close()
|
||||
|
||||
|
||||
@report.command("wg")
|
||||
def wg_report():
|
||||
"""Working group analysis report — overlaps, alignment, submission targets."""
|
||||
from .reports import Reporter
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
reporter = Reporter(cfg, db)
|
||||
try:
|
||||
path = reporter.wg_report()
|
||||
console.print(f"Report saved: [bold]{path}[/]")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ── wg (working group analysis) ─────────────────────────────────────────
|
||||
|
||||
|
||||
@main.group()
|
||||
def wg():
|
||||
"""Working group analysis — overlaps, alignment opportunities, submission targets."""
|
||||
pass
|
||||
|
||||
|
||||
@wg.command("list")
|
||||
@click.option("--min-drafts", default=1, help="Minimum drafts to show a WG")
|
||||
def wg_list(min_drafts: int):
|
||||
"""List working groups with draft counts and average scores."""
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
try:
|
||||
summaries = db.wg_summary()
|
||||
if not summaries:
|
||||
console.print("[yellow]No WG data. Run: python scripts/backfill-wg-names.py[/]")
|
||||
return
|
||||
|
||||
summaries = [s for s in summaries if s["draft_count"] >= min_drafts]
|
||||
|
||||
table = Table(title=f"Working Groups ({len(summaries)} with >= {min_drafts} drafts)")
|
||||
table.add_column("WG", style="cyan", width=12)
|
||||
table.add_column("#", justify="right", width=4)
|
||||
table.add_column("Ideas", justify="right", width=5)
|
||||
table.add_column("Nov", justify="center", width=4)
|
||||
table.add_column("Mat", justify="center", width=4)
|
||||
table.add_column("Ovl", justify="center", width=4)
|
||||
table.add_column("Mom", justify="center", width=4)
|
||||
table.add_column("Rel", justify="center", width=4)
|
||||
table.add_column("Top Categories")
|
||||
|
||||
for s in summaries:
|
||||
top_cats = sorted(s["categories"].items(), key=lambda x: x[1], reverse=True)[:3]
|
||||
cats_str = ", ".join(f"{c}({n})" for c, n in top_cats) if top_cats else "-"
|
||||
table.add_row(
|
||||
s["wg"], str(s["draft_count"]), str(s["idea_count"]),
|
||||
str(s["avg_novelty"]), str(s["avg_maturity"]),
|
||||
str(s["avg_overlap"]), str(s["avg_momentum"]),
|
||||
str(s["avg_relevance"]), cats_str,
|
||||
)
|
||||
|
||||
console.print(table)
|
||||
|
||||
# Also show individual submission count
|
||||
indiv = db.conn.execute(
|
||||
'SELECT COUNT(*) FROM drafts WHERE "group" = \'none\' OR "group" IS NULL'
|
||||
).fetchone()[0]
|
||||
console.print(f"\n[dim]Individual submissions (no WG): {indiv}[/]")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@wg.command("show")
|
||||
@click.argument("name")
|
||||
def wg_show(name: str):
|
||||
"""Show details for a specific working group."""
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
try:
|
||||
drafts = db.wg_drafts(name)
|
||||
if not drafts:
|
||||
console.print(f"[red]No drafts found for WG: {name}[/]")
|
||||
return
|
||||
|
||||
console.print(f"\n[bold]Working Group: {name}[/] ({len(drafts)} drafts)\n")
|
||||
|
||||
table = Table()
|
||||
table.add_column("Date", style="dim", width=10)
|
||||
table.add_column("Name", style="cyan")
|
||||
table.add_column("Title", max_width=50)
|
||||
table.add_column("Score", justify="right", width=6)
|
||||
|
||||
for d in drafts:
|
||||
rating = db.get_rating(d.name)
|
||||
score = f"{rating.composite_score:.1f}" if rating else "-"
|
||||
table.add_row(d.date, d.name, d.title[:50], score)
|
||||
|
||||
console.print(table)
|
||||
|
||||
# Show ideas for this WG
|
||||
ideas = []
|
||||
for d in drafts:
|
||||
ideas.extend(db.get_ideas_for_draft(d.name))
|
||||
if ideas:
|
||||
console.print(f"\n[bold]Ideas ({len(ideas)}):[/]")
|
||||
for idea in ideas[:15]:
|
||||
console.print(f" - [cyan]{idea['title']}[/]: {idea['description'][:80]}")
|
||||
if len(ideas) > 15:
|
||||
console.print(f" [dim]... and {len(ideas) - 15} more[/]")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@wg.command("overlaps")
|
||||
@click.option("--min-wgs", default=2, help="Minimum WGs sharing a category to show")
|
||||
def wg_overlaps(min_wgs: int):
|
||||
"""Find categories and ideas that span multiple WGs — alignment opportunities."""
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
try:
|
||||
# Category spread across WGs
|
||||
spread = db.category_wg_spread()
|
||||
multi = [s for s in spread if s["wg_count"] >= min_wgs
|
||||
and not all(w["wg"] == "none" for w in s["wgs"])]
|
||||
|
||||
if multi:
|
||||
console.print(f"\n[bold]Categories spanning {min_wgs}+ WGs[/]\n")
|
||||
for s in multi:
|
||||
wg_strs = [f"{w['wg']}({w['count']})" for w in s["wgs"] if w["wg"] != "none"]
|
||||
if wg_strs:
|
||||
console.print(f" [cyan]{s['category']}[/] — {s['total_drafts']} drafts across {s['wg_count']} WGs")
|
||||
console.print(f" WGs: {', '.join(wg_strs)}")
|
||||
|
||||
# Idea overlap across WGs
|
||||
idea_overlaps = db.wg_idea_overlap()
|
||||
cross_wg = [o for o in idea_overlaps
|
||||
if not all(w == "none" for w in o["wg_names"])]
|
||||
|
||||
if cross_wg:
|
||||
console.print(f"\n[bold]Ideas appearing in {min_wgs}+ WGs ({len(cross_wg)} found)[/]\n")
|
||||
for o in cross_wg[:20]:
|
||||
real_wgs = [w for w in o["wg_names"] if w != "none"]
|
||||
console.print(f" [cyan]{o['idea_title']}[/] — WGs: {', '.join(real_wgs)}")
|
||||
for entry in o["wgs"]:
|
||||
if entry["wg"] != "none":
|
||||
console.print(f" - [{entry['wg']}] {entry['draft_name']}")
|
||||
if len(cross_wg) > 20:
|
||||
console.print(f"\n [dim]... and {len(cross_wg) - 20} more[/]")
|
||||
|
||||
if not multi and not cross_wg:
|
||||
console.print("[yellow]No cross-WG overlaps found.[/]")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@wg.command("alignment")
|
||||
def wg_alignment():
|
||||
"""Identify where individual drafts should be consolidated into WG standards."""
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
try:
|
||||
# Compare individual vs WG category distribution
|
||||
dist = db.individual_vs_wg_categories()
|
||||
indiv = dist["individual"]
|
||||
adopted = dist["wg_adopted"]
|
||||
|
||||
console.print("\n[bold]Individual vs WG-Adopted Category Distribution[/]\n")
|
||||
|
||||
table = Table()
|
||||
table.add_column("Category", width=25)
|
||||
table.add_column("Individual", justify="right", width=10)
|
||||
table.add_column("WG-Adopted", justify="right", width=10)
|
||||
table.add_column("Signal", width=40)
|
||||
|
||||
all_cats = sorted(set(list(indiv.keys()) + list(adopted.keys())))
|
||||
for cat in all_cats:
|
||||
i_count = indiv.get(cat, 0)
|
||||
w_count = adopted.get(cat, 0)
|
||||
signal = ""
|
||||
if i_count >= 5 and w_count == 0:
|
||||
signal = "[yellow]High individual activity, no WG — needs WG?[/]"
|
||||
elif i_count >= 3 and w_count >= 1:
|
||||
signal = "[green]WG exists, individual drafts could target it[/]"
|
||||
elif w_count > i_count and i_count > 0:
|
||||
signal = "[dim]WG leading, some individual work[/]"
|
||||
table.add_row(cat, str(i_count), str(w_count), signal)
|
||||
|
||||
console.print(table)
|
||||
|
||||
# Find overlap clusters within individual submissions that might warrant a WG
|
||||
console.print("\n[bold]Consolidation Candidates[/]")
|
||||
console.print("[dim]Categories with many individual drafts but no WG adoption — "
|
||||
"potential for new WG or BoF[/]\n")
|
||||
|
||||
candidates = []
|
||||
for cat in all_cats:
|
||||
i_count = indiv.get(cat, 0)
|
||||
w_count = adopted.get(cat, 0)
|
||||
if i_count >= 5 and w_count == 0:
|
||||
candidates.append((cat, i_count))
|
||||
|
||||
if candidates:
|
||||
for cat, count in sorted(candidates, key=lambda x: x[1], reverse=True):
|
||||
console.print(f" [yellow]{cat}[/]: {count} individual drafts, no WG home")
|
||||
# Show sample drafts
|
||||
rows = db.conn.execute("""
|
||||
SELECT d.name, d.title FROM drafts d
|
||||
JOIN ratings r ON d.name = r.draft_name
|
||||
WHERE (d."group" = 'none' OR d."group" IS NULL)
|
||||
AND r.categories LIKE ?
|
||||
ORDER BY (r.novelty * 0.30 + r.relevance * 0.25 + r.maturity * 0.20
|
||||
+ r.momentum * 0.15 + (6 - r.overlap) * 0.10) DESC
|
||||
LIMIT 5
|
||||
""", (f"%{cat}%",)).fetchall()
|
||||
for row in rows:
|
||||
console.print(f" - {row['name']}: {row['title'][:60]}")
|
||||
console.print()
|
||||
else:
|
||||
console.print(" [green]All active categories have WG representation.[/]")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@wg.command("targets")
|
||||
def wg_targets():
|
||||
"""Suggest best WGs for submitting new work in each category."""
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
try:
|
||||
spread = db.category_wg_spread()
|
||||
summaries = {s["wg"]: s for s in db.wg_summary()}
|
||||
|
||||
console.print("\n[bold]Recommended Submission Targets by Category[/]\n")
|
||||
|
||||
for s in spread:
|
||||
cat = s["category"]
|
||||
# Filter to real WGs (not 'none')
|
||||
real_wgs = [w for w in s["wgs"] if w["wg"] != "none"]
|
||||
if not real_wgs:
|
||||
console.print(f" [cyan]{cat}[/]: [yellow]No active WG — individual submission[/]")
|
||||
continue
|
||||
|
||||
best = real_wgs[0]
|
||||
wg_info = summaries.get(best["wg"], {})
|
||||
console.print(
|
||||
f" [cyan]{cat}[/]: [bold green]{best['wg']}[/] "
|
||||
f"({best['count']} drafts"
|
||||
f"{', avg relevance ' + str(wg_info.get('avg_relevance', '?')) if wg_info else ''})"
|
||||
)
|
||||
if len(real_wgs) > 1:
|
||||
alts = ", ".join(f"{w['wg']}({w['count']})" for w in real_wgs[1:3])
|
||||
console.print(f" Also: {alts}")
|
||||
|
||||
console.print()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ── visualize ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -808,14 +1117,21 @@ def network(top: int):
|
||||
# ── ideas ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@main.command()
|
||||
@click.argument("name", required=False)
|
||||
@main.group(invoke_without_command=True)
|
||||
@click.option("--name", default=None, help="Extract ideas from a specific draft")
|
||||
@click.option("--all", "extract_all", is_flag=True, help="Extract ideas from all drafts")
|
||||
@click.option("--limit", "-n", default=50, help="Max drafts to extract (with --all)")
|
||||
@click.option("--batch", "-b", default=5, help="Drafts per API call (default 5, set 1 for individual)")
|
||||
@click.option("--cheap/--quality", default=True, help="Use Haiku (cheap) vs Sonnet (quality)")
|
||||
def ideas(name: str | None, extract_all: bool, limit: int, batch: int, cheap: bool):
|
||||
"""Extract technical ideas from drafts using Claude."""
|
||||
@click.option("--reextract", is_flag=True, help="Clear existing ideas and re-extract with current prompt")
|
||||
@click.option("--draft", "reextract_draft", default=None, help="Specific draft to re-extract (with --reextract)")
|
||||
@click.pass_context
|
||||
def ideas(ctx, name: str | None, extract_all: bool, limit: int, batch: int, cheap: bool,
|
||||
reextract: bool, reextract_draft: str | None):
|
||||
"""Extract, score, and filter technical ideas from drafts."""
|
||||
if ctx.invoked_subcommand is not None:
|
||||
return
|
||||
|
||||
from .analyzer import Analyzer
|
||||
|
||||
cfg = _get_config()
|
||||
@@ -823,7 +1139,24 @@ def ideas(name: str | None, extract_all: bool, limit: int, batch: int, cheap: bo
|
||||
analyzer = Analyzer(cfg, db)
|
||||
|
||||
try:
|
||||
if extract_all:
|
||||
if reextract:
|
||||
# Clear existing ideas, then re-extract
|
||||
deleted = db.delete_ideas(draft_name=reextract_draft)
|
||||
if reextract_draft:
|
||||
console.print(f"Cleared [bold]{deleted}[/] ideas for {reextract_draft}")
|
||||
idea_list = analyzer.extract_ideas(reextract_draft, use_cache=True)
|
||||
if idea_list:
|
||||
console.print(f"Re-extracted [bold green]{len(idea_list)}[/] ideas:")
|
||||
for idea in idea_list:
|
||||
console.print(f" [{idea.get('type', '?')}] [bold]{idea['title']}[/]")
|
||||
console.print(f" {idea['description']}\n")
|
||||
else:
|
||||
console.print("[red]Re-extraction failed or no ideas found[/]")
|
||||
else:
|
||||
console.print(f"Cleared [bold]{deleted}[/] ideas from all drafts")
|
||||
count = analyzer.extract_all_ideas(limit=limit, batch_size=batch, cheap=cheap)
|
||||
console.print(f"Re-extracted ideas from [bold green]{count}[/] drafts")
|
||||
elif extract_all:
|
||||
count = analyzer.extract_all_ideas(limit=limit, batch_size=batch, cheap=cheap)
|
||||
console.print(f"Extracted ideas from [bold green]{count}[/] drafts")
|
||||
elif name:
|
||||
@@ -836,7 +1169,166 @@ def ideas(name: str | None, extract_all: bool, limit: int, batch: int, cheap: bo
|
||||
else:
|
||||
console.print("[red]Extraction failed or no ideas found[/]")
|
||||
else:
|
||||
console.print("Provide a draft name or use --all")
|
||||
console.print("Use --name DRAFT, --all, or a subcommand: ideas score / ideas filter")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@ideas.command("score")
|
||||
@click.option("--cheap/--quality", default=True, help="Use Haiku (cheap) vs Sonnet (quality)")
|
||||
@click.option("--batch", "-b", default=20, help="Ideas per API call (default 20)")
|
||||
def ideas_score(cheap: bool, batch: int):
|
||||
"""Score ideas for novelty (1=generic, 5=genuinely novel)."""
|
||||
from .analyzer import Analyzer
|
||||
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
analyzer = Analyzer(cfg, db)
|
||||
|
||||
try:
|
||||
stats = analyzer.score_idea_novelty(batch_size=batch, cheap=cheap)
|
||||
|
||||
if stats["scored_count"] == 0:
|
||||
return
|
||||
|
||||
# Show distribution table
|
||||
dist = db.idea_score_distribution()
|
||||
table = Table(title="Novelty Score Distribution")
|
||||
table.add_column("Score", style="bold", justify="center")
|
||||
table.add_column("Label", style="dim")
|
||||
table.add_column("Count", justify="right")
|
||||
table.add_column("Bar", min_width=30)
|
||||
|
||||
labels = {
|
||||
1: "Generic building block",
|
||||
2: "Obvious extension",
|
||||
3: "Useful but expected",
|
||||
4: "Interesting contribution",
|
||||
5: "Genuinely novel",
|
||||
}
|
||||
max_count = max(dist.values()) if dist else 1
|
||||
for score in range(1, 6):
|
||||
count = dist.get(score, 0)
|
||||
bar_len = int(30 * count / max_count) if max_count > 0 else 0
|
||||
table.add_row(
|
||||
str(score), labels[score], str(count),
|
||||
"[green]" + "#" * bar_len + "[/]"
|
||||
)
|
||||
|
||||
total = sum(dist.values())
|
||||
unscored = db.idea_count() - total
|
||||
console.print(table)
|
||||
console.print(f"\nTotal scored: [bold]{total}[/] | Unscored: {unscored} | Avg: [bold]{stats['avg_score']:.1f}[/]")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@ideas.command("filter")
|
||||
@click.option("--min-score", "-m", default=2, help="Remove ideas below this score (default 2)")
|
||||
@click.option("--dry-run/--execute", default=True, help="Preview (default) or actually delete")
|
||||
def ideas_filter(min_score: int, dry_run: bool):
|
||||
"""Filter out low-novelty ideas by score threshold."""
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
|
||||
try:
|
||||
candidates = db.ideas_below_score(min_score)
|
||||
if not candidates:
|
||||
console.print(f"No ideas with novelty_score < {min_score}.")
|
||||
return
|
||||
|
||||
# Show what would be removed
|
||||
table = Table(
|
||||
title=f"Ideas with novelty_score < {min_score} "
|
||||
f"({'DRY RUN' if dry_run else 'WILL DELETE'})"
|
||||
)
|
||||
table.add_column("Score", style="bold", justify="center")
|
||||
table.add_column("Idea", style="cyan", max_width=40)
|
||||
table.add_column("Draft", max_width=50)
|
||||
table.add_column("Description", max_width=60)
|
||||
|
||||
for idea in candidates[:50]: # Show first 50
|
||||
table.add_row(
|
||||
str(idea["novelty_score"]),
|
||||
idea["title"],
|
||||
idea["draft_title"],
|
||||
idea["description"][:60] + ("..." if len(idea["description"]) > 60 else ""),
|
||||
)
|
||||
|
||||
console.print(table)
|
||||
|
||||
if len(candidates) > 50:
|
||||
console.print(f" ... and {len(candidates) - 50} more")
|
||||
|
||||
console.print(f"\nTotal to remove: [bold red]{len(candidates)}[/] / {db.idea_count()} ideas")
|
||||
|
||||
if not dry_run:
|
||||
deleted = db.delete_low_score_ideas(min_score)
|
||||
console.print(f"[bold red]Deleted {deleted} low-novelty ideas.[/]")
|
||||
console.print(f"Remaining ideas: [bold green]{db.idea_count()}[/]")
|
||||
else:
|
||||
console.print("[dim]Use --execute to actually delete.[/]")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ── dedup-ideas ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@main.command("dedup-ideas")
|
||||
@click.option("--threshold", "-t", default=0.85, type=float,
|
||||
help="Cosine similarity threshold for merging (default 0.85)")
|
||||
@click.option("--dry-run/--execute", default=True,
|
||||
help="Preview merges (default) vs actually delete duplicates")
|
||||
@click.option("--draft", "draft_name", default=None,
|
||||
help="Limit to a single draft name")
|
||||
def dedup_ideas(threshold: float, dry_run: bool, draft_name: str | None):
|
||||
"""Deduplicate similar ideas within each draft using embedding similarity."""
|
||||
from .analyzer import Analyzer
|
||||
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
analyzer = Analyzer(cfg, db)
|
||||
|
||||
try:
|
||||
mode = "[bold yellow]DRY RUN[/]" if dry_run else "[bold red]EXECUTE[/]"
|
||||
console.print(f"\n{mode} — Deduplicating ideas (threshold={threshold})")
|
||||
if draft_name:
|
||||
console.print(f"Limiting to draft: [bold]{draft_name}[/]")
|
||||
console.print()
|
||||
|
||||
result = analyzer.dedup_ideas(
|
||||
threshold=threshold, dry_run=dry_run, draft_name=draft_name
|
||||
)
|
||||
|
||||
if result["examples"]:
|
||||
table = Table(title="Merge Candidates" if dry_run else "Merged Ideas")
|
||||
table.add_column("Draft", style="dim", max_width=40)
|
||||
table.add_column("Keep", style="green")
|
||||
table.add_column("Drop", style="red")
|
||||
table.add_column("Similarity", justify="right")
|
||||
|
||||
for ex in result["examples"]:
|
||||
table.add_row(
|
||||
ex["draft"].split("/")[-1][:40],
|
||||
ex["keep"],
|
||||
ex["drop"],
|
||||
f"{ex['similarity']:.3f}",
|
||||
)
|
||||
console.print(table)
|
||||
console.print()
|
||||
|
||||
action = "Would remove" if dry_run else "Removed"
|
||||
console.print(
|
||||
f"Ideas before: [bold]{result['total_before']}[/] | "
|
||||
f"{action}: [bold]{result['merged_count']}[/] | "
|
||||
f"After: [bold]{result['total_after']}[/]"
|
||||
)
|
||||
|
||||
if dry_run and result["merged_count"] > 0:
|
||||
console.print(
|
||||
"\n[dim]Run with --execute to apply these merges.[/]"
|
||||
)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -2024,3 +2516,163 @@ def observatory_diff(since: str | None):
|
||||
console.print(f" [{d.get('source', '?')}] {d.get('name', '?')}: {d.get('title', '')[:60]}")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# ── monitor ─────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@main.group()
|
||||
def monitor():
|
||||
"""Monitor IETF Datatracker for new AI/agent drafts."""
|
||||
pass
|
||||
|
||||
|
||||
@monitor.command("run")
|
||||
@click.option("--analyze/--no-analyze", default=True, help="Analyze new drafts")
|
||||
@click.option("--embed/--no-embed", default=True, help="Generate embeddings")
|
||||
@click.option("--ideas/--no-ideas", default=True, help="Extract ideas")
|
||||
def monitor_run(analyze, embed, ideas):
|
||||
"""Run one monitoring cycle: fetch -> analyze -> embed -> ideas."""
|
||||
from .analyzer import Analyzer
|
||||
from .embeddings import Embedder
|
||||
from .fetcher import Fetcher
|
||||
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
run_id = db.start_monitor_run()
|
||||
stats = {
|
||||
"new_drafts_found": 0,
|
||||
"drafts_analyzed": 0,
|
||||
"drafts_embedded": 0,
|
||||
"ideas_extracted": 0,
|
||||
}
|
||||
|
||||
try:
|
||||
console.print("[bold]Monitor run started[/]")
|
||||
|
||||
# Determine since date from last successful run
|
||||
last_run = db.get_last_successful_run()
|
||||
since = last_run["completed_at"][:10] if last_run and last_run.get("completed_at") else cfg.fetch_since
|
||||
console.print(f" Fetching drafts since: [cyan]{since}[/]")
|
||||
|
||||
# Fetch new drafts
|
||||
fetcher = Fetcher(cfg)
|
||||
try:
|
||||
existing_count = db.count_drafts()
|
||||
drafts = fetcher.search_drafts(keywords=list(cfg.search_keywords), since=since)
|
||||
for draft in drafts:
|
||||
db.upsert_draft(draft)
|
||||
|
||||
# Download text for any missing
|
||||
missing_text = db.drafts_without_text()
|
||||
if missing_text:
|
||||
console.print(f" Downloading text for [bold]{len(missing_text)}[/] drafts...")
|
||||
texts = fetcher.download_texts(missing_text)
|
||||
for name, text in texts.items():
|
||||
draft = db.get_draft(name)
|
||||
if draft:
|
||||
draft.full_text = text
|
||||
db.upsert_draft(draft)
|
||||
finally:
|
||||
fetcher.close()
|
||||
|
||||
new_count = db.count_drafts() - existing_count
|
||||
stats["new_drafts_found"] = max(new_count, 0)
|
||||
console.print(f" New drafts found: [bold green]{stats['new_drafts_found']}[/]")
|
||||
|
||||
# Analyze unrated drafts
|
||||
if analyze:
|
||||
unrated = db.unrated_drafts(limit=200)
|
||||
if unrated:
|
||||
console.print(f" Analyzing [bold]{len(unrated)}[/] unrated drafts...")
|
||||
analyzer = Analyzer(cfg, db)
|
||||
count = analyzer.rate_all_unrated(limit=200)
|
||||
stats["drafts_analyzed"] = count
|
||||
console.print(f" Analyzed: [bold green]{count}[/]")
|
||||
|
||||
# Embed missing drafts
|
||||
if embed:
|
||||
missing_embed = db.drafts_without_embeddings(limit=500)
|
||||
if missing_embed:
|
||||
console.print(f" Embedding [bold]{len(missing_embed)}[/] drafts...")
|
||||
embedder = Embedder(cfg, db)
|
||||
count = embedder.embed_all_missing()
|
||||
stats["drafts_embedded"] = count
|
||||
console.print(f" Embedded: [bold green]{count}[/]")
|
||||
|
||||
# Extract ideas
|
||||
if ideas:
|
||||
missing_ideas = db.drafts_without_ideas(limit=500)
|
||||
if missing_ideas:
|
||||
console.print(f" Extracting ideas from [bold]{len(missing_ideas)}[/] drafts...")
|
||||
analyzer = Analyzer(cfg, db)
|
||||
count = analyzer.extract_all_ideas(limit=500, batch_size=5, cheap=True)
|
||||
stats["ideas_extracted"] = count
|
||||
console.print(f" Ideas extracted from: [bold green]{count}[/] drafts")
|
||||
|
||||
db.complete_monitor_run(run_id, stats)
|
||||
console.print("\n[bold green]Monitor run completed successfully[/]")
|
||||
|
||||
except Exception as e:
|
||||
db.fail_monitor_run(run_id, str(e))
|
||||
console.print(f"\n[bold red]Monitor run failed:[/] {e}")
|
||||
raise
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
@monitor.command("status")
|
||||
def monitor_status():
|
||||
"""Show monitoring status and recent runs."""
|
||||
cfg = _get_config()
|
||||
db = Database(cfg)
|
||||
|
||||
try:
|
||||
runs = db.get_monitor_runs(limit=20)
|
||||
last = db.get_last_successful_run()
|
||||
|
||||
# Unprocessed counts
|
||||
unrated = len(db.unrated_drafts(limit=9999))
|
||||
unembedded = len(db.drafts_without_embeddings(limit=9999))
|
||||
no_ideas = len(db.drafts_without_ideas(limit=9999))
|
||||
|
||||
console.print("\n[bold]Monitor Status[/]\n")
|
||||
|
||||
if last:
|
||||
console.print(f" Last successful run: [green]{last['completed_at']}[/]")
|
||||
console.print(f" Duration: {last['duration_seconds']:.1f}s")
|
||||
console.print(f" New drafts: {last['new_drafts_found']}")
|
||||
else:
|
||||
console.print(" [yellow]No successful runs yet[/]")
|
||||
|
||||
console.print(f"\n[bold]Unprocessed[/]")
|
||||
console.print(f" Unrated: [{'yellow' if unrated > 0 else 'green'}]{unrated}[/]")
|
||||
console.print(f" Unembedded: [{'yellow' if unembedded > 0 else 'green'}]{unembedded}[/]")
|
||||
console.print(f" No ideas: [{'yellow' if no_ideas > 0 else 'green'}]{no_ideas}[/]")
|
||||
|
||||
if runs:
|
||||
console.print(f"\n[bold]Recent Runs[/] ({len(runs)} total)\n")
|
||||
table = Table()
|
||||
table.add_column("#", justify="right", width=4)
|
||||
table.add_column("Started", width=20)
|
||||
table.add_column("Duration", justify="right", width=8)
|
||||
table.add_column("Status", width=10)
|
||||
table.add_column("New", justify="right", width=5)
|
||||
table.add_column("Analyzed", justify="right", width=8)
|
||||
table.add_column("Embedded", justify="right", width=8)
|
||||
table.add_column("Ideas", justify="right", width=6)
|
||||
for r in runs:
|
||||
status_style = {"completed": "green", "failed": "red", "running": "yellow"}.get(r["status"], "dim")
|
||||
table.add_row(
|
||||
str(r["id"]),
|
||||
r["started_at"][:19] if r["started_at"] else "",
|
||||
f"{r['duration_seconds']:.1f}s" if r["duration_seconds"] else "-",
|
||||
f"[{status_style}]{r['status']}[/{status_style}]",
|
||||
str(r["new_drafts_found"]),
|
||||
str(r["drafts_analyzed"]),
|
||||
str(r["drafts_embedded"]),
|
||||
str(r["ideas_extracted"]),
|
||||
)
|
||||
console.print(table)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -106,6 +106,14 @@ CREATE TABLE IF NOT EXISTS ideas (
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_ideas_draft ON ideas(draft_name);
|
||||
|
||||
-- Idea embeddings (for clustering)
|
||||
CREATE TABLE IF NOT EXISTS idea_embeddings (
|
||||
idea_id INTEGER PRIMARY KEY REFERENCES ideas(id),
|
||||
model TEXT NOT NULL,
|
||||
vector BLOB NOT NULL,
|
||||
created_at TEXT
|
||||
);
|
||||
|
||||
-- Gap analysis results
|
||||
CREATE TABLE IF NOT EXISTS gaps (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
@@ -184,6 +192,20 @@ CREATE TABLE IF NOT EXISTS gap_history (
|
||||
recorded_at TEXT
|
||||
);
|
||||
|
||||
-- Monitor runs
|
||||
CREATE TABLE IF NOT EXISTS monitor_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
started_at TEXT NOT NULL,
|
||||
completed_at TEXT,
|
||||
status TEXT DEFAULT 'running',
|
||||
new_drafts_found INTEGER DEFAULT 0,
|
||||
drafts_analyzed INTEGER DEFAULT 0,
|
||||
drafts_embedded INTEGER DEFAULT 0,
|
||||
ideas_extracted INTEGER DEFAULT 0,
|
||||
error_message TEXT DEFAULT '',
|
||||
duration_seconds REAL DEFAULT 0
|
||||
);
|
||||
|
||||
-- Triggers to keep FTS index in sync
|
||||
CREATE TRIGGER IF NOT EXISTS drafts_ai AFTER INSERT ON drafts BEGIN
|
||||
INSERT INTO drafts_fts(rowid, name, title, abstract, full_text)
|
||||
@@ -234,6 +256,12 @@ class Database:
|
||||
for col, typedef in migrations:
|
||||
if col not in cols:
|
||||
self._conn.execute(f"ALTER TABLE drafts ADD COLUMN {col} {typedef}")
|
||||
|
||||
# ideas table migrations
|
||||
idea_cols = {r[1] for r in self._conn.execute("PRAGMA table_info(ideas)").fetchall()}
|
||||
if "novelty_score" not in idea_cols:
|
||||
self._conn.execute("ALTER TABLE ideas ADD COLUMN novelty_score INTEGER")
|
||||
|
||||
self._conn.commit()
|
||||
|
||||
def close(self) -> None:
|
||||
@@ -501,12 +529,13 @@ class Database:
|
||||
ORDER BY da.author_order""",
|
||||
(draft_name,),
|
||||
).fetchall()
|
||||
cols = rows[0].keys() if rows else []
|
||||
return [Author(
|
||||
person_id=r["person_id"], name=r["name"],
|
||||
ascii_name=r.get("ascii_name", ""),
|
||||
affiliation=r.get("affiliation", ""),
|
||||
resource_uri=r.get("resource_uri", ""),
|
||||
fetched_at=r.get("fetched_at"),
|
||||
ascii_name=r["ascii_name"] if "ascii_name" in cols else "",
|
||||
affiliation=r["affiliation"] if "affiliation" in cols else "",
|
||||
resource_uri=r["resource_uri"] if "resource_uri" in cols else "",
|
||||
fetched_at=r["fetched_at"] if "fetched_at" in cols else None,
|
||||
) for r in rows]
|
||||
|
||||
def drafts_without_authors(self, limit: int = 500) -> list[str]:
|
||||
@@ -624,13 +653,42 @@ class Database:
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def delete_ideas(self, draft_name: str | None = None) -> int:
|
||||
"""Delete ideas from the ideas table.
|
||||
|
||||
Args:
|
||||
draft_name: If provided, delete only ideas for this draft.
|
||||
If None, delete all ideas.
|
||||
|
||||
Returns:
|
||||
Number of rows deleted.
|
||||
"""
|
||||
if draft_name:
|
||||
self.conn.execute(
|
||||
"DELETE FROM idea_embeddings WHERE idea_id IN (SELECT id FROM ideas WHERE draft_name = ?)", (draft_name,)
|
||||
)
|
||||
cursor = self.conn.execute(
|
||||
"DELETE FROM ideas WHERE draft_name = ?", (draft_name,)
|
||||
)
|
||||
else:
|
||||
self.conn.execute("DELETE FROM idea_embeddings")
|
||||
cursor = self.conn.execute("DELETE FROM ideas")
|
||||
self.conn.commit()
|
||||
return cursor.rowcount
|
||||
|
||||
def get_ideas_for_draft(self, draft_name: str) -> list[dict]:
|
||||
rows = self.conn.execute(
|
||||
"SELECT * FROM ideas WHERE draft_name = ?", (draft_name,)
|
||||
).fetchall()
|
||||
return [{"title": r["title"], "description": r["description"],
|
||||
return [{"id": r["id"], "title": r["title"], "description": r["description"],
|
||||
"type": r["idea_type"], "draft_name": r["draft_name"]} for r in rows]
|
||||
|
||||
def delete_idea(self, idea_id: int) -> None:
|
||||
"""Delete a single idea and its embedding by ID."""
|
||||
self.conn.execute("DELETE FROM idea_embeddings WHERE idea_id = ?", (idea_id,))
|
||||
self.conn.execute("DELETE FROM ideas WHERE id = ?", (idea_id,))
|
||||
self.conn.commit()
|
||||
|
||||
def drafts_without_ideas(self, limit: int = 500) -> list[str]:
|
||||
rows = self.conn.execute(
|
||||
"""SELECT d.name FROM drafts d
|
||||
@@ -653,6 +711,103 @@ class Database:
|
||||
def idea_count(self) -> int:
|
||||
return self.conn.execute("SELECT COUNT(*) FROM ideas").fetchone()[0]
|
||||
|
||||
def ideas_with_drafts(self, unscored_only: bool = False, limit: int = 5000) -> list[dict]:
|
||||
"""Return ideas joined with draft title, optionally only unscored ones."""
|
||||
where = "WHERE i.novelty_score IS NULL" if unscored_only else ""
|
||||
rows = self.conn.execute(
|
||||
f"""SELECT i.id, i.draft_name, i.title, i.description, i.idea_type,
|
||||
i.novelty_score, d.title AS draft_title
|
||||
FROM ideas i JOIN drafts d ON i.draft_name = d.name
|
||||
{where}
|
||||
ORDER BY i.id LIMIT ?""",
|
||||
(limit,),
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
def update_idea_score(self, idea_id: int, score: int) -> None:
|
||||
"""Set the novelty_score for a single idea."""
|
||||
self.conn.execute(
|
||||
"UPDATE ideas SET novelty_score = ? WHERE id = ?",
|
||||
(score, idea_id),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def update_idea_scores_bulk(self, scores: dict[int, int]) -> None:
|
||||
"""Bulk-update novelty scores. scores maps idea_id -> score."""
|
||||
self.conn.executemany(
|
||||
"UPDATE ideas SET novelty_score = ? WHERE id = ?",
|
||||
[(score, idea_id) for idea_id, score in scores.items()],
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def delete_low_score_ideas(self, min_score: int) -> int:
|
||||
"""Delete ideas with novelty_score below min_score. Returns count deleted."""
|
||||
# Also clean up associated idea embeddings
|
||||
self.conn.execute(
|
||||
"""DELETE FROM idea_embeddings WHERE idea_id IN
|
||||
(SELECT id FROM ideas WHERE novelty_score IS NOT NULL AND novelty_score < ?)""",
|
||||
(min_score,),
|
||||
)
|
||||
cursor = self.conn.execute(
|
||||
"DELETE FROM ideas WHERE novelty_score IS NOT NULL AND novelty_score < ?",
|
||||
(min_score,),
|
||||
)
|
||||
self.conn.commit()
|
||||
return cursor.rowcount
|
||||
|
||||
def idea_score_distribution(self) -> dict[int, int]:
|
||||
"""Return {score: count} for scored ideas."""
|
||||
rows = self.conn.execute(
|
||||
"SELECT novelty_score, COUNT(*) as cnt FROM ideas "
|
||||
"WHERE novelty_score IS NOT NULL GROUP BY novelty_score ORDER BY novelty_score"
|
||||
).fetchall()
|
||||
return {r["novelty_score"]: r["cnt"] for r in rows}
|
||||
|
||||
def ideas_below_score(self, min_score: int) -> list[dict]:
|
||||
"""Return ideas with novelty_score below min_score."""
|
||||
rows = self.conn.execute(
|
||||
"""SELECT i.id, i.draft_name, i.title, i.description, i.novelty_score,
|
||||
d.title AS draft_title
|
||||
FROM ideas i JOIN drafts d ON i.draft_name = d.name
|
||||
WHERE i.novelty_score IS NOT NULL AND i.novelty_score < ?
|
||||
ORDER BY i.novelty_score, i.title""",
|
||||
(min_score,),
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
# --- Idea Embeddings ---
|
||||
|
||||
def store_idea_embedding(self, idea_id: int, model: str, vector: np.ndarray) -> None:
|
||||
self.conn.execute(
|
||||
"""INSERT INTO idea_embeddings (idea_id, model, vector, created_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(idea_id) DO UPDATE SET
|
||||
model=excluded.model, vector=excluded.vector, created_at=excluded.created_at
|
||||
""",
|
||||
(idea_id, model, vector.astype(np.float32).tobytes(),
|
||||
datetime.now(timezone.utc).isoformat()),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def all_idea_embeddings(self) -> dict[int, np.ndarray]:
|
||||
rows = self.conn.execute("SELECT idea_id, vector FROM idea_embeddings").fetchall()
|
||||
return {
|
||||
r["idea_id"]: np.frombuffer(r["vector"], dtype=np.float32)
|
||||
for r in rows
|
||||
}
|
||||
|
||||
def ideas_without_embeddings(self, limit: int = 500) -> list[dict]:
|
||||
rows = self.conn.execute(
|
||||
"""SELECT i.id, i.title, i.description, i.idea_type, i.draft_name
|
||||
FROM ideas i
|
||||
LEFT JOIN idea_embeddings ie ON i.id = ie.idea_id
|
||||
WHERE ie.idea_id IS NULL
|
||||
LIMIT ?""",
|
||||
(limit,),
|
||||
).fetchall()
|
||||
return [{"id": r["id"], "title": r["title"], "description": r["description"],
|
||||
"type": r["idea_type"], "draft_name": r["draft_name"]} for r in rows]
|
||||
|
||||
# --- Gaps ---
|
||||
|
||||
def insert_gaps(self, gaps: list[dict]) -> None:
|
||||
@@ -981,6 +1136,250 @@ class Database:
|
||||
for r in rows
|
||||
]
|
||||
|
||||
# --- Working Groups ---
|
||||
|
||||
def wg_summary(self) -> list[dict]:
|
||||
"""Return per-WG summary: group, draft_count, avg scores, categories, idea_count.
|
||||
|
||||
Excludes 'none' (individual submissions) — those are returned separately.
|
||||
"""
|
||||
rows = self.conn.execute("""
|
||||
SELECT d."group" as wg, COUNT(*) as draft_count,
|
||||
AVG(r.novelty) as avg_novelty, AVG(r.maturity) as avg_maturity,
|
||||
AVG(r.overlap) as avg_overlap, AVG(r.momentum) as avg_momentum,
|
||||
AVG(r.relevance) as avg_relevance,
|
||||
(SELECT COUNT(*) FROM ideas i WHERE i.draft_name IN
|
||||
(SELECT name FROM drafts WHERE "group" = d."group")) as idea_count
|
||||
FROM drafts d
|
||||
LEFT JOIN ratings r ON d.name = r.draft_name
|
||||
WHERE d."group" IS NOT NULL AND d."group" != '' AND d."group" != 'none'
|
||||
GROUP BY d."group"
|
||||
ORDER BY draft_count DESC
|
||||
""").fetchall()
|
||||
|
||||
# Build categories per WG from a separate query
|
||||
cat_rows = self.conn.execute("""
|
||||
SELECT d."group" as wg, r.categories
|
||||
FROM drafts d JOIN ratings r ON d.name = r.draft_name
|
||||
WHERE d."group" IS NOT NULL AND d."group" != '' AND d."group" != 'none'
|
||||
""").fetchall()
|
||||
wg_cats: dict[str, dict[str, int]] = {}
|
||||
for cr in cat_rows:
|
||||
wg = cr["wg"]
|
||||
if wg not in wg_cats:
|
||||
wg_cats[wg] = {}
|
||||
try:
|
||||
for c in json.loads(cr["categories"]):
|
||||
c = normalize_category(c)
|
||||
wg_cats[wg][c] = wg_cats[wg].get(c, 0) + 1
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
|
||||
results = []
|
||||
for r in rows:
|
||||
results.append({
|
||||
"wg": r["wg"],
|
||||
"draft_count": r["draft_count"],
|
||||
"avg_novelty": round(r["avg_novelty"] or 0, 1),
|
||||
"avg_maturity": round(r["avg_maturity"] or 0, 1),
|
||||
"avg_overlap": round(r["avg_overlap"] or 0, 1),
|
||||
"avg_momentum": round(r["avg_momentum"] or 0, 1),
|
||||
"avg_relevance": round(r["avg_relevance"] or 0, 1),
|
||||
"categories": wg_cats.get(r["wg"], {}),
|
||||
"idea_count": r["idea_count"],
|
||||
})
|
||||
return results
|
||||
|
||||
def wg_drafts(self, wg: str) -> list[Draft]:
|
||||
"""Return all drafts for a specific working group."""
|
||||
rows = self.conn.execute(
|
||||
'SELECT * FROM drafts WHERE "group" = ? ORDER BY time DESC', (wg,)
|
||||
).fetchall()
|
||||
return [self._row_to_draft(r) for r in rows]
|
||||
|
||||
def wg_category_matrix(self) -> dict[str, dict[str, int]]:
|
||||
"""Return {wg: {category: count}} matrix for all WGs (excluding 'none')."""
|
||||
rows = self.conn.execute("""
|
||||
SELECT d."group" as wg, r.categories
|
||||
FROM drafts d
|
||||
JOIN ratings r ON d.name = r.draft_name
|
||||
WHERE d."group" IS NOT NULL AND d."group" != '' AND d."group" != 'none'
|
||||
""").fetchall()
|
||||
matrix: dict[str, dict[str, int]] = {}
|
||||
for r in rows:
|
||||
wg = r["wg"]
|
||||
if wg not in matrix:
|
||||
matrix[wg] = {}
|
||||
try:
|
||||
for c in json.loads(r["categories"]):
|
||||
c = normalize_category(c)
|
||||
matrix[wg][c] = matrix[wg].get(c, 0) + 1
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
return matrix
|
||||
|
||||
def wg_idea_overlap(self) -> list[dict]:
|
||||
"""Find ideas that appear across multiple WGs — signals for alignment.
|
||||
|
||||
Returns list of {idea_title, wgs: [{wg, draft_name, draft_title}], wg_count}.
|
||||
"""
|
||||
rows = self.conn.execute("""
|
||||
SELECT i.title as idea_title, i.description, d."group" as wg,
|
||||
d.name as draft_name, d.title as draft_title
|
||||
FROM ideas i
|
||||
JOIN drafts d ON i.draft_name = d.name
|
||||
WHERE d."group" IS NOT NULL AND d."group" != ''
|
||||
ORDER BY i.title, d."group"
|
||||
""").fetchall()
|
||||
|
||||
# Group by idea title
|
||||
from collections import defaultdict
|
||||
idea_groups: dict[str, list[dict]] = defaultdict(list)
|
||||
for r in rows:
|
||||
idea_groups[r["idea_title"]].append({
|
||||
"wg": r["wg"],
|
||||
"draft_name": r["draft_name"],
|
||||
"draft_title": r["draft_title"],
|
||||
})
|
||||
|
||||
# Only keep ideas spanning 2+ distinct WGs
|
||||
results = []
|
||||
for title, entries in idea_groups.items():
|
||||
wgs = set(e["wg"] for e in entries)
|
||||
if len(wgs) >= 2:
|
||||
results.append({
|
||||
"idea_title": title,
|
||||
"wgs": entries,
|
||||
"wg_count": len(wgs),
|
||||
"wg_names": sorted(wgs),
|
||||
})
|
||||
return sorted(results, key=lambda x: x["wg_count"], reverse=True)
|
||||
|
||||
def individual_vs_wg_categories(self) -> dict[str, dict[str, int]]:
|
||||
"""Compare category distribution: individual submissions vs WG-adopted.
|
||||
|
||||
Returns {"individual": {cat: count}, "wg_adopted": {cat: count}}.
|
||||
"""
|
||||
rows = self.conn.execute("""
|
||||
SELECT CASE WHEN d."group" = 'none' OR d."group" IS NULL THEN 'individual'
|
||||
ELSE 'wg_adopted' END as stream,
|
||||
r.categories
|
||||
FROM drafts d
|
||||
JOIN ratings r ON d.name = r.draft_name
|
||||
""").fetchall()
|
||||
result: dict[str, dict[str, int]] = {"individual": {}, "wg_adopted": {}}
|
||||
for r in rows:
|
||||
stream = r["stream"]
|
||||
try:
|
||||
for c in json.loads(r["categories"]):
|
||||
c = normalize_category(c)
|
||||
result[stream][c] = result[stream].get(c, 0) + 1
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
return result
|
||||
|
||||
def category_wg_spread(self) -> list[dict]:
|
||||
"""For each category, which WGs contribute drafts? High spread = alignment opportunity.
|
||||
|
||||
Returns [{category, wgs: [{wg, count}], wg_count, total_drafts}].
|
||||
"""
|
||||
rows = self.conn.execute("""
|
||||
SELECT d."group" as wg, r.categories
|
||||
FROM drafts d
|
||||
JOIN ratings r ON d.name = r.draft_name
|
||||
WHERE d."group" IS NOT NULL AND d."group" != ''
|
||||
""").fetchall()
|
||||
|
||||
from collections import defaultdict
|
||||
cat_wgs: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
|
||||
for r in rows:
|
||||
wg = r["wg"]
|
||||
try:
|
||||
for c in json.loads(r["categories"]):
|
||||
c = normalize_category(c)
|
||||
cat_wgs[c][wg] += 1
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
|
||||
results = []
|
||||
for cat, wg_counts in cat_wgs.items():
|
||||
wg_list = sorted(wg_counts.items(), key=lambda x: x[1], reverse=True)
|
||||
results.append({
|
||||
"category": cat,
|
||||
"wgs": [{"wg": wg, "count": cnt} for wg, cnt in wg_list],
|
||||
"wg_count": len(wg_list),
|
||||
"total_drafts": sum(wg_counts.values()),
|
||||
})
|
||||
return sorted(results, key=lambda x: x["wg_count"], reverse=True)
|
||||
|
||||
# --- Monitor Runs ---
|
||||
|
||||
def start_monitor_run(self) -> int:
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
cur = self.conn.execute(
|
||||
"INSERT INTO monitor_runs (started_at, status) VALUES (?, 'running')",
|
||||
(now,),
|
||||
)
|
||||
self.conn.commit()
|
||||
return cur.lastrowid
|
||||
|
||||
def complete_monitor_run(self, run_id: int, stats: dict) -> None:
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
started = self.conn.execute(
|
||||
"SELECT started_at FROM monitor_runs WHERE id = ?", (run_id,)
|
||||
).fetchone()
|
||||
duration = 0.0
|
||||
if started:
|
||||
try:
|
||||
start_dt = datetime.fromisoformat(started["started_at"])
|
||||
duration = (datetime.now(timezone.utc) - start_dt).total_seconds()
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
self.conn.execute(
|
||||
"""UPDATE monitor_runs SET
|
||||
status='completed', completed_at=?,
|
||||
new_drafts_found=?, drafts_analyzed=?,
|
||||
drafts_embedded=?, ideas_extracted=?,
|
||||
duration_seconds=?
|
||||
WHERE id=?""",
|
||||
(now, stats.get("new_drafts_found", 0), stats.get("drafts_analyzed", 0),
|
||||
stats.get("drafts_embedded", 0), stats.get("ideas_extracted", 0),
|
||||
duration, run_id),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def fail_monitor_run(self, run_id: int, error: str) -> None:
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
started = self.conn.execute(
|
||||
"SELECT started_at FROM monitor_runs WHERE id = ?", (run_id,)
|
||||
).fetchone()
|
||||
duration = 0.0
|
||||
if started:
|
||||
try:
|
||||
start_dt = datetime.fromisoformat(started["started_at"])
|
||||
duration = (datetime.now(timezone.utc) - start_dt).total_seconds()
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
self.conn.execute(
|
||||
"""UPDATE monitor_runs SET
|
||||
status='failed', completed_at=?, error_message=?, duration_seconds=?
|
||||
WHERE id=?""",
|
||||
(now, error, duration, run_id),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def get_monitor_runs(self, limit: int = 20) -> list[dict]:
|
||||
rows = self.conn.execute(
|
||||
"SELECT * FROM monitor_runs ORDER BY started_at DESC LIMIT ?", (limit,)
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
def get_last_successful_run(self) -> dict | None:
|
||||
row = self.conn.execute(
|
||||
"SELECT * FROM monitor_runs WHERE status='completed' ORDER BY started_at DESC LIMIT 1"
|
||||
).fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
# --- Helpers ---
|
||||
|
||||
@staticmethod
|
||||
|
||||
@@ -1673,3 +1673,143 @@ class Reporter:
|
||||
path = self.output_dir / "co-occurrence.md"
|
||||
path.write_text(report)
|
||||
return str(path)
|
||||
|
||||
def wg_report(self) -> str:
|
||||
"""Generate working group analysis report."""
|
||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||
summaries = self.db.wg_summary()
|
||||
spread = self.db.category_wg_spread()
|
||||
idea_overlaps = self.db.wg_idea_overlap()
|
||||
indiv_vs_wg = self.db.individual_vs_wg_categories()
|
||||
total = self.db.count_drafts()
|
||||
|
||||
indiv_count = self.db.conn.execute(
|
||||
'SELECT COUNT(*) FROM drafts WHERE "group" = \'none\' OR "group" IS NULL'
|
||||
).fetchone()[0]
|
||||
wg_count = total - indiv_count
|
||||
|
||||
lines = [
|
||||
f"# Working Group Analysis",
|
||||
f"*Generated {now} — {total} drafts ({wg_count} WG-adopted, {indiv_count} individual)*\n",
|
||||
]
|
||||
|
||||
# WG summary table
|
||||
lines.extend([
|
||||
"## Working Group Overview\n",
|
||||
"| WG | Drafts | Ideas | Novelty | Maturity | Overlap | Momentum | Relevance |",
|
||||
"|:---|-------:|------:|--------:|---------:|--------:|---------:|----------:|",
|
||||
])
|
||||
for s in summaries:
|
||||
lines.append(
|
||||
f"| **{s['wg']}** | {s['draft_count']} | {s['idea_count']} "
|
||||
f"| {s['avg_novelty']} | {s['avg_maturity']} | {s['avg_overlap']} "
|
||||
f"| {s['avg_momentum']} | {s['avg_relevance']} |"
|
||||
)
|
||||
|
||||
# Category spread — where topics live across WGs
|
||||
multi_wg = [s for s in spread if s["wg_count"] >= 2
|
||||
and not all(w["wg"] == "none" for w in s["wgs"])]
|
||||
if multi_wg:
|
||||
lines.extend([
|
||||
"\n## Cross-WG Category Spread\n",
|
||||
"Categories appearing in multiple WGs — potential coordination or alignment needed.\n",
|
||||
"| Category | WG Count | Total Drafts | WGs |",
|
||||
"|:---------|:--------:|-------------:|:----|",
|
||||
])
|
||||
for s in multi_wg:
|
||||
real_wgs = [f"{w['wg']}({w['count']})" for w in s["wgs"] if w["wg"] != "none"]
|
||||
lines.append(
|
||||
f"| {s['category']} | {s['wg_count']} | {s['total_drafts']} "
|
||||
f"| {', '.join(real_wgs)} |"
|
||||
)
|
||||
|
||||
# Idea overlap across WGs
|
||||
cross_wg_ideas = [o for o in idea_overlaps
|
||||
if not all(w == "none" for w in o["wg_names"])]
|
||||
if cross_wg_ideas:
|
||||
lines.extend([
|
||||
"\n## Cross-WG Idea Overlap\n",
|
||||
"Same technical ideas appearing in different WGs — strongest signals for alignment.\n",
|
||||
])
|
||||
for o in cross_wg_ideas[:30]:
|
||||
real_wgs = [w for w in o["wg_names"] if w != "none"]
|
||||
lines.append(f"### {o['idea_title']} ({len(real_wgs)} WGs: {', '.join(real_wgs)})\n")
|
||||
for entry in o["wgs"]:
|
||||
if entry["wg"] != "none":
|
||||
lines.append(f"- **[{entry['wg']}]** [{entry['draft_name']}]"
|
||||
f"(https://datatracker.ietf.org/doc/{entry['draft_name']}/) — "
|
||||
f"{entry['draft_title']}")
|
||||
lines.append("")
|
||||
|
||||
# Individual vs WG comparison
|
||||
indiv = indiv_vs_wg["individual"]
|
||||
adopted = indiv_vs_wg["wg_adopted"]
|
||||
all_cats = sorted(set(list(indiv.keys()) + list(adopted.keys())))
|
||||
|
||||
lines.extend([
|
||||
"\n## Individual vs WG-Adopted Distribution\n",
|
||||
"| Category | Individual | WG-Adopted | Assessment |",
|
||||
"|:---------|----------:|-----------:|:-----------|",
|
||||
])
|
||||
consolidation_candidates = []
|
||||
for cat in all_cats:
|
||||
i_count = indiv.get(cat, 0)
|
||||
w_count = adopted.get(cat, 0)
|
||||
if i_count >= 5 and w_count == 0:
|
||||
assessment = "**Needs WG** — high individual activity, no WG home"
|
||||
consolidation_candidates.append((cat, i_count))
|
||||
elif i_count >= 3 and w_count >= 1:
|
||||
assessment = "WG exists — individual drafts could target it"
|
||||
elif w_count > i_count and i_count > 0:
|
||||
assessment = "WG leading"
|
||||
elif w_count == 0 and i_count > 0:
|
||||
assessment = "Individual only"
|
||||
else:
|
||||
assessment = "-"
|
||||
lines.append(f"| {cat} | {i_count} | {w_count} | {assessment} |")
|
||||
|
||||
# Consolidation recommendations
|
||||
if consolidation_candidates:
|
||||
lines.extend([
|
||||
"\n## Consolidation Candidates\n",
|
||||
"Categories with significant individual draft activity but no WG — "
|
||||
"candidates for new WG charter or BoF.\n",
|
||||
])
|
||||
for cat, count in sorted(consolidation_candidates, key=lambda x: x[1], reverse=True):
|
||||
lines.append(f"### {cat} ({count} individual drafts)\n")
|
||||
rows = self.db.conn.execute("""
|
||||
SELECT d.name, d.title, r.summary FROM drafts d
|
||||
JOIN ratings r ON d.name = r.draft_name
|
||||
WHERE (d."group" = 'none' OR d."group" IS NULL)
|
||||
AND r.categories LIKE ?
|
||||
ORDER BY (r.novelty * 0.30 + r.relevance * 0.25 + r.maturity * 0.20
|
||||
+ r.momentum * 0.15 + (6 - r.overlap) * 0.10) DESC
|
||||
LIMIT 8
|
||||
""", (f"%{cat}%",)).fetchall()
|
||||
for row in rows:
|
||||
lines.append(
|
||||
f"- [{row['name']}](https://datatracker.ietf.org/doc/{row['name']}/) — "
|
||||
f"{row['title']}"
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
# Submission targets
|
||||
lines.extend([
|
||||
"\n## Recommended Submission Targets\n",
|
||||
"For each category, the best WG to submit new work to.\n",
|
||||
"| Category | Best WG | Alternatives |",
|
||||
"|:---------|:--------|:-------------|",
|
||||
])
|
||||
for s in spread:
|
||||
real_wgs = [w for w in s["wgs"] if w["wg"] != "none"]
|
||||
if not real_wgs:
|
||||
lines.append(f"| {s['category']} | *Individual submission* | - |")
|
||||
else:
|
||||
best = real_wgs[0]["wg"]
|
||||
alts = ", ".join(f"{w['wg']}({w['count']})" for w in real_wgs[1:3]) or "-"
|
||||
lines.append(f"| {s['category']} | **{best}** | {alts} |")
|
||||
|
||||
report = "\n".join(lines)
|
||||
path = self.output_dir / "wg-analysis.md"
|
||||
path.write_text(report)
|
||||
return str(path)
|
||||
|
||||
80
src/webui/PLAN.md
Normal file
80
src/webui/PLAN.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# IETF Draft Analyzer — Web Dashboard Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
A read-only Flask dashboard for exploring and visualizing 361+ IETF Internet-Drafts on AI/agent topics. All data comes from the existing SQLite database (`data/drafts.db`) via the `Database` class from `src/ietf_analyzer/db.py`.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Backend**: Flask (simple routes, no blueprints)
|
||||
- **Database**: Existing SQLite via `ietf_analyzer.db.Database` (read-only)
|
||||
- **CSS**: Tailwind CSS via CDN (dark theme: slate/gray palette)
|
||||
- **Charts**: Plotly.js via CDN (all interactive charts rendered client-side)
|
||||
- **Fonts**: Inter via Google Fonts CDN
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
src/webui/
|
||||
__init__.py # Empty package init
|
||||
app.py # Flask app, all routes
|
||||
data.py # Data access layer (wraps Database queries, returns JSON-ready dicts)
|
||||
templates/
|
||||
base.html # Dark-themed base with sidebar nav, Tailwind, Plotly CDN
|
||||
overview.html # Dashboard home: key stats, charts
|
||||
drafts.html # Draft explorer: search, filter, sortable table
|
||||
draft_detail.html # Single draft detail page
|
||||
ideas.html # Ideas explorer with type breakdown
|
||||
gaps.html # Gap analysis display
|
||||
ratings.html # Rating distributions and comparisons
|
||||
landscape.html # UMAP/t-SNE scatter (embeddings)
|
||||
authors.html # Author network and top contributors
|
||||
about.html # About page with project info
|
||||
```
|
||||
|
||||
## Pages & Routes
|
||||
|
||||
| Route | Template | Description |
|
||||
|-------|----------|-------------|
|
||||
| `/` | `overview.html` | Dashboard home: total drafts, rated count, author count, idea count, gap count. Charts: category treemap, timeline, score distribution histogram. |
|
||||
| `/drafts` | `drafts.html` | Searchable, filterable, sortable table of all drafts with ratings. Pagination. Category chip filters. Score range slider. |
|
||||
| `/drafts/<name>` | `draft_detail.html` | Single draft: all rating dimensions with notes, categories, authors, ideas extracted, references. |
|
||||
| `/ideas` | `ideas.html` | All extracted ideas grouped by type. Bar chart of idea types. Searchable. |
|
||||
| `/gaps` | `gaps.html` | Gap analysis results: severity badges, categories, evidence. |
|
||||
| `/ratings` | `ratings.html` | Rating analytics: dimension distributions (violin/box), category radar profiles, top-scored drafts. |
|
||||
| `/landscape` | `landscape.html` | Embedding scatter plot (pre-computed coordinates served as JSON). |
|
||||
| `/authors` | `authors.html` | Top authors table, org contributions bar chart, co-author network graph. |
|
||||
| `/about` | `about.html` | Project description, data freshness, counts. |
|
||||
|
||||
## Data Layer (`data.py`)
|
||||
|
||||
Thin wrapper around `Database` that returns plain dicts/lists ready for `jsonify()` or template rendering:
|
||||
|
||||
- `get_overview_stats()` — counts for drafts, ratings, authors, ideas, gaps
|
||||
- `get_drafts_page(page, per_page, search, category, min_score, sort)` — paginated draft list with ratings
|
||||
- `get_draft_detail(name)` — single draft + rating + authors + ideas + refs
|
||||
- `get_category_counts()` — {category: count} for filter chips
|
||||
- `get_rating_distributions()` — arrays for each dimension for Plotly
|
||||
- `get_timeline_data()` — monthly counts by category for stacked area
|
||||
- `get_ideas_by_type()` — grouped idea counts
|
||||
- `get_all_gaps()` — gap list with severity
|
||||
- `get_top_authors(limit)` — author leaderboard
|
||||
- `get_org_data(limit)` — organization contributions
|
||||
- `get_landscape_coords()` — pre-computed 2D coordinates + metadata
|
||||
|
||||
## Design System
|
||||
|
||||
- **Dark theme**: `bg-slate-900` body, `bg-slate-800` cards, `bg-slate-700` hover states
|
||||
- **Accent**: Blue-500 (`#3b82f6`) for links, active states, charts
|
||||
- **Text**: `text-slate-100` primary, `text-slate-400` secondary
|
||||
- **Cards**: Rounded corners (`rounded-xl`), subtle border (`border-slate-700`)
|
||||
- **Sidebar**: Fixed left, 240px wide, collapsible on mobile
|
||||
- **Charts**: Plotly dark theme (`plotly_dark` template), consistent color palette
|
||||
|
||||
## Key Decisions
|
||||
|
||||
1. **Read-only**: No writes to DB. All data comes from CLI pipeline runs.
|
||||
2. **Server-side rendering**: Templates with Jinja2, chart data passed as JSON.
|
||||
3. **No build step**: All CSS/JS from CDN. Zero npm/webpack complexity.
|
||||
4. **Reuse existing queries**: `data.py` calls `Database` methods directly.
|
||||
5. **Responsive**: Tailwind responsive utilities, sidebar collapses to hamburger.
|
||||
1
src/webui/__init__.py
Normal file
1
src/webui/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
# IETF Draft Analyzer — Web Dashboard
|
||||
297
src/webui/app.py
Normal file
297
src/webui/app.py
Normal file
@@ -0,0 +1,297 @@
|
||||
"""IETF Draft Analyzer — Web Dashboard.
|
||||
|
||||
Run with: python src/webui/app.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Ensure project src is on path
|
||||
_project_root = Path(__file__).resolve().parent.parent.parent
|
||||
sys.path.insert(0, str(_project_root / "src"))
|
||||
|
||||
from flask import Flask, render_template, request, jsonify, abort, g
|
||||
|
||||
from webui.data import (
|
||||
get_db,
|
||||
get_overview_stats,
|
||||
get_category_counts,
|
||||
get_drafts_page,
|
||||
get_draft_detail,
|
||||
get_rating_distributions,
|
||||
get_timeline_data,
|
||||
get_ideas_by_type,
|
||||
get_all_gaps,
|
||||
get_gap_detail,
|
||||
get_generated_drafts,
|
||||
read_generated_draft,
|
||||
get_top_authors,
|
||||
get_org_data,
|
||||
get_category_radar_data,
|
||||
get_score_histogram,
|
||||
get_coauthor_network,
|
||||
get_cross_org_data,
|
||||
get_landscape_tsne,
|
||||
get_similarity_graph,
|
||||
get_timeline_animation_data,
|
||||
get_idea_clusters,
|
||||
get_monitor_status,
|
||||
get_author_network_full,
|
||||
)
|
||||
|
||||
app = Flask(
|
||||
__name__,
|
||||
template_folder=str(Path(__file__).parent / "templates"),
|
||||
)
|
||||
app.config["SECRET_KEY"] = "ietf-dashboard-dev"
|
||||
|
||||
|
||||
# --- Database lifecycle (per-request to avoid SQLite threading issues) ---
|
||||
|
||||
|
||||
def db():
|
||||
if "db" not in g:
|
||||
g.db = get_db()
|
||||
return g.db
|
||||
|
||||
|
||||
# --- Routes ---
|
||||
|
||||
|
||||
@app.route("/")
|
||||
def overview():
|
||||
stats = get_overview_stats(db())
|
||||
categories = get_category_counts(db())
|
||||
timeline = get_timeline_data(db())
|
||||
scores = get_score_histogram(db())
|
||||
radar = get_category_radar_data(db())
|
||||
return render_template(
|
||||
"overview.html",
|
||||
stats=stats,
|
||||
categories=categories,
|
||||
timeline=timeline,
|
||||
scores=scores,
|
||||
radar=radar,
|
||||
)
|
||||
|
||||
|
||||
@app.route("/drafts")
|
||||
def drafts():
|
||||
page = request.args.get("page", 1, type=int)
|
||||
search = request.args.get("q", "")
|
||||
category = request.args.get("cat", "")
|
||||
min_score = request.args.get("min_score", 0.0, type=float)
|
||||
sort = request.args.get("sort", "score")
|
||||
sort_dir = request.args.get("dir", "desc")
|
||||
|
||||
result = get_drafts_page(
|
||||
db(),
|
||||
page=page,
|
||||
search=search,
|
||||
category=category,
|
||||
min_score=min_score,
|
||||
sort=sort,
|
||||
sort_dir=sort_dir,
|
||||
)
|
||||
categories = get_category_counts(db())
|
||||
return render_template(
|
||||
"drafts.html",
|
||||
result=result,
|
||||
categories=categories,
|
||||
search=search,
|
||||
current_cat=category,
|
||||
min_score=min_score,
|
||||
sort=sort,
|
||||
sort_dir=sort_dir,
|
||||
)
|
||||
|
||||
|
||||
@app.route("/drafts/<path:name>")
|
||||
def draft_detail(name: str):
|
||||
detail = get_draft_detail(db(), name)
|
||||
if not detail:
|
||||
abort(404)
|
||||
return render_template("draft_detail.html", draft=detail)
|
||||
|
||||
|
||||
@app.route("/ideas")
|
||||
def ideas():
|
||||
data = get_ideas_by_type(db())
|
||||
return render_template("ideas.html", data=data)
|
||||
|
||||
|
||||
@app.route("/gaps")
|
||||
def gaps():
|
||||
gap_list = get_all_gaps(db())
|
||||
generated = get_generated_drafts()
|
||||
return render_template("gaps.html", gaps=gap_list, generated_drafts=generated)
|
||||
|
||||
|
||||
@app.route("/gaps/demo")
|
||||
def gaps_demo():
|
||||
"""Show a pre-generated example draft so users can see output without API calls."""
|
||||
generated = get_generated_drafts()
|
||||
# Default to the first generated draft, or allow selection via query param
|
||||
selected = request.args.get("file", "")
|
||||
draft_text = None
|
||||
draft_info = None
|
||||
if selected:
|
||||
draft_text = read_generated_draft(selected)
|
||||
for g in generated:
|
||||
if g["filename"] == selected:
|
||||
draft_info = g
|
||||
break
|
||||
elif generated:
|
||||
draft_info = generated[0]
|
||||
draft_text = read_generated_draft(draft_info["filename"])
|
||||
return render_template(
|
||||
"gap_demo.html",
|
||||
generated_drafts=generated,
|
||||
draft_text=draft_text,
|
||||
draft_info=draft_info,
|
||||
selected=selected,
|
||||
)
|
||||
|
||||
|
||||
@app.route("/gaps/<int:gap_id>")
|
||||
def gap_detail(gap_id: int):
|
||||
gap = get_gap_detail(db(), gap_id)
|
||||
if not gap:
|
||||
abort(404)
|
||||
generated = get_generated_drafts()
|
||||
return render_template("gap_detail.html", gap=gap, generated_drafts=generated)
|
||||
|
||||
|
||||
@app.route("/gaps/<int:gap_id>/generate", methods=["POST"])
|
||||
def gap_generate(gap_id: int):
|
||||
"""Trigger draft generation for a gap. Returns JSON with the generated text."""
|
||||
gap = get_gap_detail(db(), gap_id)
|
||||
if not gap:
|
||||
return jsonify({"error": "Gap not found"}), 404
|
||||
|
||||
try:
|
||||
from ietf_analyzer.config import Config
|
||||
from ietf_analyzer.analyzer import Analyzer
|
||||
from ietf_analyzer.draftgen import DraftGenerator
|
||||
|
||||
cfg = Config.load()
|
||||
database = db()
|
||||
analyzer = Analyzer(cfg, database)
|
||||
generator = DraftGenerator(cfg, database, analyzer)
|
||||
|
||||
# Generate into a file named after the gap
|
||||
slug = gap["topic"].lower().replace(" ", "-")[:40]
|
||||
output_path = str(Path(_project_root) / "data" / "reports" / "generated-drafts" / f"draft-gap-{gap_id}-{slug}.txt")
|
||||
path = generator.generate(gap["topic"], output_path=output_path)
|
||||
draft_text = Path(path).read_text(errors="replace")
|
||||
|
||||
return jsonify({
|
||||
"success": True,
|
||||
"text": draft_text,
|
||||
"filename": Path(path).name,
|
||||
"path": path,
|
||||
})
|
||||
except Exception as e:
|
||||
return jsonify({"error": str(e)}), 500
|
||||
|
||||
|
||||
@app.route("/ratings")
|
||||
def ratings():
|
||||
distributions = get_rating_distributions(db())
|
||||
radar = get_category_radar_data(db())
|
||||
return render_template(
|
||||
"ratings.html",
|
||||
dist=distributions,
|
||||
radar=radar,
|
||||
)
|
||||
|
||||
|
||||
@app.route("/landscape")
|
||||
def landscape():
|
||||
distributions = get_rating_distributions(db())
|
||||
tsne_data = get_landscape_tsne(db())
|
||||
return render_template(
|
||||
"landscape.html",
|
||||
dist=distributions,
|
||||
tsne_data=tsne_data,
|
||||
)
|
||||
|
||||
|
||||
@app.route("/timeline")
|
||||
def timeline_animation():
|
||||
data = get_timeline_animation_data(db())
|
||||
return render_template("timeline.html", animation=data)
|
||||
|
||||
|
||||
@app.route("/idea-clusters")
|
||||
def idea_clusters():
|
||||
data = get_idea_clusters(db())
|
||||
return render_template("idea_clusters.html", clusters=data)
|
||||
|
||||
|
||||
@app.route("/similarity")
|
||||
def similarity():
|
||||
network = get_similarity_graph(db())
|
||||
return render_template("similarity.html", network=network)
|
||||
|
||||
|
||||
@app.route("/authors")
|
||||
def authors():
|
||||
top = get_top_authors(db(), limit=50)
|
||||
orgs = get_org_data(db(), limit=20)
|
||||
network = get_author_network_full(db())
|
||||
cross_org = get_cross_org_data(db(), limit=20)
|
||||
return render_template(
|
||||
"authors.html",
|
||||
authors=top,
|
||||
orgs=orgs,
|
||||
orgs_data=orgs,
|
||||
network=network,
|
||||
cross_org=cross_org,
|
||||
)
|
||||
|
||||
|
||||
@app.route("/monitor")
|
||||
def monitor_page():
|
||||
status = get_monitor_status(db())
|
||||
return render_template("monitor.html", status=status)
|
||||
|
||||
|
||||
@app.route("/about")
|
||||
def about():
|
||||
stats = get_overview_stats(db())
|
||||
return render_template("about.html", stats=stats)
|
||||
|
||||
|
||||
# --- API endpoints for AJAX (used by client-side charts) ---
|
||||
|
||||
|
||||
@app.route("/api/drafts")
|
||||
def api_drafts():
|
||||
page = request.args.get("page", 1, type=int)
|
||||
search = request.args.get("q", "")
|
||||
category = request.args.get("cat", "")
|
||||
min_score = request.args.get("min_score", 0.0, type=float)
|
||||
sort = request.args.get("sort", "score")
|
||||
sort_dir = request.args.get("dir", "desc")
|
||||
return jsonify(
|
||||
get_drafts_page(db(), page=page, search=search, category=category,
|
||||
min_score=min_score, sort=sort, sort_dir=sort_dir)
|
||||
)
|
||||
|
||||
|
||||
@app.route("/api/stats")
|
||||
def api_stats():
|
||||
return jsonify(get_overview_stats(db()))
|
||||
|
||||
|
||||
@app.route("/api/authors/network")
|
||||
def api_author_network():
|
||||
return jsonify(get_author_network_full(db()))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print("Starting IETF Draft Analyzer Dashboard on http://127.0.0.1:5000")
|
||||
app.run(debug=True, host="127.0.0.1", port=5000)
|
||||
767
src/webui/data.py
Normal file
767
src/webui/data.py
Normal file
@@ -0,0 +1,767 @@
|
||||
"""Data access layer for the web dashboard.
|
||||
|
||||
Thin wrapper around ietf_analyzer.db.Database that returns plain dicts
|
||||
ready for JSON serialization or Jinja2 template rendering.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from collections import Counter, defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
# Add project root to path so we can import ietf_analyzer
|
||||
_project_root = Path(__file__).resolve().parent.parent.parent
|
||||
if str(_project_root) not in sys.path:
|
||||
sys.path.insert(0, str(_project_root / "src"))
|
||||
|
||||
from ietf_analyzer.config import Config
|
||||
from ietf_analyzer.db import Database
|
||||
|
||||
|
||||
def get_db() -> Database:
|
||||
"""Get a Database instance using default config."""
|
||||
config = Config.load()
|
||||
return Database(config)
|
||||
|
||||
|
||||
def get_overview_stats(db: Database) -> dict:
|
||||
"""Return high-level stats for the dashboard home page."""
|
||||
total_drafts = db.count_drafts()
|
||||
rated_pairs = db.drafts_with_ratings(limit=1000)
|
||||
rated_count = len(rated_pairs)
|
||||
author_count = db.author_count()
|
||||
idea_count = db.idea_count()
|
||||
gaps = db.all_gaps()
|
||||
input_tok, output_tok = db.total_tokens_used()
|
||||
|
||||
return {
|
||||
"total_drafts": total_drafts,
|
||||
"rated_count": rated_count,
|
||||
"author_count": author_count,
|
||||
"idea_count": idea_count,
|
||||
"gap_count": len(gaps),
|
||||
"input_tokens": input_tok,
|
||||
"output_tokens": output_tok,
|
||||
}
|
||||
|
||||
|
||||
def get_category_counts(db: Database) -> dict[str, int]:
|
||||
"""Return {category: draft_count} for all categories."""
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
counts: dict[str, int] = Counter()
|
||||
for _, rating in pairs:
|
||||
for cat in rating.categories:
|
||||
counts[cat] += 1
|
||||
return dict(counts.most_common())
|
||||
|
||||
|
||||
def get_drafts_page(
|
||||
db: Database,
|
||||
page: int = 1,
|
||||
per_page: int = 50,
|
||||
search: str = "",
|
||||
category: str = "",
|
||||
min_score: float = 0.0,
|
||||
sort: str = "score",
|
||||
sort_dir: str = "desc",
|
||||
) -> dict:
|
||||
"""Return a paginated, filtered list of drafts with ratings.
|
||||
|
||||
Returns dict with keys: drafts, total, page, per_page, pages.
|
||||
"""
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
|
||||
# Filter
|
||||
filtered = []
|
||||
for draft, rating in pairs:
|
||||
if min_score > 0 and rating.composite_score < min_score:
|
||||
continue
|
||||
if category and category not in rating.categories:
|
||||
continue
|
||||
if search:
|
||||
haystack = f"{draft.name} {draft.title} {rating.summary}".lower()
|
||||
if not all(w in haystack for w in search.lower().split()):
|
||||
continue
|
||||
filtered.append((draft, rating))
|
||||
|
||||
# Sort
|
||||
sort_keys = {
|
||||
"score": lambda p: p[1].composite_score,
|
||||
"name": lambda p: p[0].name,
|
||||
"date": lambda p: p[0].time or "",
|
||||
"novelty": lambda p: p[1].novelty,
|
||||
"maturity": lambda p: p[1].maturity,
|
||||
"relevance": lambda p: p[1].relevance,
|
||||
"overlap": lambda p: p[1].overlap,
|
||||
"momentum": lambda p: p[1].momentum,
|
||||
}
|
||||
key_fn = sort_keys.get(sort, sort_keys["score"])
|
||||
reverse = sort_dir == "desc"
|
||||
filtered.sort(key=key_fn, reverse=reverse)
|
||||
|
||||
total = len(filtered)
|
||||
pages = max(1, (total + per_page - 1) // per_page)
|
||||
page = max(1, min(page, pages))
|
||||
start = (page - 1) * per_page
|
||||
page_items = filtered[start : start + per_page]
|
||||
|
||||
drafts = []
|
||||
for draft, rating in page_items:
|
||||
drafts.append({
|
||||
"name": draft.name,
|
||||
"title": draft.title,
|
||||
"date": draft.date,
|
||||
"url": draft.datatracker_url,
|
||||
"pages": draft.pages or 0,
|
||||
"group": draft.group or "individual",
|
||||
"score": round(rating.composite_score, 2),
|
||||
"novelty": rating.novelty,
|
||||
"maturity": rating.maturity,
|
||||
"overlap": rating.overlap,
|
||||
"momentum": rating.momentum,
|
||||
"relevance": rating.relevance,
|
||||
"categories": rating.categories,
|
||||
"summary": rating.summary,
|
||||
})
|
||||
|
||||
return {
|
||||
"drafts": drafts,
|
||||
"total": total,
|
||||
"page": page,
|
||||
"per_page": per_page,
|
||||
"pages": pages,
|
||||
}
|
||||
|
||||
|
||||
def get_draft_detail(db: Database, name: str) -> dict | None:
|
||||
"""Return full detail for a single draft."""
|
||||
draft = db.get_draft(name)
|
||||
if not draft:
|
||||
return None
|
||||
|
||||
rating = db.get_rating(name)
|
||||
authors = db.get_authors_for_draft(name)
|
||||
ideas = db.get_ideas_for_draft(name)
|
||||
refs = db.get_refs_for_draft(name)
|
||||
|
||||
result = {
|
||||
"name": draft.name,
|
||||
"title": draft.title,
|
||||
"rev": draft.rev,
|
||||
"abstract": draft.abstract,
|
||||
"date": draft.date,
|
||||
"time": draft.time,
|
||||
"url": draft.datatracker_url,
|
||||
"text_url": draft.text_url,
|
||||
"pages": draft.pages,
|
||||
"words": draft.words,
|
||||
"group": draft.group or "individual",
|
||||
"categories": draft.categories,
|
||||
"tags": draft.tags,
|
||||
"authors": [
|
||||
{"name": a.name, "affiliation": a.affiliation, "person_id": a.person_id}
|
||||
for a in authors
|
||||
],
|
||||
"ideas": ideas,
|
||||
"refs": [{"type": t, "id": rid} for t, rid in refs],
|
||||
}
|
||||
|
||||
if rating:
|
||||
result["rating"] = {
|
||||
"score": round(rating.composite_score, 2),
|
||||
"novelty": rating.novelty,
|
||||
"maturity": rating.maturity,
|
||||
"overlap": rating.overlap,
|
||||
"momentum": rating.momentum,
|
||||
"relevance": rating.relevance,
|
||||
"summary": rating.summary,
|
||||
"novelty_note": rating.novelty_note,
|
||||
"maturity_note": rating.maturity_note,
|
||||
"overlap_note": rating.overlap_note,
|
||||
"momentum_note": rating.momentum_note,
|
||||
"relevance_note": rating.relevance_note,
|
||||
"categories": rating.categories,
|
||||
}
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def get_rating_distributions(db: Database) -> dict:
|
||||
"""Return arrays for each rating dimension, suitable for Plotly."""
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
dims = {
|
||||
"novelty": [],
|
||||
"maturity": [],
|
||||
"overlap": [],
|
||||
"momentum": [],
|
||||
"relevance": [],
|
||||
"scores": [],
|
||||
"categories": [],
|
||||
"names": [],
|
||||
}
|
||||
for draft, rating in pairs:
|
||||
dims["novelty"].append(rating.novelty)
|
||||
dims["maturity"].append(rating.maturity)
|
||||
dims["overlap"].append(rating.overlap)
|
||||
dims["momentum"].append(rating.momentum)
|
||||
dims["relevance"].append(rating.relevance)
|
||||
dims["scores"].append(round(rating.composite_score, 2))
|
||||
dims["categories"].append(rating.categories[0] if rating.categories else "Other")
|
||||
dims["names"].append(draft.name)
|
||||
return dims
|
||||
|
||||
|
||||
def get_timeline_data(db: Database) -> dict:
|
||||
"""Return monthly counts by category for timeline chart."""
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
all_drafts = db.list_drafts(limit=1000, order_by="time ASC")
|
||||
rating_map = {d.name: r for d, r in pairs}
|
||||
|
||||
month_cat: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
|
||||
for d in all_drafts:
|
||||
month = d.time[:7] if d.time else "unknown"
|
||||
r = rating_map.get(d.name)
|
||||
if r:
|
||||
cat = r.categories[0] if r.categories else "Other"
|
||||
month_cat[month][cat] += 1
|
||||
|
||||
months = sorted(month_cat.keys())
|
||||
cat_totals: Counter = Counter()
|
||||
for mc in month_cat.values():
|
||||
for c, cnt in mc.items():
|
||||
cat_totals[c] += cnt
|
||||
top_cats = [c for c, _ in cat_totals.most_common(10)]
|
||||
|
||||
series = {}
|
||||
for cat in top_cats:
|
||||
series[cat] = [month_cat[m].get(cat, 0) for m in months]
|
||||
|
||||
return {"months": months, "series": series, "categories": top_cats}
|
||||
|
||||
|
||||
def get_ideas_by_type(db: Database) -> dict:
|
||||
"""Return ideas grouped by type with counts."""
|
||||
all_ideas = db.all_ideas()
|
||||
type_counts = Counter(i.get("type", "other") or "other" for i in all_ideas)
|
||||
return {
|
||||
"total": len(all_ideas),
|
||||
"by_type": dict(type_counts.most_common()),
|
||||
"ideas": all_ideas,
|
||||
}
|
||||
|
||||
|
||||
def get_all_gaps(db: Database) -> list[dict]:
|
||||
"""Return all gap analysis results."""
|
||||
return db.all_gaps()
|
||||
|
||||
|
||||
def get_gap_detail(db: Database, gap_id: int) -> dict | None:
|
||||
"""Return a single gap by ID, or None if not found."""
|
||||
gaps = db.all_gaps()
|
||||
for g in gaps:
|
||||
if g["id"] == gap_id:
|
||||
return g
|
||||
return None
|
||||
|
||||
|
||||
def get_generated_drafts() -> list[dict]:
|
||||
"""Return list of pre-generated draft files in data/reports/generated-drafts/."""
|
||||
drafts_dir = _project_root / "data" / "reports" / "generated-drafts"
|
||||
if not drafts_dir.exists():
|
||||
return []
|
||||
results = []
|
||||
for f in sorted(drafts_dir.glob("draft-*.txt")):
|
||||
# Extract title from first non-empty content line after header
|
||||
title = f.stem
|
||||
text = f.read_text(errors="replace")
|
||||
for line in text.splitlines():
|
||||
stripped = line.strip()
|
||||
if stripped and not stripped.startswith("Internet-Draft") and \
|
||||
not stripped.startswith("Intended status") and \
|
||||
not stripped.startswith("Expires:") and stripped != "":
|
||||
title = stripped
|
||||
break
|
||||
results.append({
|
||||
"filename": f.name,
|
||||
"stem": f.stem,
|
||||
"title": title,
|
||||
"size": f.stat().st_size,
|
||||
"path": str(f),
|
||||
})
|
||||
return results
|
||||
|
||||
|
||||
def read_generated_draft(filename: str) -> str | None:
|
||||
"""Read a generated draft file by filename. Returns text or None."""
|
||||
drafts_dir = _project_root / "data" / "reports" / "generated-drafts"
|
||||
path = drafts_dir / filename
|
||||
if not path.exists() or not path.is_file():
|
||||
return None
|
||||
# Safety: ensure we're not reading outside the directory
|
||||
if not str(path.resolve()).startswith(str(drafts_dir.resolve())):
|
||||
return None
|
||||
return path.read_text(errors="replace")
|
||||
|
||||
|
||||
def get_top_authors(db: Database, limit: int = 30) -> list[dict]:
|
||||
"""Return top authors by draft count."""
|
||||
rows = db.top_authors(limit=limit)
|
||||
return [
|
||||
{"name": name, "affiliation": aff, "draft_count": cnt, "drafts": drafts}
|
||||
for name, aff, cnt, drafts in rows
|
||||
]
|
||||
|
||||
|
||||
def get_org_data(db: Database, limit: int = 20) -> list[dict]:
|
||||
"""Return organization contribution data."""
|
||||
rows = db.top_orgs(limit=limit)
|
||||
return [
|
||||
{"org": org, "author_count": authors, "draft_count": drafts}
|
||||
for org, authors, drafts in rows
|
||||
]
|
||||
|
||||
|
||||
def get_category_radar_data(db: Database) -> dict:
|
||||
"""Return average rating profiles per category for radar chart."""
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
cat_ratings: dict[str, list] = defaultdict(list)
|
||||
for _, r in pairs:
|
||||
for c in r.categories:
|
||||
cat_ratings[c].append(r)
|
||||
|
||||
top_cats = sorted(cat_ratings.keys(), key=lambda c: len(cat_ratings[c]), reverse=True)[:8]
|
||||
result = {}
|
||||
for cat in top_cats:
|
||||
ratings = cat_ratings[cat]
|
||||
n = len(ratings)
|
||||
result[cat] = {
|
||||
"count": n,
|
||||
"novelty": round(sum(r.novelty for r in ratings) / n, 2),
|
||||
"maturity": round(sum(r.maturity for r in ratings) / n, 2),
|
||||
"relevance": round(sum(r.relevance for r in ratings) / n, 2),
|
||||
"momentum": round(sum(r.momentum for r in ratings) / n, 2),
|
||||
"low_overlap": round(sum(6 - r.overlap for r in ratings) / n, 2),
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
def get_score_histogram(db: Database) -> list[float]:
|
||||
"""Return list of composite scores for histogram."""
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
return [round(r.composite_score, 2) for _, r in pairs]
|
||||
|
||||
|
||||
def get_coauthor_network(db: Database, min_shared: int = 1) -> dict:
|
||||
"""Return co-authorship network data for force-directed graph.
|
||||
|
||||
Returns {nodes: [{id, name, org, draft_count}], edges: [{source, target, weight}]}
|
||||
"""
|
||||
pairs = db.coauthor_pairs()
|
||||
top = db.top_authors(limit=100)
|
||||
|
||||
# Build node set from authors who have co-authorships
|
||||
author_info = {name: {"org": aff, "draft_count": cnt} for name, aff, cnt, _ in top}
|
||||
node_set = set()
|
||||
edges = []
|
||||
for a, b, shared in pairs:
|
||||
if shared >= min_shared:
|
||||
node_set.add(a)
|
||||
node_set.add(b)
|
||||
edges.append({"source": a, "target": b, "weight": shared})
|
||||
|
||||
nodes = []
|
||||
for name in node_set:
|
||||
info = author_info.get(name, {"org": "", "draft_count": 1})
|
||||
nodes.append({
|
||||
"id": name,
|
||||
"name": name,
|
||||
"org": info["org"],
|
||||
"draft_count": info["draft_count"],
|
||||
})
|
||||
|
||||
return {"nodes": nodes, "edges": edges}
|
||||
|
||||
|
||||
def get_similarity_graph(db: Database, threshold: float = 0.75) -> dict:
|
||||
"""Return draft similarity network for force-directed graph.
|
||||
|
||||
Returns {nodes: [{name, title, category, score}],
|
||||
edges: [{source, target, similarity}],
|
||||
stats: {node_count, edge_count, avg_similarity}}
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
embeddings = db.all_embeddings()
|
||||
if len(embeddings) < 2:
|
||||
return {"nodes": [], "edges": [], "stats": {"node_count": 0, "edge_count": 0, "avg_similarity": 0}}
|
||||
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
rating_map = {d.name: r for d, r in pairs}
|
||||
draft_map = {d.name: d for d, _ in pairs}
|
||||
|
||||
# Filter to drafts with both embeddings and ratings
|
||||
names = [n for n in embeddings if n in rating_map]
|
||||
if len(names) < 2:
|
||||
return {"nodes": [], "edges": [], "stats": {"node_count": 0, "edge_count": 0, "avg_similarity": 0}}
|
||||
|
||||
matrix = np.array([embeddings[n] for n in names])
|
||||
|
||||
# L2-normalize and compute cosine similarity
|
||||
norms = np.linalg.norm(matrix, axis=1, keepdims=True)
|
||||
norms[norms == 0] = 1.0
|
||||
normalized = matrix / norms
|
||||
sim_matrix = normalized @ normalized.T
|
||||
|
||||
# Find pairs above threshold (upper triangle only)
|
||||
edges = []
|
||||
node_set = set()
|
||||
for i in range(len(names)):
|
||||
for j in range(i + 1, len(names)):
|
||||
sim = float(sim_matrix[i, j])
|
||||
if sim >= threshold:
|
||||
edges.append({"source": names[i], "target": names[j], "similarity": round(sim, 4)})
|
||||
node_set.add(names[i])
|
||||
node_set.add(names[j])
|
||||
|
||||
# Build nodes from connected drafts only
|
||||
nodes = []
|
||||
for name in names:
|
||||
if name not in node_set:
|
||||
continue
|
||||
r = rating_map[name]
|
||||
d = draft_map.get(name)
|
||||
nodes.append({
|
||||
"name": name,
|
||||
"title": d.title if d else name,
|
||||
"category": r.categories[0] if r.categories else "Other",
|
||||
"score": round(r.composite_score, 2),
|
||||
})
|
||||
|
||||
avg_sim = round(sum(e["similarity"] for e in edges) / max(len(edges), 1), 4)
|
||||
|
||||
return {
|
||||
"nodes": nodes,
|
||||
"edges": edges,
|
||||
"stats": {"node_count": len(nodes), "edge_count": len(edges), "avg_similarity": avg_sim},
|
||||
}
|
||||
|
||||
|
||||
def get_cross_org_data(db: Database, limit: int = 20) -> list[dict]:
|
||||
"""Return cross-org collaboration pairs."""
|
||||
rows = db.cross_org_collaborations(limit=limit)
|
||||
return [
|
||||
{"org_a": a, "org_b": b, "shared_drafts": cnt}
|
||||
for a, b, cnt in rows
|
||||
]
|
||||
|
||||
|
||||
def get_author_network_full(db: Database) -> dict:
|
||||
"""Return enriched co-authorship network with avg scores and cluster info.
|
||||
|
||||
Returns {
|
||||
nodes: [{id, name, org, draft_count, avg_score, drafts: [name,...]}],
|
||||
edges: [{source, target, weight}],
|
||||
clusters: [{id, members: [name,...], org_mix: {org: count}, size}],
|
||||
}
|
||||
"""
|
||||
pairs = db.coauthor_pairs()
|
||||
top = db.top_authors(limit=500)
|
||||
|
||||
# Build rating lookup for avg scores
|
||||
rated = db.drafts_with_ratings(limit=2000)
|
||||
draft_score = {d.name: r.composite_score for d, r in rated}
|
||||
|
||||
# Author info map
|
||||
author_info = {}
|
||||
for name, aff, cnt, drafts in top:
|
||||
scores = [draft_score[dn] for dn in drafts if dn in draft_score]
|
||||
avg = round(sum(scores) / len(scores), 2) if scores else 0
|
||||
author_info[name] = {
|
||||
"org": aff, "draft_count": cnt, "drafts": drafts, "avg_score": avg
|
||||
}
|
||||
|
||||
# Build node set: authors with 2+ drafts OR 1+ co-authorship
|
||||
node_set = set()
|
||||
edges = []
|
||||
for a, b, shared in pairs:
|
||||
if shared >= 1:
|
||||
node_set.add(a)
|
||||
node_set.add(b)
|
||||
edges.append({"source": a, "target": b, "weight": shared})
|
||||
|
||||
# Also include authors with 2+ drafts even if no co-authorships
|
||||
for name, info in author_info.items():
|
||||
if info["draft_count"] >= 2:
|
||||
node_set.add(name)
|
||||
|
||||
nodes = []
|
||||
for name in node_set:
|
||||
info = author_info.get(name, {"org": "", "draft_count": 1, "drafts": [], "avg_score": 0})
|
||||
nodes.append({
|
||||
"id": name,
|
||||
"name": name,
|
||||
"org": info["org"],
|
||||
"draft_count": info["draft_count"],
|
||||
"avg_score": info["avg_score"],
|
||||
"drafts": info["drafts"][:8], # cap for JSON size
|
||||
})
|
||||
|
||||
# Cluster detection via connected components (BFS)
|
||||
adjacency: dict[str, set[str]] = defaultdict(set)
|
||||
for e in edges:
|
||||
adjacency[e["source"]].add(e["target"])
|
||||
adjacency[e["target"]].add(e["source"])
|
||||
|
||||
visited: set[str] = set()
|
||||
clusters = []
|
||||
|
||||
for node in sorted(node_set):
|
||||
if node in visited:
|
||||
continue
|
||||
component: list[str] = []
|
||||
queue = [node]
|
||||
while queue:
|
||||
current = queue.pop(0)
|
||||
if current in visited:
|
||||
continue
|
||||
visited.add(current)
|
||||
component.append(current)
|
||||
for neighbor in adjacency.get(current, []):
|
||||
if neighbor not in visited:
|
||||
queue.append(neighbor)
|
||||
|
||||
if len(component) >= 2:
|
||||
org_mix: dict[str, int] = Counter()
|
||||
for m in component:
|
||||
org = author_info.get(m, {}).get("org", "")
|
||||
if org:
|
||||
org_mix[org] += 1
|
||||
clusters.append({
|
||||
"id": len(clusters),
|
||||
"members": component,
|
||||
"org_mix": dict(org_mix.most_common()),
|
||||
"size": len(component),
|
||||
})
|
||||
|
||||
clusters.sort(key=lambda c: c["size"], reverse=True)
|
||||
|
||||
return {"nodes": nodes, "edges": edges, "clusters": clusters}
|
||||
|
||||
|
||||
def get_idea_clusters(db: Database) -> dict:
|
||||
"""Cluster ideas by embedding similarity, return clusters + t-SNE scatter."""
|
||||
import numpy as np
|
||||
|
||||
embeddings = db.all_idea_embeddings()
|
||||
if not embeddings:
|
||||
return {"clusters": [], "scatter": [], "stats": {"total": 0, "clustered": 0, "num_clusters": 0}, "empty": True}
|
||||
|
||||
# Fetch ideas with IDs for metadata lookup
|
||||
rows = db.conn.execute("SELECT id, title, description, idea_type, draft_name FROM ideas").fetchall()
|
||||
idea_map = {r["id"]: {"title": r["title"], "description": r["description"],
|
||||
"type": r["idea_type"], "draft_name": r["draft_name"]} for r in rows}
|
||||
|
||||
# Build matrix from embeddings that have matching ideas
|
||||
idea_ids = [iid for iid in embeddings if iid in idea_map]
|
||||
if len(idea_ids) < 5:
|
||||
return {"clusters": [], "scatter": [], "stats": {"total": len(idea_ids), "clustered": 0, "num_clusters": 0}, "empty": True}
|
||||
|
||||
matrix = np.array([embeddings[iid] for iid in idea_ids])
|
||||
|
||||
# Agglomerative clustering with cosine distance
|
||||
try:
|
||||
from sklearn.cluster import AgglomerativeClustering
|
||||
clustering = AgglomerativeClustering(
|
||||
n_clusters=None, distance_threshold=0.5,
|
||||
metric='cosine', linkage='average',
|
||||
)
|
||||
labels = clustering.fit_predict(matrix)
|
||||
except Exception:
|
||||
return {"clusters": [], "scatter": [], "stats": {"total": len(idea_ids), "clustered": 0, "num_clusters": 0}, "empty": True}
|
||||
|
||||
# Build cluster data
|
||||
cluster_ideas: dict[int, list] = defaultdict(list)
|
||||
for idx, iid in enumerate(idea_ids):
|
||||
cluster_ideas[labels[idx]].append(iid)
|
||||
|
||||
# Filter to clusters with 2+ ideas
|
||||
stop = {"a", "an", "the", "of", "for", "in", "to", "and", "or", "with", "on", "by", "is", "as", "at", "from", "that", "this", "it"}
|
||||
clusters = []
|
||||
for cid in sorted(cluster_ideas.keys()):
|
||||
members = cluster_ideas[cid]
|
||||
if len(members) < 2:
|
||||
continue
|
||||
ideas_in_cluster = [idea_map[iid] for iid in members if iid in idea_map]
|
||||
# Theme: most common significant words in titles
|
||||
words = Counter()
|
||||
for idea in ideas_in_cluster:
|
||||
for w in idea["title"].lower().split():
|
||||
w_clean = w.strip("()[].,;:-\"'")
|
||||
if len(w_clean) > 2 and w_clean not in stop:
|
||||
words[w_clean] += 1
|
||||
top_words = [w for w, _ in words.most_common(4)]
|
||||
theme = " ".join(top_words).title() if top_words else f"Cluster {cid}"
|
||||
|
||||
drafts = list({idea["draft_name"] for idea in ideas_in_cluster})
|
||||
clusters.append({
|
||||
"id": len(clusters),
|
||||
"theme": theme,
|
||||
"size": len(ideas_in_cluster),
|
||||
"ideas": ideas_in_cluster[:20],
|
||||
"drafts": drafts,
|
||||
})
|
||||
|
||||
# t-SNE for scatter
|
||||
scatter = []
|
||||
try:
|
||||
from sklearn.manifold import TSNE
|
||||
perp = min(30, len(idea_ids) - 1)
|
||||
tsne = TSNE(n_components=2, perplexity=perp, random_state=42, max_iter=500)
|
||||
coords = tsne.fit_transform(matrix)
|
||||
for idx, iid in enumerate(idea_ids):
|
||||
info = idea_map.get(iid, {})
|
||||
scatter.append({
|
||||
"x": round(float(coords[idx, 0]), 3),
|
||||
"y": round(float(coords[idx, 1]), 3),
|
||||
"cluster_id": int(labels[idx]),
|
||||
"title": info.get("title", ""),
|
||||
"draft_name": info.get("draft_name", ""),
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
total = len(idea_ids)
|
||||
clustered = sum(c["size"] for c in clusters)
|
||||
return {
|
||||
"clusters": clusters,
|
||||
"scatter": scatter,
|
||||
"stats": {"total": total, "clustered": clustered, "num_clusters": len(clusters)},
|
||||
"empty": False,
|
||||
}
|
||||
|
||||
|
||||
def get_timeline_animation_data(db: Database) -> dict:
|
||||
"""Compute t-SNE on all drafts, return points with month info + category_monthly.
|
||||
|
||||
t-SNE is computed once on ALL drafts so coordinates are stable across
|
||||
animation frames. Each point carries a ``month`` field (YYYY-MM) so the
|
||||
front-end can build cumulative animation frames.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
embeddings = db.all_embeddings()
|
||||
if len(embeddings) < 5:
|
||||
return {"points": [], "months": [], "category_monthly": {}}
|
||||
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
rating_map = {d.name: r for d, r in pairs}
|
||||
draft_map = {d.name: d for d, _ in pairs}
|
||||
|
||||
# Filter to drafts that have both embeddings and ratings
|
||||
names = [n for n in embeddings if n in rating_map]
|
||||
if len(names) < 5:
|
||||
return {"points": [], "months": [], "category_monthly": {}}
|
||||
|
||||
matrix = np.array([embeddings[n] for n in names])
|
||||
|
||||
try:
|
||||
from sklearn.manifold import TSNE
|
||||
tsne = TSNE(n_components=2, perplexity=min(30, len(names) - 1),
|
||||
random_state=42, max_iter=500)
|
||||
coords = tsne.fit_transform(matrix)
|
||||
except Exception:
|
||||
return {"points": [], "months": [], "category_monthly": {}}
|
||||
|
||||
# Build points with month
|
||||
points = []
|
||||
month_set: set[str] = set()
|
||||
category_monthly: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
|
||||
|
||||
for i, name in enumerate(names):
|
||||
r = rating_map[name]
|
||||
d = draft_map.get(name)
|
||||
month = (d.time[:7] if d and d.time else "unknown")
|
||||
cat = r.categories[0] if r.categories else "Other"
|
||||
month_set.add(month)
|
||||
category_monthly[month][cat] += 1
|
||||
points.append({
|
||||
"name": name,
|
||||
"title": d.title if d else name,
|
||||
"x": round(float(coords[i, 0]), 3),
|
||||
"y": round(float(coords[i, 1]), 3),
|
||||
"category": cat,
|
||||
"score": round(r.composite_score, 2),
|
||||
"month": month,
|
||||
})
|
||||
|
||||
months = sorted(month_set)
|
||||
# Convert defaultdict to plain dict for JSON
|
||||
cat_monthly_plain = {m: dict(cats) for m, cats in category_monthly.items()}
|
||||
|
||||
return {
|
||||
"points": points,
|
||||
"months": months,
|
||||
"category_monthly": cat_monthly_plain,
|
||||
}
|
||||
|
||||
|
||||
def get_monitor_status(db: Database) -> dict:
|
||||
"""Return monitoring status data for dashboard."""
|
||||
runs = db.get_monitor_runs(limit=20)
|
||||
last = runs[0] if runs else None
|
||||
unrated = len(db.unrated_drafts(limit=9999))
|
||||
unembedded = len(db.drafts_without_embeddings(limit=9999))
|
||||
no_ideas = len(db.drafts_without_ideas(limit=9999))
|
||||
return {
|
||||
"last_run": last,
|
||||
"runs": runs,
|
||||
"unprocessed": {"unrated": unrated, "unembedded": unembedded, "no_ideas": no_ideas},
|
||||
"total_runs": len(runs),
|
||||
}
|
||||
|
||||
|
||||
def get_landscape_tsne(db: Database) -> list[dict]:
|
||||
"""Compute t-SNE from embeddings, return [{name, title, x, y, category, score}].
|
||||
|
||||
Uses cached coordinates if available, otherwise computes fresh.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
embeddings = db.all_embeddings()
|
||||
if len(embeddings) < 5:
|
||||
return []
|
||||
|
||||
pairs = db.drafts_with_ratings(limit=1000)
|
||||
rating_map = {d.name: r for d, r in pairs}
|
||||
draft_map = {d.name: d for d, _ in pairs}
|
||||
|
||||
# Filter to drafts that have both embeddings and ratings
|
||||
names = [n for n in embeddings if n in rating_map]
|
||||
if len(names) < 5:
|
||||
return []
|
||||
|
||||
matrix = np.array([embeddings[n] for n in names])
|
||||
|
||||
try:
|
||||
from sklearn.manifold import TSNE
|
||||
tsne = TSNE(n_components=2, perplexity=min(30, len(names) - 1),
|
||||
random_state=42, max_iter=500)
|
||||
coords = tsne.fit_transform(matrix)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
result = []
|
||||
for i, name in enumerate(names):
|
||||
r = rating_map[name]
|
||||
d = draft_map.get(name)
|
||||
result.append({
|
||||
"name": name,
|
||||
"title": d.title if d else name,
|
||||
"x": round(float(coords[i, 0]), 3),
|
||||
"y": round(float(coords[i, 1]), 3),
|
||||
"category": r.categories[0] if r.categories else "Other",
|
||||
"score": round(r.composite_score, 2),
|
||||
})
|
||||
return result
|
||||
65
src/webui/templates/about.html
Normal file
65
src/webui/templates/about.html
Normal file
@@ -0,0 +1,65 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "about" %}
|
||||
|
||||
{% block title %}About — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="max-w-3xl">
|
||||
<h1 class="text-2xl font-bold text-white mb-6">About IETF Draft Analyzer</h1>
|
||||
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6 mb-6">
|
||||
<h2 class="text-lg font-semibold text-white mb-3">What is this?</h2>
|
||||
<p class="text-sm text-slate-400 leading-relaxed mb-4">
|
||||
A tool for tracking, categorizing, rating, and mapping IETF Internet-Drafts
|
||||
focused on AI and agent-related topics. It uses Claude for analysis and rating,
|
||||
Ollama for embeddings, and SQLite for storage.
|
||||
</p>
|
||||
<p class="text-sm text-slate-400 leading-relaxed">
|
||||
The dashboard provides interactive visualizations of the draft landscape,
|
||||
including category breakdowns, rating distributions, author networks,
|
||||
extracted ideas, and gap analysis.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6 mb-6">
|
||||
<h2 class="text-lg font-semibold text-white mb-3">Current Data</h2>
|
||||
<div class="grid grid-cols-2 gap-4 text-sm">
|
||||
<div>
|
||||
<div class="text-slate-500">Total Drafts</div>
|
||||
<div class="text-xl font-bold text-blue-400">{{ stats.total_drafts }}</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-slate-500">Rated Drafts</div>
|
||||
<div class="text-xl font-bold text-green-400">{{ stats.rated_count }}</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-slate-500">Authors Tracked</div>
|
||||
<div class="text-xl font-bold text-purple-400">{{ stats.author_count }}</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-slate-500">Ideas Extracted</div>
|
||||
<div class="text-xl font-bold text-amber-400">{{ stats.idea_count }}</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-slate-500">Gaps Identified</div>
|
||||
<div class="text-xl font-bold text-red-400">{{ stats.gap_count }}</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-slate-500">API Tokens Used</div>
|
||||
<div class="text-xl font-bold text-slate-300">{{ "{:,}".format(stats.input_tokens + stats.output_tokens) }}</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6">
|
||||
<h2 class="text-lg font-semibold text-white mb-3">Tech Stack</h2>
|
||||
<ul class="text-sm text-slate-400 space-y-2">
|
||||
<li><span class="text-slate-200 font-medium">Analysis:</span> Claude (Sonnet for analysis, Haiku for bulk)</li>
|
||||
<li><span class="text-slate-200 font-medium">Embeddings:</span> Ollama (nomic-embed-text)</li>
|
||||
<li><span class="text-slate-200 font-medium">Storage:</span> SQLite with FTS5 full-text search</li>
|
||||
<li><span class="text-slate-200 font-medium">Dashboard:</span> Flask, Tailwind CSS, Plotly.js</li>
|
||||
<li><span class="text-slate-200 font-medium">Data source:</span> IETF Datatracker API</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
598
src/webui/templates/authors.html
Normal file
598
src/webui/templates/authors.html
Normal file
@@ -0,0 +1,598 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "authors" %}
|
||||
|
||||
{% block title %}Author Network — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block extra_head %}
|
||||
<script src="https://d3js.org/d3.v7.min.js"></script>
|
||||
<style>
|
||||
#networkSvg {
|
||||
width: 100%;
|
||||
height: 600px;
|
||||
cursor: grab;
|
||||
}
|
||||
#networkSvg:active { cursor: grabbing; }
|
||||
#networkSvg .node { cursor: pointer; }
|
||||
#networkSvg .node circle { stroke: rgba(255,255,255,0.15); stroke-width: 1.5px; transition: r 0.2s; }
|
||||
#networkSvg .node:hover circle { stroke: #60a5fa; stroke-width: 2.5px; }
|
||||
#networkSvg .node text { pointer-events: none; }
|
||||
#networkSvg .link { stroke-opacity: 0.25; }
|
||||
#networkSvg .link:hover { stroke-opacity: 0.7; }
|
||||
.tooltip-card {
|
||||
position: absolute; pointer-events: none; z-index: 50;
|
||||
background: #1e293b; border: 1px solid #334155; border-radius: 8px;
|
||||
padding: 10px 14px; font-size: 12px; color: #e2e8f0;
|
||||
box-shadow: 0 8px 24px rgba(0,0,0,0.4);
|
||||
max-width: 280px; opacity: 0; transition: opacity 0.15s;
|
||||
}
|
||||
.tooltip-card.visible { opacity: 1; }
|
||||
.legend-swatch { width: 12px; height: 12px; border-radius: 3px; display: inline-block; }
|
||||
.cluster-card { transition: all 0.2s; }
|
||||
.cluster-card:hover { border-color: #3b82f6 !important; }
|
||||
.filter-btn { transition: all 0.15s; }
|
||||
.filter-btn:hover { background: rgba(59, 130, 246, 0.2); }
|
||||
.filter-btn.active { background: rgba(59, 130, 246, 0.3); border-color: #3b82f6; color: #60a5fa; }
|
||||
</style>
|
||||
{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Author Network</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Interactive collaboration graph of {{ network.nodes | length }} authors across {{ orgs | length }} organizations</p>
|
||||
</div>
|
||||
|
||||
<!-- Summary stats -->
|
||||
<div class="grid grid-cols-2 md:grid-cols-5 gap-4 mb-6">
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Authors Shown</div>
|
||||
<div class="text-2xl font-bold text-white mt-1">{{ network.nodes | length }}</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Organizations</div>
|
||||
<div class="text-2xl font-bold text-white mt-1">{{ orgs | length }}</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Co-Author Links</div>
|
||||
<div class="text-2xl font-bold text-white mt-1">{{ network.edges | length }}</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-amber-500 to-amber-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Clusters</div>
|
||||
<div class="text-2xl font-bold text-white mt-1">{{ network.clusters | length }}</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-rose-500 to-rose-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Multi-Draft</div>
|
||||
<div class="text-2xl font-bold text-white mt-1">{{ authors | selectattr('draft_count', 'gt', 1) | list | length }}</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- D3 Force-directed Network Graph -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6 relative">
|
||||
<div class="flex flex-wrap items-center justify-between gap-3 mb-3">
|
||||
<div>
|
||||
<h2 class="text-sm font-semibold text-slate-300">Co-Authorship Network</h2>
|
||||
<p class="text-xs text-slate-500 mt-0.5">Node size = draft count. Color = organization. Edge thickness = shared drafts. Drag nodes to rearrange. Scroll to zoom.</p>
|
||||
</div>
|
||||
<div class="flex gap-2 items-center">
|
||||
<button id="resetZoom" class="text-xs px-3 py-1.5 rounded-lg border border-slate-700 text-slate-400 hover:text-white hover:border-slate-500 transition">Reset View</button>
|
||||
<select id="highlightOrg" class="text-xs px-3 py-1.5 rounded-lg border border-slate-700 bg-slate-800 text-slate-300 focus:outline-none focus:border-blue-500">
|
||||
<option value="">All Organizations</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
<div class="relative">
|
||||
<svg id="networkSvg"></svg>
|
||||
<div id="tooltip" class="tooltip-card"></div>
|
||||
</div>
|
||||
<!-- Legend -->
|
||||
<div id="legend" class="flex flex-wrap gap-3 mt-3 pt-3 border-t border-slate-800"></div>
|
||||
</div>
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
|
||||
<!-- Organization bar chart -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Organizations by Draft Count</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Color intensity = number of authors from that org.</p>
|
||||
<div id="orgChart" style="height: 500px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Cross-org collaboration -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Cross-Organization Collaboration</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Organizations co-authoring drafts together.</p>
|
||||
<div id="crossOrgChart" style="height: 500px;"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Collaboration Clusters -->
|
||||
{% if network.clusters %}
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Collaboration Clusters</h2>
|
||||
<p class="text-xs text-slate-500 mb-4">Connected groups of authors who co-author drafts. Click a cluster to highlight it in the graph.</p>
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4" id="clusterGrid">
|
||||
{% for c in network.clusters[:12] %}
|
||||
<div class="cluster-card bg-slate-800/50 rounded-lg border border-slate-700/50 p-4 cursor-pointer" data-cluster-id="{{ c.id }}" onclick="highlightCluster({{ c.id }})">
|
||||
<div class="flex items-center justify-between mb-2">
|
||||
<span class="text-sm font-semibold text-white">Cluster #{{ c.id + 1 }}</span>
|
||||
<span class="text-xs px-2 py-0.5 rounded-full bg-blue-500/20 text-blue-400">{{ c.size }} authors</span>
|
||||
</div>
|
||||
<div class="flex flex-wrap gap-1 mb-2">
|
||||
{% for org, count in c.org_mix.items() %}
|
||||
<span class="text-xs px-2 py-0.5 rounded-full bg-slate-700 text-slate-300">{{ org }} ({{ count }})</span>
|
||||
{% endfor %}
|
||||
</div>
|
||||
<div class="text-xs text-slate-500 truncate" title="{{ c.members | join(', ') }}">
|
||||
{{ c.members[:5] | join(', ') }}{% if c.members | length > 5 %} +{{ c.members | length - 5 }} more{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<!-- Top Authors Table and Org Stats side by side -->
|
||||
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6 mb-6">
|
||||
<!-- Top authors table -->
|
||||
<div class="lg:col-span-2 bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<div class="p-4 border-b border-slate-800 flex items-center justify-between">
|
||||
<h2 class="text-sm font-semibold text-slate-300">Top Authors</h2>
|
||||
<span class="text-xs text-slate-500">Showing top {{ authors | length }}</span>
|
||||
</div>
|
||||
<div class="overflow-x-auto max-h-[600px] overflow-y-auto">
|
||||
<table class="w-full text-sm" id="authorsTable">
|
||||
<thead class="sticky top-0 z-10">
|
||||
<tr class="border-b border-slate-800 bg-slate-900">
|
||||
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="index">#</th>
|
||||
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="name">Author</th>
|
||||
<th class="px-4 py-2.5 text-left text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="org">Organization</th>
|
||||
<th class="px-4 py-2.5 text-right text-xs font-medium text-slate-400 cursor-pointer hover:text-slate-200" data-sort="drafts">Drafts</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody class="divide-y divide-slate-800/50" id="authorsBody">
|
||||
{% for a in authors %}
|
||||
<tr class="hover:bg-slate-800/50 transition author-row" data-name="{{ a.name }}" data-org="{{ a.affiliation }}" data-count="{{ a.draft_count }}">
|
||||
<td class="px-4 py-2.5 text-slate-500 text-xs">{{ loop.index }}</td>
|
||||
<td class="px-4 py-2.5">
|
||||
<a href="/drafts?q={{ a.name | urlencode }}" class="text-blue-400 hover:text-blue-300 transition">{{ a.name }}</a>
|
||||
</td>
|
||||
<td class="px-4 py-2.5 text-slate-500 text-xs truncate max-w-[200px]" title="{{ a.affiliation }}">{{ a.affiliation }}</td>
|
||||
<td class="px-4 py-2.5 text-right">
|
||||
<span class="px-2 py-0.5 rounded-full text-xs font-medium
|
||||
{% if a.draft_count >= 5 %}bg-green-500/20 text-green-400
|
||||
{% elif a.draft_count >= 3 %}bg-blue-500/20 text-blue-400
|
||||
{% else %}bg-slate-700/50 text-slate-400{% endif %}">
|
||||
{{ a.draft_count }}
|
||||
</span>
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Organization stats cards -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<div class="p-4 border-b border-slate-800">
|
||||
<h2 class="text-sm font-semibold text-slate-300">Organization Stats</h2>
|
||||
</div>
|
||||
<div class="max-h-[600px] overflow-y-auto divide-y divide-slate-800/50">
|
||||
{% for o in orgs %}
|
||||
<div class="px-4 py-3 hover:bg-slate-800/30 transition cursor-pointer org-card" data-org="{{ o.org }}" onclick="filterByOrg('{{ o.org | e }}')">
|
||||
<div class="flex items-center justify-between mb-1">
|
||||
<span class="text-sm text-slate-200 font-medium truncate max-w-[180px]" title="{{ o.org }}">{{ o.org }}</span>
|
||||
<span class="text-xs px-2 py-0.5 rounded-full bg-blue-500/20 text-blue-400">{{ o.draft_count }} drafts</span>
|
||||
</div>
|
||||
<div class="flex items-center gap-3 text-xs text-slate-500">
|
||||
<span>{{ o.author_count }} author{{ 's' if o.author_count != 1 }}</span>
|
||||
<span class="text-slate-700">|</span>
|
||||
<span>{{ (o.draft_count / o.author_count) | round(1) }} drafts/author</span>
|
||||
</div>
|
||||
<div class="mt-1.5 w-full bg-slate-800 rounded-full h-1.5">
|
||||
<div class="bg-blue-500/60 h-1.5 rounded-full" style="width: {{ (o.draft_count / orgs[0].draft_count * 100) | round }}%"></div>
|
||||
</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
// --- Shared Plotly config ---
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'transparent',
|
||||
plot_bgcolor: 'transparent',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 20, r: 20, b: 40, l: 50 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const CFG = { responsive: true, displayModeBar: false };
|
||||
|
||||
const PALETTE = [
|
||||
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
|
||||
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
|
||||
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
|
||||
'#fb923c', '#34d399', '#c084fc', '#38bdf8', '#fbbf24',
|
||||
];
|
||||
|
||||
// --- Data from server ---
|
||||
const network = {{ network | tojson }};
|
||||
|
||||
// ===========================================================
|
||||
// D3.js Force-Directed Co-Authorship Network
|
||||
// ===========================================================
|
||||
(function() {
|
||||
if (network.nodes.length === 0) {
|
||||
document.getElementById('networkSvg').outerHTML =
|
||||
'<p class="text-slate-500 text-sm text-center py-20">No co-authorship data available</p>';
|
||||
return;
|
||||
}
|
||||
|
||||
const svg = d3.select('#networkSvg');
|
||||
const container = svg.node().parentElement;
|
||||
const width = container.clientWidth;
|
||||
const height = 600;
|
||||
svg.attr('viewBox', [0, 0, width, height]);
|
||||
|
||||
// Build org color map (top orgs by frequency)
|
||||
const orgCounts = {};
|
||||
network.nodes.forEach(n => {
|
||||
if (n.org) orgCounts[n.org] = (orgCounts[n.org] || 0) + 1;
|
||||
});
|
||||
const orgsSorted = Object.entries(orgCounts).sort((a,b) => b[1] - a[1]);
|
||||
const orgColor = {};
|
||||
orgsSorted.forEach(([org], i) => {
|
||||
orgColor[org] = i < PALETTE.length ? PALETTE[i] : '#475569';
|
||||
});
|
||||
|
||||
// Populate org dropdown
|
||||
const orgSelect = document.getElementById('highlightOrg');
|
||||
orgsSorted.slice(0, 30).forEach(([org, cnt]) => {
|
||||
const opt = document.createElement('option');
|
||||
opt.value = org;
|
||||
opt.textContent = `${org} (${cnt})`;
|
||||
orgSelect.appendChild(opt);
|
||||
});
|
||||
|
||||
// Populate legend
|
||||
const legendEl = document.getElementById('legend');
|
||||
orgsSorted.slice(0, 12).forEach(([org]) => {
|
||||
const item = document.createElement('div');
|
||||
item.className = 'flex items-center gap-1.5 text-xs text-slate-400';
|
||||
item.innerHTML = `<span class="legend-swatch" style="background:${orgColor[org]}"></span>${org}`;
|
||||
legendEl.appendChild(item);
|
||||
});
|
||||
|
||||
// Build cluster lookup
|
||||
const clusterOf = {};
|
||||
(network.clusters || []).forEach(c => {
|
||||
c.members.forEach(m => { clusterOf[m] = c.id; });
|
||||
});
|
||||
|
||||
// Prepare simulation data (deep copy to avoid mutating)
|
||||
const nodes = network.nodes.map(n => ({...n}));
|
||||
const links = network.edges.map(e => ({
|
||||
source: e.source,
|
||||
target: e.target,
|
||||
weight: e.weight,
|
||||
}));
|
||||
|
||||
// Size scale
|
||||
const maxDrafts = d3.max(nodes, n => n.draft_count) || 1;
|
||||
const rScale = d3.scaleSqrt().domain([1, maxDrafts]).range([4, 22]);
|
||||
|
||||
// Force simulation
|
||||
const simulation = d3.forceSimulation(nodes)
|
||||
.force('link', d3.forceLink(links)
|
||||
.id(d => d.id)
|
||||
.distance(d => 80 / Math.sqrt(d.weight))
|
||||
.strength(d => 0.3 * d.weight)
|
||||
)
|
||||
.force('charge', d3.forceManyBody().strength(-120).distanceMax(300))
|
||||
.force('center', d3.forceCenter(width / 2, height / 2))
|
||||
.force('collision', d3.forceCollide().radius(d => rScale(d.draft_count) + 3))
|
||||
.force('x', d3.forceX(width / 2).strength(0.03))
|
||||
.force('y', d3.forceY(height / 2).strength(0.03));
|
||||
|
||||
// Zoom behavior
|
||||
const g = svg.append('g');
|
||||
const zoom = d3.zoom()
|
||||
.scaleExtent([0.2, 5])
|
||||
.on('zoom', (event) => g.attr('transform', event.transform));
|
||||
svg.call(zoom);
|
||||
|
||||
document.getElementById('resetZoom').addEventListener('click', () => {
|
||||
svg.transition().duration(500).call(zoom.transform, d3.zoomIdentity);
|
||||
});
|
||||
|
||||
// Draw edges
|
||||
const linkGroup = g.append('g').attr('class', 'links');
|
||||
const link = linkGroup.selectAll('line')
|
||||
.data(links)
|
||||
.join('line')
|
||||
.attr('class', 'link')
|
||||
.attr('stroke', '#475569')
|
||||
.attr('stroke-width', d => Math.max(1, d.weight * 1.5));
|
||||
|
||||
// Draw nodes
|
||||
const nodeGroup = g.append('g').attr('class', 'nodes');
|
||||
const node = nodeGroup.selectAll('g')
|
||||
.data(nodes)
|
||||
.join('g')
|
||||
.attr('class', 'node')
|
||||
.call(d3.drag()
|
||||
.on('start', dragStarted)
|
||||
.on('drag', dragged)
|
||||
.on('end', dragEnded)
|
||||
);
|
||||
|
||||
node.append('circle')
|
||||
.attr('r', d => rScale(d.draft_count))
|
||||
.attr('fill', d => orgColor[d.org] || '#475569')
|
||||
.attr('opacity', 0.85);
|
||||
|
||||
// Labels for nodes with 3+ drafts
|
||||
node.filter(d => d.draft_count >= 3)
|
||||
.append('text')
|
||||
.text(d => {
|
||||
const parts = d.name.split(' ');
|
||||
return parts[parts.length - 1];
|
||||
})
|
||||
.attr('dy', d => -(rScale(d.draft_count) + 4))
|
||||
.attr('text-anchor', 'middle')
|
||||
.attr('fill', '#94a3b8')
|
||||
.attr('font-size', '9px')
|
||||
.attr('font-family', 'Inter, system-ui, sans-serif');
|
||||
|
||||
// Tooltip
|
||||
const tooltip = document.getElementById('tooltip');
|
||||
|
||||
node.on('mouseover', function(event, d) {
|
||||
const draftList = (d.drafts || []).slice(0, 5).map(dn => {
|
||||
const short = dn.replace(/^draft-/, '');
|
||||
return `<div class="truncate text-slate-400">${short}</div>`;
|
||||
}).join('');
|
||||
const moreCount = (d.drafts || []).length > 5 ? `<div class="text-slate-500 mt-1">+${d.drafts.length - 5} more</div>` : '';
|
||||
|
||||
tooltip.innerHTML = `
|
||||
<div class="font-semibold text-white mb-1">${d.name}</div>
|
||||
<div class="text-slate-400 text-xs mb-2">${d.org || 'Unknown org'}</div>
|
||||
<div class="flex gap-4 text-xs mb-2">
|
||||
<span><span class="text-blue-400 font-medium">${d.draft_count}</span> drafts</span>
|
||||
<span>avg <span class="text-emerald-400 font-medium">${d.avg_score}</span></span>
|
||||
</div>
|
||||
<div class="text-xs">${draftList}${moreCount}</div>
|
||||
`;
|
||||
tooltip.classList.add('visible');
|
||||
|
||||
// Highlight connected
|
||||
const connected = new Set();
|
||||
links.forEach(l => {
|
||||
const sid = typeof l.source === 'object' ? l.source.id : l.source;
|
||||
const tid = typeof l.target === 'object' ? l.target.id : l.target;
|
||||
if (sid === d.id) connected.add(tid);
|
||||
if (tid === d.id) connected.add(sid);
|
||||
});
|
||||
connected.add(d.id);
|
||||
|
||||
node.select('circle')
|
||||
.attr('opacity', n => connected.has(n.id) ? 1 : 0.15);
|
||||
node.selectAll('text')
|
||||
.attr('opacity', n => connected.has(n.id) ? 1 : 0.15);
|
||||
link
|
||||
.attr('stroke-opacity', l => {
|
||||
const sid = typeof l.source === 'object' ? l.source.id : l.source;
|
||||
const tid = typeof l.target === 'object' ? l.target.id : l.target;
|
||||
return (sid === d.id || tid === d.id) ? 0.7 : 0.03;
|
||||
});
|
||||
})
|
||||
.on('mousemove', function(event) {
|
||||
const rect = container.getBoundingClientRect();
|
||||
tooltip.style.left = (event.clientX - rect.left + 15) + 'px';
|
||||
tooltip.style.top = (event.clientY - rect.top - 10) + 'px';
|
||||
})
|
||||
.on('mouseout', function() {
|
||||
tooltip.classList.remove('visible');
|
||||
node.select('circle').attr('opacity', 0.85);
|
||||
node.selectAll('text').attr('opacity', 1);
|
||||
link.attr('stroke-opacity', 0.25);
|
||||
})
|
||||
.on('click', function(event, d) {
|
||||
// Navigate to drafts search for this author
|
||||
window.open(`/drafts?q=${encodeURIComponent(d.name)}`, '_blank');
|
||||
});
|
||||
|
||||
// Tick handler
|
||||
simulation.on('tick', () => {
|
||||
link
|
||||
.attr('x1', d => d.source.x)
|
||||
.attr('y1', d => d.source.y)
|
||||
.attr('x2', d => d.target.x)
|
||||
.attr('y2', d => d.target.y);
|
||||
node.attr('transform', d => `translate(${d.x},${d.y})`);
|
||||
});
|
||||
|
||||
// Drag handlers
|
||||
function dragStarted(event, d) {
|
||||
if (!event.active) simulation.alphaTarget(0.3).restart();
|
||||
d.fx = d.x; d.fy = d.y;
|
||||
}
|
||||
function dragged(event, d) {
|
||||
d.fx = event.x; d.fy = event.y;
|
||||
}
|
||||
function dragEnded(event, d) {
|
||||
if (!event.active) simulation.alphaTarget(0);
|
||||
d.fx = null; d.fy = null;
|
||||
}
|
||||
|
||||
// Org filter dropdown
|
||||
orgSelect.addEventListener('change', function() {
|
||||
const org = this.value;
|
||||
if (!org) {
|
||||
node.select('circle').attr('opacity', 0.85);
|
||||
node.selectAll('text').attr('opacity', 1);
|
||||
link.attr('stroke-opacity', 0.25);
|
||||
return;
|
||||
}
|
||||
const inOrg = new Set(nodes.filter(n => n.org === org).map(n => n.id));
|
||||
node.select('circle')
|
||||
.attr('opacity', n => inOrg.has(n.id) ? 1 : 0.08);
|
||||
node.selectAll('text')
|
||||
.attr('opacity', n => inOrg.has(n.id) ? 1 : 0.08);
|
||||
link.attr('stroke-opacity', l => {
|
||||
const sid = typeof l.source === 'object' ? l.source.id : l.source;
|
||||
const tid = typeof l.target === 'object' ? l.target.id : l.target;
|
||||
return (inOrg.has(sid) && inOrg.has(tid)) ? 0.6 : 0.02;
|
||||
});
|
||||
});
|
||||
|
||||
// Expose cluster highlighting globally
|
||||
window.highlightCluster = function(clusterId) {
|
||||
const cluster = (network.clusters || []).find(c => c.id === clusterId);
|
||||
if (!cluster) return;
|
||||
const members = new Set(cluster.members);
|
||||
|
||||
// Reset org dropdown
|
||||
orgSelect.value = '';
|
||||
|
||||
node.select('circle')
|
||||
.transition().duration(300)
|
||||
.attr('opacity', n => members.has(n.id) ? 1 : 0.08);
|
||||
node.selectAll('text')
|
||||
.transition().duration(300)
|
||||
.attr('opacity', n => members.has(n.id) ? 1 : 0.08);
|
||||
link.transition().duration(300)
|
||||
.attr('stroke-opacity', l => {
|
||||
const sid = typeof l.source === 'object' ? l.source.id : l.source;
|
||||
const tid = typeof l.target === 'object' ? l.target.id : l.target;
|
||||
return (members.has(sid) && members.has(tid)) ? 0.7 : 0.02;
|
||||
});
|
||||
|
||||
// Highlight cluster card
|
||||
document.querySelectorAll('.cluster-card').forEach(c => {
|
||||
c.classList.toggle('border-blue-500', c.dataset.clusterId == clusterId);
|
||||
});
|
||||
|
||||
// Zoom to fit cluster members
|
||||
const clusterNodes = nodes.filter(n => members.has(n.id));
|
||||
if (clusterNodes.length > 0) {
|
||||
const xs = clusterNodes.map(n => n.x);
|
||||
const ys = clusterNodes.map(n => n.y);
|
||||
const x0 = Math.min(...xs) - 50, x1 = Math.max(...xs) + 50;
|
||||
const y0 = Math.min(...ys) - 50, y1 = Math.max(...ys) + 50;
|
||||
const scale = Math.min(width / (x1 - x0), height / (y1 - y0), 3);
|
||||
const cx = (x0 + x1) / 2, cy = (y0 + y1) / 2;
|
||||
svg.transition().duration(500).call(
|
||||
zoom.transform,
|
||||
d3.zoomIdentity.translate(width/2, height/2).scale(scale).translate(-cx, -cy)
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
// Filter by org (called from org stats cards)
|
||||
window.filterByOrg = function(org) {
|
||||
orgSelect.value = org;
|
||||
orgSelect.dispatchEvent(new Event('change'));
|
||||
};
|
||||
})();
|
||||
|
||||
// ===========================================================
|
||||
// Organization Bar Chart (Plotly)
|
||||
// ===========================================================
|
||||
const orgsData = {{ orgs_data | tojson }};
|
||||
const orgNames = orgsData.map(o => o.org).reverse();
|
||||
const orgDrafts = orgsData.map(o => o.draft_count).reverse();
|
||||
const orgAuthors = orgsData.map(o => o.author_count).reverse();
|
||||
|
||||
Plotly.newPlot('orgChart', [{
|
||||
y: orgNames, x: orgDrafts,
|
||||
type: 'bar', orientation: 'h',
|
||||
marker: {
|
||||
color: orgAuthors,
|
||||
colorscale: [[0, '#1e3a5f'], [0.5, '#3b82f6'], [1, '#60a5fa']],
|
||||
showscale: true,
|
||||
colorbar: {
|
||||
title: { text: 'Authors', font: { color: '#94a3b8', size: 10 } },
|
||||
tickfont: { color: '#94a3b8', size: 10 },
|
||||
thickness: 12, len: 0.5,
|
||||
},
|
||||
},
|
||||
text: orgDrafts.map((d, i) => `${d} drafts, ${orgAuthors[i]} authors`),
|
||||
textposition: 'none',
|
||||
hovertemplate: '<b>%{y}</b><br>%{text}<extra></extra>',
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
margin: { t: 10, r: 80, b: 40, l: 180 },
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Draft Count', font: { size: 11 } } },
|
||||
}, CFG);
|
||||
|
||||
// ===========================================================
|
||||
// Cross-Org Collaboration Chart (Plotly)
|
||||
// ===========================================================
|
||||
const crossOrg = {{ cross_org | tojson }};
|
||||
if (crossOrg.length > 0) {
|
||||
const coLabels = crossOrg.map(c => `${c.org_a} + ${c.org_b}`).reverse();
|
||||
const coValues = crossOrg.map(c => c.shared_drafts).reverse();
|
||||
|
||||
Plotly.newPlot('crossOrgChart', [{
|
||||
y: coLabels, x: coValues,
|
||||
type: 'bar', orientation: 'h',
|
||||
marker: {
|
||||
color: coValues.map((v, i) => {
|
||||
const pct = i / Math.max(coValues.length - 1, 1);
|
||||
return `hsl(${160 + pct * 60}, 65%, 50%)`;
|
||||
}),
|
||||
},
|
||||
hovertemplate: '<b>%{y}</b><br>%{x} shared draft(s)<extra></extra>',
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
margin: { t: 10, r: 40, b: 40, l: 240 },
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Shared Drafts', font: { size: 11 } }, dtick: 1 },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, automargin: true, tickfont: { size: 10 } },
|
||||
}, CFG);
|
||||
} else {
|
||||
document.getElementById('crossOrgChart').innerHTML =
|
||||
'<p class="text-slate-500 text-sm text-center mt-20">No cross-org data available</p>';
|
||||
}
|
||||
|
||||
// ===========================================================
|
||||
// Sortable Authors Table
|
||||
// ===========================================================
|
||||
(function() {
|
||||
const table = document.getElementById('authorsTable');
|
||||
const tbody = document.getElementById('authorsBody');
|
||||
let sortCol = null, sortAsc = true;
|
||||
|
||||
table.querySelectorAll('th[data-sort]').forEach(th => {
|
||||
th.addEventListener('click', () => {
|
||||
const col = th.dataset.sort;
|
||||
if (sortCol === col) { sortAsc = !sortAsc; } else { sortCol = col; sortAsc = true; }
|
||||
|
||||
table.querySelectorAll('th[data-sort]').forEach(h =>
|
||||
h.textContent = h.textContent.replace(/ [▲▼]/, ''));
|
||||
th.textContent += sortAsc ? ' ▲' : ' ▼';
|
||||
|
||||
const rows = Array.from(tbody.querySelectorAll('tr'));
|
||||
rows.sort((a, b) => {
|
||||
let va, vb;
|
||||
if (col === 'name') { va = a.dataset.name.toLowerCase(); vb = b.dataset.name.toLowerCase(); }
|
||||
else if (col === 'org') { va = a.dataset.org.toLowerCase(); vb = b.dataset.org.toLowerCase(); }
|
||||
else if (col === 'drafts') { va = parseInt(a.dataset.count); vb = parseInt(b.dataset.count); }
|
||||
else { va = parseInt(a.cells[0].textContent); vb = parseInt(b.cells[0].textContent); }
|
||||
|
||||
if (typeof va === 'number') return sortAsc ? va - vb : vb - va;
|
||||
return sortAsc ? va.localeCompare(vb) : vb.localeCompare(va);
|
||||
});
|
||||
rows.forEach(r => tbody.appendChild(r));
|
||||
});
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
{% endblock %}
|
||||
165
src/webui/templates/base.html
Normal file
165
src/webui/templates/base.html
Normal file
@@ -0,0 +1,165 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en" class="dark">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>{% block title %}IETF Draft Analyzer{% endblock %}</title>
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script src="https://cdn.plot.ly/plotly-2.35.0.min.js"></script>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
|
||||
<script>
|
||||
tailwind.config = {
|
||||
darkMode: 'class',
|
||||
theme: {
|
||||
extend: {
|
||||
fontFamily: {
|
||||
sans: ['Inter', 'system-ui', 'sans-serif'],
|
||||
mono: ['JetBrains Mono', 'monospace'],
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<style>
|
||||
body { font-family: 'Inter', system-ui, sans-serif; }
|
||||
.sidebar-link {
|
||||
transition: all 0.15s ease;
|
||||
}
|
||||
.sidebar-link:hover, .sidebar-link.active {
|
||||
background: rgba(59, 130, 246, 0.15);
|
||||
color: #60a5fa;
|
||||
}
|
||||
.sidebar-link.active {
|
||||
border-right: 3px solid #3b82f6;
|
||||
}
|
||||
.stat-card {
|
||||
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
|
||||
backdrop-filter: blur(10px);
|
||||
}
|
||||
.score-badge {
|
||||
display: inline-block;
|
||||
padding: 2px 8px;
|
||||
border-radius: 9999px;
|
||||
font-weight: 600;
|
||||
font-size: 0.8rem;
|
||||
}
|
||||
.score-high { background: rgba(34, 197, 94, 0.2); color: #4ade80; }
|
||||
.score-mid { background: rgba(234, 179, 8, 0.2); color: #facc15; }
|
||||
.score-low { background: rgba(239, 68, 68, 0.2); color: #f87171; }
|
||||
.dim-bar {
|
||||
display: inline-block;
|
||||
height: 8px;
|
||||
border-radius: 4px;
|
||||
background: #3b82f6;
|
||||
vertical-align: middle;
|
||||
}
|
||||
/* Plotly dark overrides */
|
||||
.js-plotly-plot .plotly .modebar { right: 8px !important; }
|
||||
/* Scrollbar */
|
||||
::-webkit-scrollbar { width: 6px; height: 6px; }
|
||||
::-webkit-scrollbar-track { background: #1e293b; }
|
||||
::-webkit-scrollbar-thumb { background: #475569; border-radius: 3px; }
|
||||
::-webkit-scrollbar-thumb:hover { background: #64748b; }
|
||||
/* Mobile sidebar */
|
||||
@media (max-width: 768px) {
|
||||
.sidebar { transform: translateX(-100%); }
|
||||
.sidebar.open { transform: translateX(0); }
|
||||
}
|
||||
</style>
|
||||
{% block extra_head %}{% endblock %}
|
||||
</head>
|
||||
<body class="bg-slate-950 text-slate-100 min-h-screen">
|
||||
<!-- Mobile menu button -->
|
||||
<button id="menuBtn" class="md:hidden fixed top-4 left-4 z-50 p-2 bg-slate-800 rounded-lg border border-slate-700">
|
||||
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h16"/>
|
||||
</svg>
|
||||
</button>
|
||||
|
||||
<!-- Sidebar -->
|
||||
<aside id="sidebar" class="sidebar fixed top-0 left-0 h-full w-60 bg-slate-900 border-r border-slate-800 z-40 flex flex-col transition-transform md:translate-x-0">
|
||||
<div class="p-5 border-b border-slate-800">
|
||||
<h1 class="text-lg font-bold text-white tracking-tight">IETF Draft Analyzer</h1>
|
||||
<p class="text-xs text-slate-500 mt-1">AI/Agent Standards Tracker</p>
|
||||
</div>
|
||||
<nav class="flex-1 py-4 overflow-y-auto">
|
||||
<a href="/" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'overview' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2V6zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2V6zM4 16a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2v-2zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2v-2z"/></svg>
|
||||
Overview
|
||||
</a>
|
||||
<a href="/drafts" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'drafts' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
|
||||
Draft Explorer
|
||||
</a>
|
||||
<a href="/ratings" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'ratings' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M11.049 2.927c.3-.921 1.603-.921 1.902 0l1.519 4.674a1 1 0 00.95.69h4.915c.969 0 1.371 1.24.588 1.81l-3.976 2.888a1 1 0 00-.363 1.118l1.518 4.674c.3.922-.755 1.688-1.538 1.118l-3.976-2.888a1 1 0 00-1.176 0l-3.976 2.888c-.783.57-1.838-.197-1.538-1.118l1.518-4.674a1 1 0 00-.363-1.118l-3.976-2.888c-.784-.57-.38-1.81.588-1.81h4.914a1 1 0 00.951-.69l1.519-4.674z"/></svg>
|
||||
Ratings
|
||||
</a>
|
||||
<a href="/ideas" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'ideas' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9.663 17h4.673M12 3v1m6.364 1.636l-.707.707M21 12h-1M4 12H3m3.343-5.657l-.707-.707m2.828 9.9a5 5 0 117.072 0l-.548.547A3.374 3.374 0 0014 18.469V19a2 2 0 11-4 0v-.531c0-.895-.356-1.754-.988-2.386l-.548-.547z"/></svg>
|
||||
Ideas
|
||||
</a>
|
||||
<a href="/idea-clusters" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'idea_clusters' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2V6zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2V6zM4 16a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2v-2z"/><circle cx="19" cy="19" r="3" stroke="currentColor" stroke-width="2" fill="none"/></svg>
|
||||
Idea Clusters
|
||||
</a>
|
||||
<a href="/gaps" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'gaps' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-2.5L13.732 4.5c-.77-.833-2.694-.833-3.464 0L3.34 16.5c-.77.833.192 2.5 1.732 2.5z"/></svg>
|
||||
Gap Explorer
|
||||
</a>
|
||||
<a href="/timeline" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'timeline' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 8v4l3 3m6-3a9 9 0 11-18 0 9 9 0 0118 0z"/></svg>
|
||||
Timeline
|
||||
</a>
|
||||
<a href="/landscape" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'landscape' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 20l-5.447-2.724A1 1 0 013 16.382V5.618a1 1 0 011.447-.894L9 7m0 13l6-3m-6 3V7m6 10l4.553 2.276A1 1 0 0021 18.382V7.618a1 1 0 00-.553-.894L15 4m0 13V4m0 0L9 7"/></svg>
|
||||
Landscape
|
||||
</a>
|
||||
<a href="/similarity" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'similarity' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/><circle cx="5" cy="6" r="1.5" fill="currentColor" stroke="none"/><circle cx="19" cy="18" r="1.5" fill="currentColor" stroke="none"/><circle cx="18" cy="6" r="1.5" fill="currentColor" stroke="none"/></svg>
|
||||
Similarity
|
||||
</a>
|
||||
<a href="/authors" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'authors' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M17 20h5v-2a3 3 0 00-5.356-1.857M17 20H7m10 0v-2c0-.656-.126-1.283-.356-1.857M7 20H2v-2a3 3 0 015.356-1.857M7 20v-2c0-.656.126-1.283.356-1.857m0 0a5.002 5.002 0 019.288 0M15 7a3 3 0 11-6 0 3 3 0 016 0zm6 3a2 2 0 11-4 0 2 2 0 014 0zM7 10a2 2 0 11-4 0 2 2 0 014 0z"/></svg>
|
||||
Authors
|
||||
</a>
|
||||
<a href="/monitor" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'monitor' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M5.636 18.364a9 9 0 010-12.728m12.728 0a9 9 0 010 12.728m-9.9-2.829a5 5 0 010-7.07m7.072 0a5 5 0 010 7.07M13 12a1 1 0 11-2 0 1 1 0 012 0z"/></svg>
|
||||
Monitor
|
||||
</a>
|
||||
<div class="border-t border-slate-800 mt-4 pt-4">
|
||||
<a href="/about" class="sidebar-link flex items-center gap-3 px-5 py-2.5 text-sm text-slate-300 {{ 'active' if active_page == 'about' }}">
|
||||
<svg class="w-4 h-4 opacity-60" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z"/></svg>
|
||||
About
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
<div class="p-4 border-t border-slate-800 text-xs text-slate-600">
|
||||
IETF Draft Analyzer v0.3
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<!-- Main content -->
|
||||
<main class="md:ml-60 min-h-screen">
|
||||
<div class="p-6 md:p-8 max-w-7xl mx-auto">
|
||||
{% block content %}{% endblock %}
|
||||
</div>
|
||||
</main>
|
||||
|
||||
<script>
|
||||
// Mobile sidebar toggle
|
||||
const menuBtn = document.getElementById('menuBtn');
|
||||
const sidebar = document.getElementById('sidebar');
|
||||
menuBtn?.addEventListener('click', () => sidebar.classList.toggle('open'));
|
||||
// Close on click outside
|
||||
document.addEventListener('click', (e) => {
|
||||
if (window.innerWidth < 768 && sidebar.classList.contains('open') &&
|
||||
!sidebar.contains(e.target) && !menuBtn.contains(e.target)) {
|
||||
sidebar.classList.remove('open');
|
||||
}
|
||||
});
|
||||
</script>
|
||||
{% block extra_scripts %}{% endblock %}
|
||||
</body>
|
||||
</html>
|
||||
298
src/webui/templates/draft_detail.html
Normal file
298
src/webui/templates/draft_detail.html
Normal file
@@ -0,0 +1,298 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "drafts" %}
|
||||
|
||||
{% block title %}{{ draft.name }} — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block extra_head %}
|
||||
<style>
|
||||
.detail-card {
|
||||
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
|
||||
backdrop-filter: blur(10px);
|
||||
}
|
||||
.score-ring {
|
||||
width: 100px;
|
||||
height: 100px;
|
||||
border-radius: 50%;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
margin: 0 auto;
|
||||
position: relative;
|
||||
}
|
||||
.score-ring::before {
|
||||
content: '';
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
border-radius: 50%;
|
||||
padding: 4px;
|
||||
mask: linear-gradient(#fff 0 0) content-box, linear-gradient(#fff 0 0);
|
||||
mask-composite: exclude;
|
||||
-webkit-mask: linear-gradient(#fff 0 0) content-box, linear-gradient(#fff 0 0);
|
||||
-webkit-mask-composite: xor;
|
||||
}
|
||||
.score-ring-high::before { background: linear-gradient(135deg, #22c55e, #4ade80); }
|
||||
.score-ring-mid::before { background: linear-gradient(135deg, #eab308, #facc15); }
|
||||
.score-ring-low::before { background: linear-gradient(135deg, #ef4444, #f87171); }
|
||||
.dim-progress {
|
||||
height: 8px;
|
||||
border-radius: 4px;
|
||||
background: rgba(51, 65, 85, 0.5);
|
||||
overflow: hidden;
|
||||
}
|
||||
.dim-progress-fill {
|
||||
height: 100%;
|
||||
border-radius: 4px;
|
||||
transition: width 0.6s ease;
|
||||
}
|
||||
.dim-high { background: linear-gradient(90deg, #22c55e, #4ade80); }
|
||||
.dim-mid { background: linear-gradient(90deg, #eab308, #facc15); }
|
||||
.dim-low { background: linear-gradient(90deg, #ef4444, #f87171); }
|
||||
.idea-type-protocol { background: rgba(59, 130, 246, 0.15); color: #60a5fa; border-color: rgba(59, 130, 246, 0.3); }
|
||||
.idea-type-mechanism { background: rgba(168, 85, 247, 0.15); color: #c084fc; border-color: rgba(168, 85, 247, 0.3); }
|
||||
.idea-type-framework { background: rgba(34, 197, 94, 0.15); color: #4ade80; border-color: rgba(34, 197, 94, 0.3); }
|
||||
.idea-type-architecture { background: rgba(234, 179, 8, 0.15); color: #facc15; border-color: rgba(234, 179, 8, 0.3); }
|
||||
.idea-type-default { background: rgba(100, 116, 139, 0.15); color: #94a3b8; border-color: rgba(100, 116, 139, 0.3); }
|
||||
.ref-rfc { background: rgba(34, 197, 94, 0.15); color: #4ade80; }
|
||||
.ref-draft { background: rgba(59, 130, 246, 0.15); color: #60a5fa; }
|
||||
.ref-other { background: rgba(234, 179, 8, 0.15); color: #facc15; }
|
||||
</style>
|
||||
{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<!-- Breadcrumb + Header -->
|
||||
<div class="mb-6">
|
||||
<a href="/drafts" class="inline-flex items-center gap-1.5 text-sm text-slate-500 hover:text-slate-300 transition group">
|
||||
<svg class="w-4 h-4 group-hover:-translate-x-0.5 transition-transform" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7"/>
|
||||
</svg>
|
||||
Back to Explorer
|
||||
</a>
|
||||
<h1 class="text-xl font-bold text-white mt-3 leading-snug">{{ draft.title }}</h1>
|
||||
<div class="flex flex-wrap items-center gap-3 mt-2">
|
||||
<span class="text-sm text-slate-400 font-mono">{{ draft.name }}</span>
|
||||
{% if draft.rev %}
|
||||
<span class="text-xs px-2 py-0.5 rounded bg-slate-800 text-slate-500 border border-slate-700">rev {{ draft.rev }}</span>
|
||||
{% endif %}
|
||||
<span class="text-xs text-slate-600">{{ draft.date }}</span>
|
||||
{% if draft.rating %}
|
||||
<span class="score-badge {% if draft.rating.score >= 3.5 %}score-high{% elif draft.rating.score >= 2.5 %}score-mid{% else %}score-low{% endif %}">
|
||||
{{ draft.rating.score }}
|
||||
</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6">
|
||||
<!-- Left column: Main content -->
|
||||
<div class="lg:col-span-2 space-y-6">
|
||||
<!-- Abstract -->
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
|
||||
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h7"/></svg>
|
||||
Abstract
|
||||
</h2>
|
||||
<p class="text-sm text-slate-400 leading-relaxed">{{ draft.abstract or "No abstract available." }}</p>
|
||||
</div>
|
||||
|
||||
<!-- Rating Analysis -->
|
||||
{% if draft.rating %}
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
|
||||
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 19v-6a2 2 0 00-2-2H5a2 2 0 00-2 2v6a2 2 0 002 2h2a2 2 0 002-2zm0 0V9a2 2 0 012-2h2a2 2 0 012 2v10m-6 0a2 2 0 002 2h2a2 2 0 002-2m0 0V5a2 2 0 012-2h2a2 2 0 012 2v14a2 2 0 01-2 2h-2a2 2 0 01-2-2z"/></svg>
|
||||
AI Rating Analysis
|
||||
</h2>
|
||||
{% if draft.rating.summary %}
|
||||
<p class="text-sm text-slate-400 mb-5 leading-relaxed">{{ draft.rating.summary }}</p>
|
||||
{% endif %}
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
|
||||
{% for dim, label, icon in [
|
||||
("novelty", "Novelty", "M13 10V3L4 14h7v7l9-11h-7z"),
|
||||
("maturity", "Maturity", "M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z"),
|
||||
("overlap", "Overlap", "M8 16H6a2 2 0 01-2-2V6a2 2 0 012-2h8a2 2 0 012 2v2m-6 12h8a2 2 0 002-2v-8a2 2 0 00-2-2h-8a2 2 0 00-2 2v8a2 2 0 002 2z"),
|
||||
("momentum", "Momentum", "M13 7h8m0 0v8m0-8l-8 8-4-4-6 6"),
|
||||
("relevance", "Relevance", "M5 3v4M3 5h4M6 17v4m-2-2h4m5-16l2.286 6.857L21 12l-5.714 2.143L13 21l-2.286-6.857L5 12l5.714-2.143L13 3z")
|
||||
] %}
|
||||
{% set val = draft.rating[dim] %}
|
||||
<div class="bg-slate-800/30 rounded-lg p-4 border border-slate-800/50">
|
||||
<div class="flex items-center justify-between mb-2">
|
||||
<div class="flex items-center gap-2">
|
||||
<svg class="w-3.5 h-3.5 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="{{ icon }}"/></svg>
|
||||
<span class="text-xs font-semibold text-slate-300 uppercase tracking-wide">{{ label }}</span>
|
||||
</div>
|
||||
<span class="text-lg font-bold {% if val >= 4 %}text-green-400{% elif val >= 3 %}text-amber-400{% else %}text-red-400{% endif %}">{{ val }}<span class="text-xs text-slate-600 font-normal">/5</span></span>
|
||||
</div>
|
||||
<div class="dim-progress mb-2">
|
||||
<div class="dim-progress-fill {% if val >= 4 %}dim-high{% elif val >= 3 %}dim-mid{% else %}dim-low{% endif %}" style="width: {{ val * 20 }}%"></div>
|
||||
</div>
|
||||
{% if draft.rating[dim + '_note'] %}
|
||||
<p class="text-xs text-slate-500 leading-relaxed">{{ draft.rating[dim + '_note'] }}</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<!-- Ideas -->
|
||||
{% if draft.ideas %}
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-4 flex items-center gap-2">
|
||||
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9.663 17h4.673M12 3v1m6.364 1.636l-.707.707M21 12h-1M4 12H3m3.343-5.657l-.707-.707m2.828 9.9a5 5 0 117.072 0l-.548.547A3.374 3.374 0 0014 18.469V19a2 2 0 11-4 0v-.531c0-.895-.356-1.754-.988-2.386l-.548-.547z"/></svg>
|
||||
Extracted Ideas <span class="text-slate-600 font-normal">({{ draft.ideas|length }})</span>
|
||||
</h2>
|
||||
<div class="space-y-3">
|
||||
{% for idea in draft.ideas %}
|
||||
<div class="bg-slate-800/30 rounded-lg p-4 border border-slate-800/50">
|
||||
<div class="flex items-start gap-2 mb-1">
|
||||
<span class="text-sm font-medium text-slate-200 leading-snug">{{ idea.title }}</span>
|
||||
{% if idea.type %}
|
||||
{% set type_lower = idea.type|lower %}
|
||||
<span class="flex-shrink-0 px-2 py-0.5 rounded-full text-[10px] font-medium border
|
||||
{% if type_lower == 'protocol' %}idea-type-protocol
|
||||
{% elif type_lower == 'mechanism' %}idea-type-mechanism
|
||||
{% elif type_lower == 'framework' %}idea-type-framework
|
||||
{% elif type_lower == 'architecture' %}idea-type-architecture
|
||||
{% else %}idea-type-default{% endif %}">
|
||||
{{ idea.type }}
|
||||
</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if idea.description %}
|
||||
<p class="text-xs text-slate-500 leading-relaxed mt-1">{{ idea.description }}</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Right column: Sidebar -->
|
||||
<div class="space-y-6">
|
||||
<!-- Score Card -->
|
||||
{% if draft.rating %}
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-6 text-center">
|
||||
<div class="score-ring {% if draft.rating.score >= 3.5 %}score-ring-high{% elif draft.rating.score >= 2.5 %}score-ring-mid{% else %}score-ring-low{% endif %}">
|
||||
<div>
|
||||
<div class="text-3xl font-bold {% if draft.rating.score >= 3.5 %}text-green-400{% elif draft.rating.score >= 2.5 %}text-amber-400{% else %}text-red-400{% endif %}">
|
||||
{{ draft.rating.score }}
|
||||
</div>
|
||||
<div class="text-[10px] text-slate-500 uppercase tracking-wider">Score</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- Mini dimension summary -->
|
||||
<div class="mt-4 grid grid-cols-5 gap-1 text-center">
|
||||
{% for dim, abbr in [("novelty","N"), ("maturity","M"), ("overlap","O"), ("momentum","Mo"), ("relevance","R")] %}
|
||||
{% set v = draft.rating[dim] %}
|
||||
<div>
|
||||
<div class="text-xs font-bold {% if v >= 4 %}text-green-400{% elif v >= 3 %}text-amber-400{% else %}text-red-400{% endif %}">{{ v }}</div>
|
||||
<div class="text-[9px] text-slate-600 uppercase">{{ abbr }}</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<!-- Metadata -->
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
|
||||
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z"/></svg>
|
||||
Metadata
|
||||
</h2>
|
||||
<dl class="space-y-2.5 text-sm">
|
||||
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Date</dt><dd class="text-slate-300">{{ draft.date }}</dd></div>
|
||||
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Revision</dt><dd class="text-slate-300">{{ draft.rev or 'N/A' }}</dd></div>
|
||||
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Pages</dt><dd class="text-slate-300">{{ draft.pages or 'N/A' }}</dd></div>
|
||||
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Words</dt><dd class="text-slate-300">{{ '{:,}'.format(draft.words) if draft.words else 'N/A' }}</dd></div>
|
||||
<div class="flex justify-between"><dt class="text-slate-500 text-xs">Working Group</dt><dd class="text-slate-300">{{ draft.group }}</dd></div>
|
||||
</dl>
|
||||
<div class="mt-4 space-y-2">
|
||||
<a href="{{ draft.url }}" target="_blank" rel="noopener"
|
||||
class="flex items-center justify-center gap-2 px-3 py-2 bg-blue-600 text-white rounded-lg text-xs font-medium hover:bg-blue-500 transition">
|
||||
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10 6H6a2 2 0 00-2 2v10a2 2 0 002 2h10a2 2 0 002-2v-4M14 4h6m0 0v6m0-6L10 14"/></svg>
|
||||
View on Datatracker
|
||||
</a>
|
||||
{% if draft.text_url %}
|
||||
<a href="{{ draft.text_url }}" target="_blank" rel="noopener"
|
||||
class="flex items-center justify-center gap-2 px-3 py-2 border border-slate-700 text-slate-300 rounded-lg text-xs font-medium hover:border-slate-500 hover:text-white transition">
|
||||
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 10v6m0 0l-3-3m3 3l3-3m2 8H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
|
||||
Read Full Text
|
||||
</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Authors -->
|
||||
{% if draft.authors %}
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
|
||||
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M17 20h5v-2a3 3 0 00-5.356-1.857M17 20H7m10 0v-2c0-.656-.126-1.283-.356-1.857M7 20H2v-2a3 3 0 015.356-1.857M7 20v-2c0-.656.126-1.283.356-1.857m0 0a5.002 5.002 0 019.288 0M15 7a3 3 0 11-6 0 3 3 0 016 0z"/></svg>
|
||||
Authors <span class="text-slate-600 font-normal">({{ draft.authors|length }})</span>
|
||||
</h2>
|
||||
<ul class="space-y-2.5">
|
||||
{% for a in draft.authors %}
|
||||
<li class="flex items-start gap-2">
|
||||
<div class="w-7 h-7 rounded-full bg-slate-800 border border-slate-700 flex items-center justify-center flex-shrink-0 mt-0.5">
|
||||
<span class="text-xs font-semibold text-slate-400">{{ a.name[0]|upper if a.name else '?' }}</span>
|
||||
</div>
|
||||
<div>
|
||||
<a href="/drafts?q={{ a.name | urlencode }}" class="text-sm text-blue-400 hover:text-blue-300 transition">{{ a.name }}</a>
|
||||
{% if a.affiliation %}
|
||||
<div class="text-xs text-slate-500">{{ a.affiliation }}</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<!-- Categories -->
|
||||
{% if draft.rating and draft.rating.categories %}
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
|
||||
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M7 7h.01M7 3h5c.512 0 1.024.195 1.414.586l7 7a2 2 0 010 2.828l-7 7a2 2 0 01-2.828 0l-7-7A1.994 1.994 0 013 12V7a4 4 0 014-4z"/></svg>
|
||||
Categories
|
||||
</h2>
|
||||
<div class="flex flex-wrap gap-1.5">
|
||||
{% for cat in draft.rating.categories %}
|
||||
<a href="/drafts?cat={{ cat }}"
|
||||
class="px-2.5 py-1 rounded-full text-xs bg-slate-800/60 text-slate-400 border border-slate-700 hover:border-blue-500 hover:text-blue-400 transition">
|
||||
{{ cat }}
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<!-- References -->
|
||||
{% if draft.refs %}
|
||||
<div class="detail-card rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3 flex items-center gap-2">
|
||||
<svg class="w-4 h-4 text-slate-500" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13.828 10.172a4 4 0 00-5.656 0l-4 4a4 4 0 105.656 5.656l1.102-1.101m-.758-4.899a4 4 0 005.656 0l4-4a4 4 0 00-5.656-5.656l-1.1 1.1"/></svg>
|
||||
References <span class="text-slate-600 font-normal">({{ draft.refs|length }})</span>
|
||||
</h2>
|
||||
<div class="flex flex-wrap gap-1.5 max-h-48 overflow-y-auto">
|
||||
{% for ref in draft.refs %}
|
||||
{% if ref.type == 'rfc' %}
|
||||
<a href="https://www.rfc-editor.org/rfc/{{ ref.id }}" target="_blank" rel="noopener"
|
||||
class="px-2 py-0.5 rounded text-[10px] font-medium ref-rfc hover:opacity-80 transition">
|
||||
RFC {{ ref.id.replace('rfc', '') }}
|
||||
</a>
|
||||
{% elif ref.type == 'draft' %}
|
||||
<a href="/drafts/{{ ref.id }}"
|
||||
class="px-2 py-0.5 rounded text-[10px] font-medium ref-draft hover:opacity-80 transition">
|
||||
{{ ref.id }}
|
||||
</a>
|
||||
{% else %}
|
||||
<span class="px-2 py-0.5 rounded text-[10px] font-medium ref-other">
|
||||
{{ ref.type|upper }} {{ ref.id }}
|
||||
</span>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
369
src/webui/templates/drafts.html
Normal file
369
src/webui/templates/drafts.html
Normal file
@@ -0,0 +1,369 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "drafts" %}
|
||||
|
||||
{% block title %}Draft Explorer — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block extra_head %}
|
||||
<style>
|
||||
.filter-bar {
|
||||
background: linear-gradient(135deg, rgba(30, 41, 59, 0.8), rgba(30, 41, 59, 0.4));
|
||||
backdrop-filter: blur(10px);
|
||||
}
|
||||
.draft-row {
|
||||
transition: all 0.15s ease;
|
||||
}
|
||||
.draft-row:hover {
|
||||
background: rgba(59, 130, 246, 0.05);
|
||||
}
|
||||
.dim-bar-bg {
|
||||
display: inline-block;
|
||||
width: 40px;
|
||||
height: 6px;
|
||||
border-radius: 3px;
|
||||
background: rgba(51, 65, 85, 0.6);
|
||||
vertical-align: middle;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
}
|
||||
.dim-bar-fill {
|
||||
display: block;
|
||||
height: 100%;
|
||||
border-radius: 3px;
|
||||
}
|
||||
.dim-fill-high { background: #4ade80; }
|
||||
.dim-fill-mid { background: #facc15; }
|
||||
.dim-fill-low { background: #f87171; }
|
||||
.cat-pill {
|
||||
display: inline-block;
|
||||
padding: 1px 8px;
|
||||
border-radius: 9999px;
|
||||
font-size: 0.65rem;
|
||||
font-weight: 500;
|
||||
background: rgba(51, 65, 85, 0.5);
|
||||
color: #94a3b8;
|
||||
border: 1px solid rgba(71, 85, 105, 0.4);
|
||||
white-space: nowrap;
|
||||
}
|
||||
.cat-pill-active {
|
||||
background: rgba(59, 130, 246, 0.2);
|
||||
color: #60a5fa;
|
||||
border-color: rgba(59, 130, 246, 0.4);
|
||||
}
|
||||
.range-slider {
|
||||
-webkit-appearance: none;
|
||||
appearance: none;
|
||||
height: 4px;
|
||||
border-radius: 2px;
|
||||
background: #334155;
|
||||
outline: none;
|
||||
}
|
||||
.range-slider::-webkit-slider-thumb {
|
||||
-webkit-appearance: none;
|
||||
appearance: none;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border-radius: 50%;
|
||||
background: #3b82f6;
|
||||
cursor: pointer;
|
||||
border: 2px solid #1e293b;
|
||||
}
|
||||
.range-slider::-moz-range-thumb {
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border-radius: 50%;
|
||||
background: #3b82f6;
|
||||
cursor: pointer;
|
||||
border: 2px solid #1e293b;
|
||||
}
|
||||
.page-btn {
|
||||
padding: 6px 12px;
|
||||
border-radius: 8px;
|
||||
font-size: 0.8rem;
|
||||
font-weight: 500;
|
||||
transition: all 0.15s ease;
|
||||
}
|
||||
.page-btn-active {
|
||||
background: #3b82f6;
|
||||
color: white;
|
||||
}
|
||||
.page-btn-inactive {
|
||||
background: rgba(30, 41, 59, 0.6);
|
||||
border: 1px solid #334155;
|
||||
color: #94a3b8;
|
||||
}
|
||||
.page-btn-inactive:hover {
|
||||
border-color: #475569;
|
||||
color: #e2e8f0;
|
||||
}
|
||||
</style>
|
||||
{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<!-- Header -->
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Draft Explorer</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Browse, search, and filter {{ result.total }} rated Internet-Drafts on AI and agent topics.</p>
|
||||
</div>
|
||||
|
||||
<!-- Filter Bar -->
|
||||
<div class="filter-bar rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<form method="get" action="/drafts" id="filterForm">
|
||||
<!-- Row 1: Search + Sort + Submit -->
|
||||
<div class="flex flex-wrap gap-3 items-end">
|
||||
<!-- Search -->
|
||||
<div class="flex-1 min-w-[200px]">
|
||||
<label class="block text-xs font-medium text-slate-500 mb-1.5">Search</label>
|
||||
<input type="text" name="q" value="{{ search }}" placeholder="Search by name, title, or summary..."
|
||||
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-4 py-2 text-sm text-slate-200 placeholder-slate-500 focus:outline-none focus:border-blue-500 focus:ring-1 focus:ring-blue-500/30 transition">
|
||||
</div>
|
||||
<!-- Category dropdown -->
|
||||
<div class="min-w-[180px]">
|
||||
<label class="block text-xs font-medium text-slate-500 mb-1.5">Category</label>
|
||||
<select name="cat"
|
||||
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
|
||||
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
|
||||
<option value="">All categories</option>
|
||||
{% for cat, count in categories.items() %}
|
||||
<option value="{{ cat }}" {% if current_cat == cat %}selected{% endif %}>{{ cat }} ({{ count }})</option>
|
||||
{% endfor %}
|
||||
</select>
|
||||
</div>
|
||||
<!-- Sort -->
|
||||
<div class="min-w-[150px]">
|
||||
<label class="block text-xs font-medium text-slate-500 mb-1.5">Sort by</label>
|
||||
<select name="sort"
|
||||
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
|
||||
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
|
||||
<option value="score" {% if sort == 'score' %}selected{% endif %}>Score</option>
|
||||
<option value="date" {% if sort == 'date' %}selected{% endif %}>Date</option>
|
||||
<option value="novelty" {% if sort == 'novelty' %}selected{% endif %}>Novelty</option>
|
||||
<option value="maturity" {% if sort == 'maturity' %}selected{% endif %}>Maturity</option>
|
||||
<option value="relevance" {% if sort == 'relevance' %}selected{% endif %}>Relevance</option>
|
||||
<option value="momentum" {% if sort == 'momentum' %}selected{% endif %}>Momentum</option>
|
||||
<option value="overlap" {% if sort == 'overlap' %}selected{% endif %}>Overlap</option>
|
||||
<option value="name" {% if sort == 'name' %}selected{% endif %}>Name</option>
|
||||
</select>
|
||||
</div>
|
||||
<!-- Sort direction -->
|
||||
<div class="min-w-[110px]">
|
||||
<label class="block text-xs font-medium text-slate-500 mb-1.5">Direction</label>
|
||||
<select name="dir"
|
||||
class="w-full bg-slate-800/60 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500 transition appearance-none"
|
||||
style="background-image: url('data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 fill=%22none%22 viewBox=%220 0 20 20%22><path stroke=%22%236b7280%22 stroke-linecap=%22round%22 stroke-linejoin=%22round%22 stroke-width=%221.5%22 d=%22M6 8l4 4 4-4%22/></svg>'); background-position: right 0.5rem center; background-repeat: no-repeat; background-size: 1.2em 1.2em; padding-right: 2rem;">
|
||||
<option value="desc" {% if sort_dir == 'desc' %}selected{% endif %}>Descending</option>
|
||||
<option value="asc" {% if sort_dir == 'asc' %}selected{% endif %}>Ascending</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Row 2: Min Score slider -->
|
||||
<div class="mt-4 flex flex-wrap items-center gap-4">
|
||||
<div class="flex items-center gap-3">
|
||||
<label class="text-xs font-medium text-slate-500 whitespace-nowrap">Min Score:</label>
|
||||
<input type="range" name="min_score" id="scoreSlider" value="{{ min_score }}" step="0.5" min="0" max="5"
|
||||
class="range-slider w-40" oninput="document.getElementById('scoreVal').textContent = this.value">
|
||||
<span id="scoreVal" class="text-sm font-mono font-semibold text-blue-400 w-8">{{ min_score }}</span>
|
||||
</div>
|
||||
<div class="flex gap-2 ml-auto">
|
||||
<button type="submit" class="px-5 py-2 bg-blue-600 text-white rounded-lg text-sm font-medium hover:bg-blue-500 transition-colors">
|
||||
Apply Filters
|
||||
</button>
|
||||
<a href="/drafts" class="px-4 py-2 border border-slate-700 text-slate-400 rounded-lg text-sm hover:border-slate-500 hover:text-slate-300 transition-colors">
|
||||
Reset
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Row 3: Category pills (quick filter) -->
|
||||
{% if categories %}
|
||||
<div class="mt-4 pt-3 border-t border-slate-800/50">
|
||||
<div class="flex flex-wrap gap-1.5">
|
||||
<a href="/drafts?q={{ search }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
|
||||
class="cat-pill {% if not current_cat %}cat-pill-active{% endif %}">All</a>
|
||||
{% for cat, count in categories.items() %}
|
||||
<a href="/drafts?cat={{ cat }}&q={{ search }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
|
||||
class="cat-pill {% if current_cat == cat %}cat-pill-active{% endif %}">
|
||||
{{ cat }} <span class="opacity-50">{{ count }}</span>
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<!-- Results count -->
|
||||
<div class="flex items-center justify-between mb-4">
|
||||
<p class="text-sm text-slate-500">
|
||||
Showing <span class="text-slate-300 font-medium">{{ result.drafts|length }}</span> of
|
||||
<span class="text-slate-300 font-medium">{{ result.total }}</span> drafts
|
||||
{% if search %} matching "<span class="text-blue-400">{{ search }}</span>"{% endif %}
|
||||
{% if current_cat %} in <span class="text-blue-400">{{ current_cat }}</span>{% endif %}
|
||||
{% if min_score > 0 %} with score >= <span class="text-blue-400">{{ min_score }}</span>{% endif %}
|
||||
</p>
|
||||
{% if result.pages > 1 %}
|
||||
<p class="text-xs text-slate-600">Page {{ result.page }} of {{ result.pages }}</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Draft Table -->
|
||||
<div class="bg-slate-900/60 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<div class="overflow-x-auto">
|
||||
<table class="w-full text-sm">
|
||||
<thead>
|
||||
<tr class="border-b border-slate-800 bg-slate-900/80">
|
||||
{% macro sort_header(field, label, extra_class="", title="") %}
|
||||
{% set is_active = sort == field %}
|
||||
{% set next_dir = 'asc' if (is_active and sort_dir == 'desc') else 'desc' %}
|
||||
<th class="px-4 py-3 text-left text-xs font-medium uppercase tracking-wide {{ extra_class }} {{ 'text-blue-400' if is_active else 'text-slate-500' }}">
|
||||
<a href="/drafts?q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ field }}&dir={{ next_dir }}"
|
||||
class="hover:text-blue-400 transition inline-flex items-center gap-1"
|
||||
{% if title %}title="{{ title }}"{% endif %}>
|
||||
{{ label }}
|
||||
{% if is_active %}
|
||||
<svg class="w-3 h-3 {{ 'rotate-180' if sort_dir == 'asc' else '' }}" fill="currentColor" viewBox="0 0 20 20">
|
||||
<path fill-rule="evenodd" d="M5.293 7.293a1 1 0 011.414 0L10 10.586l3.293-3.293a1 1 0 111.414 1.414l-4 4a1 1 0 01-1.414 0l-4-4a1 1 0 010-1.414z" clip-rule="evenodd"/>
|
||||
</svg>
|
||||
{% endif %}
|
||||
</a>
|
||||
</th>
|
||||
{% endmacro %}
|
||||
{{ sort_header("score", "Score", "w-20") }}
|
||||
{{ sort_header("name", "Draft") }}
|
||||
{{ sort_header("date", "Date", "w-24 hidden md:table-cell") }}
|
||||
{{ sort_header("novelty", "Nov", "w-20 hidden lg:table-cell", "Novelty") }}
|
||||
{{ sort_header("maturity", "Mat", "w-20 hidden lg:table-cell", "Maturity") }}
|
||||
{{ sort_header("relevance", "Rel", "w-20 hidden lg:table-cell", "Relevance") }}
|
||||
{{ sort_header("momentum", "Mom", "w-20 hidden xl:table-cell", "Momentum") }}
|
||||
{{ sort_header("overlap", "Ovl", "w-20 hidden xl:table-cell", "Overlap") }}
|
||||
<th class="px-4 py-3 text-left text-xs font-medium text-slate-500 uppercase tracking-wide hidden md:table-cell">Categories</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody class="divide-y divide-slate-800/30">
|
||||
{% for d in result.drafts %}
|
||||
<tr class="draft-row">
|
||||
<!-- Score badge -->
|
||||
<td class="px-4 py-3">
|
||||
<span class="score-badge {% if d.score >= 3.5 %}score-high{% elif d.score >= 2.5 %}score-mid{% else %}score-low{% endif %}">
|
||||
{{ d.score }}
|
||||
</span>
|
||||
</td>
|
||||
<!-- Draft name + title -->
|
||||
<td class="px-4 py-3">
|
||||
<a href="/drafts/{{ d.name }}" class="text-blue-400 hover:text-blue-300 font-medium text-sm transition">
|
||||
{{ d.title }}
|
||||
</a>
|
||||
<div class="text-xs text-slate-600 mt-0.5 font-mono">{{ d.name }}</div>
|
||||
{% if d.summary %}
|
||||
<div class="text-xs text-slate-500 mt-1 line-clamp-1 max-w-lg">{{ d.summary }}</div>
|
||||
{% endif %}
|
||||
</td>
|
||||
<!-- Date -->
|
||||
<td class="px-4 py-3 text-xs text-slate-500 hidden md:table-cell whitespace-nowrap">{{ d.date }}</td>
|
||||
<!-- Dimension bars -->
|
||||
{% macro dim_cell(value) %}
|
||||
<td class="px-4 py-3 hidden lg:table-cell">
|
||||
<div class="flex items-center gap-1.5">
|
||||
<span class="dim-bar-bg">
|
||||
<span class="dim-bar-fill {% if value >= 4 %}dim-fill-high{% elif value >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
|
||||
style="width: {{ (value / 5 * 100)|int }}%"></span>
|
||||
</span>
|
||||
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ value }}</span>
|
||||
</div>
|
||||
</td>
|
||||
{% endmacro %}
|
||||
{{ dim_cell(d.novelty) }}
|
||||
{{ dim_cell(d.maturity) }}
|
||||
{{ dim_cell(d.relevance) }}
|
||||
<td class="px-4 py-3 hidden xl:table-cell">
|
||||
<div class="flex items-center gap-1.5">
|
||||
<span class="dim-bar-bg">
|
||||
<span class="dim-bar-fill {% if d.momentum >= 4 %}dim-fill-high{% elif d.momentum >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
|
||||
style="width: {{ (d.momentum / 5 * 100)|int }}%"></span>
|
||||
</span>
|
||||
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ d.momentum }}</span>
|
||||
</div>
|
||||
</td>
|
||||
<td class="px-4 py-3 hidden xl:table-cell">
|
||||
<div class="flex items-center gap-1.5">
|
||||
<span class="dim-bar-bg">
|
||||
<span class="dim-bar-fill {% if d.overlap >= 4 %}dim-fill-high{% elif d.overlap >= 3 %}dim-fill-mid{% else %}dim-fill-low{% endif %}"
|
||||
style="width: {{ (d.overlap / 5 * 100)|int }}%"></span>
|
||||
</span>
|
||||
<span class="text-xs text-slate-500 font-mono w-4 text-right">{{ d.overlap }}</span>
|
||||
</div>
|
||||
</td>
|
||||
<!-- Categories -->
|
||||
<td class="px-4 py-3 hidden md:table-cell">
|
||||
<div class="flex flex-wrap gap-1">
|
||||
{% for cat in d.categories[:3] %}
|
||||
<span class="cat-pill">{{ cat }}</span>
|
||||
{% endfor %}
|
||||
{% if d.categories|length > 3 %}
|
||||
<span class="cat-pill opacity-50">+{{ d.categories|length - 3 }}</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
{% if not result.drafts %}
|
||||
<tr>
|
||||
<td colspan="9" class="px-4 py-12 text-center text-slate-500">
|
||||
<svg class="w-12 h-12 mx-auto mb-3 opacity-30" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z"/>
|
||||
</svg>
|
||||
<p class="text-sm">No drafts match your filters.</p>
|
||||
<a href="/drafts" class="text-blue-400 text-sm hover:text-blue-300 mt-1 inline-block">Clear all filters</a>
|
||||
</td>
|
||||
</tr>
|
||||
{% endif %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Pagination -->
|
||||
{% if result.pages > 1 %}
|
||||
<nav class="flex items-center justify-center gap-1.5 mt-6">
|
||||
{% if result.page > 1 %}
|
||||
<a href="/drafts?page={{ result.page - 1 }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
|
||||
class="page-btn page-btn-inactive">
|
||||
<svg class="w-4 h-4 inline" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7"/></svg>
|
||||
Prev
|
||||
</a>
|
||||
{% endif %}
|
||||
|
||||
{% set start_page = [1, result.page - 2]|max %}
|
||||
{% set end_page = [result.pages, result.page + 2]|min %}
|
||||
|
||||
{% if start_page > 1 %}
|
||||
<a href="/drafts?page=1&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
|
||||
class="page-btn page-btn-inactive">1</a>
|
||||
{% if start_page > 2 %}<span class="text-slate-600 px-1">...</span>{% endif %}
|
||||
{% endif %}
|
||||
|
||||
{% for p in range(start_page, end_page + 1) %}
|
||||
{% if p == result.page %}
|
||||
<span class="page-btn page-btn-active">{{ p }}</span>
|
||||
{% else %}
|
||||
<a href="/drafts?page={{ p }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
|
||||
class="page-btn page-btn-inactive">{{ p }}</a>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
|
||||
{% if end_page < result.pages %}
|
||||
{% if end_page < result.pages - 1 %}<span class="text-slate-600 px-1">...</span>{% endif %}
|
||||
<a href="/drafts?page={{ result.pages }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
|
||||
class="page-btn page-btn-inactive">{{ result.pages }}</a>
|
||||
{% endif %}
|
||||
|
||||
{% if result.page < result.pages %}
|
||||
<a href="/drafts?page={{ result.page + 1 }}&q={{ search }}&cat={{ current_cat }}&min_score={{ min_score }}&sort={{ sort }}&dir={{ sort_dir }}"
|
||||
class="page-btn page-btn-inactive">
|
||||
Next
|
||||
<svg class="w-4 h-4 inline" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"/></svg>
|
||||
</a>
|
||||
{% endif %}
|
||||
</nav>
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
118
src/webui/templates/gap_demo.html
Normal file
118
src/webui/templates/gap_demo.html
Normal file
@@ -0,0 +1,118 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "gaps" %}
|
||||
|
||||
{% block title %}Draft Demo — Gap Explorer{% endblock %}
|
||||
|
||||
{% block extra_head %}
|
||||
<style>
|
||||
.draft-viewer {
|
||||
max-height: 75vh;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.draft-viewer::-webkit-scrollbar { width: 6px; }
|
||||
.draft-viewer::-webkit-scrollbar-track { background: #0f172a; }
|
||||
.draft-viewer::-webkit-scrollbar-thumb { background: #334155; border-radius: 3px; }
|
||||
.draft-tab {
|
||||
transition: all 0.15s ease;
|
||||
}
|
||||
.draft-tab:hover { background: rgba(59, 130, 246, 0.1); }
|
||||
.draft-tab.active {
|
||||
background: rgba(59, 130, 246, 0.15);
|
||||
border-color: #3b82f6;
|
||||
color: #60a5fa;
|
||||
}
|
||||
</style>
|
||||
{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<!-- Breadcrumb -->
|
||||
<nav class="mb-6 text-sm">
|
||||
<a href="/gaps" class="text-blue-400 hover:text-blue-300 transition">Gap Explorer</a>
|
||||
<span class="text-slate-600 mx-2">/</span>
|
||||
<span class="text-slate-400">Demo Drafts</span>
|
||||
</nav>
|
||||
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Generated Draft Demo</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">
|
||||
Pre-generated Internet-Drafts addressing identified gaps.
|
||||
These were generated by the gap-to-draft pipeline using Claude to write each section.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{% if not generated_drafts %}
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-8 text-center">
|
||||
<svg class="w-12 h-12 text-slate-700 mx-auto mb-3" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
|
||||
<p class="text-slate-500">No generated drafts found yet.</p>
|
||||
<p class="text-slate-600 text-sm mt-1">Use the gap detail page to generate one, or run <code class="text-blue-400">ietf draft-gen</code> from the CLI.</p>
|
||||
</div>
|
||||
{% else %}
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-4 gap-6">
|
||||
<!-- Draft selector sidebar -->
|
||||
<div class="lg:col-span-1">
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<div class="px-4 py-3 border-b border-slate-800">
|
||||
<h3 class="text-sm font-semibold text-slate-300">{{ generated_drafts | length }} Generated Draft{{ 's' if generated_drafts | length != 1 }}</h3>
|
||||
</div>
|
||||
<div class="divide-y divide-slate-800/50">
|
||||
{% for gd in generated_drafts %}
|
||||
<a href="/gaps/demo?file={{ gd.filename | urlencode }}"
|
||||
class="draft-tab block px-4 py-3 border-l-2
|
||||
{% if (selected and gd.filename == selected) or (not selected and loop.first) %}active border-blue-500
|
||||
{% else %}border-transparent{% endif %}">
|
||||
<div class="text-xs font-medium text-slate-300 truncate">{{ gd.title }}</div>
|
||||
<div class="text-[10px] text-slate-600 mt-0.5 font-mono">{{ gd.stem }}</div>
|
||||
<div class="text-[10px] text-slate-600 mt-0.5">{{ (gd.size / 1024) | round(1) }} KB</div>
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Draft viewer -->
|
||||
<div class="lg:col-span-3">
|
||||
{% if draft_text %}
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<div class="flex items-center justify-between px-4 py-3 border-b border-slate-800">
|
||||
<div>
|
||||
<h3 class="text-sm font-semibold text-white">{{ draft_info.title if draft_info else 'Draft' }}</h3>
|
||||
<span class="text-[10px] text-slate-600 font-mono">{{ draft_info.filename if draft_info }}</span>
|
||||
</div>
|
||||
<button onclick="downloadCurrentDraft()" class="inline-flex items-center gap-1.5 px-3 py-1.5 bg-slate-800 hover:bg-slate-700 text-slate-300 text-xs font-medium rounded-lg transition">
|
||||
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 10v6m0 0l-3-3m3 3l3-3m2 8H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
|
||||
Download .txt
|
||||
</button>
|
||||
</div>
|
||||
<div class="draft-viewer p-4">
|
||||
<pre class="text-xs text-slate-300 font-mono leading-relaxed whitespace-pre-wrap">{{ draft_text }}</pre>
|
||||
</div>
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-8 text-center">
|
||||
<p class="text-slate-500">Select a draft from the list to view it.</p>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
function downloadCurrentDraft() {
|
||||
const text = {{ draft_text | tojson if draft_text else '""' }};
|
||||
const filename = {{ (draft_info.filename if draft_info else 'draft.txt') | tojson }};
|
||||
if (!text) return;
|
||||
const blob = new Blob([text], { type: 'text/plain' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = filename;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
URL.revokeObjectURL(url);
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
211
src/webui/templates/gap_detail.html
Normal file
211
src/webui/templates/gap_detail.html
Normal file
@@ -0,0 +1,211 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "gaps" %}
|
||||
|
||||
{% block title %}{{ gap.topic }} — Gap Explorer{% endblock %}
|
||||
|
||||
{% block extra_head %}
|
||||
<style>
|
||||
.draft-output {
|
||||
max-height: 70vh;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.draft-output::-webkit-scrollbar { width: 6px; }
|
||||
.draft-output::-webkit-scrollbar-track { background: #0f172a; }
|
||||
.draft-output::-webkit-scrollbar-thumb { background: #334155; border-radius: 3px; }
|
||||
.generating-spinner {
|
||||
display: inline-block;
|
||||
width: 1rem;
|
||||
height: 1rem;
|
||||
border: 2px solid rgba(255,255,255,0.3);
|
||||
border-top-color: white;
|
||||
border-radius: 50%;
|
||||
animation: spin 0.8s linear infinite;
|
||||
}
|
||||
@keyframes spin { to { transform: rotate(360deg); } }
|
||||
</style>
|
||||
{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<!-- Breadcrumb -->
|
||||
<nav class="mb-6 text-sm">
|
||||
<a href="/gaps" class="text-blue-400 hover:text-blue-300 transition">Gap Explorer</a>
|
||||
<span class="text-slate-600 mx-2">/</span>
|
||||
<span class="text-slate-400">{{ gap.topic }}</span>
|
||||
</nav>
|
||||
|
||||
<!-- Gap header -->
|
||||
<div class="bg-slate-900 rounded-xl border
|
||||
{% if gap.severity == 'critical' %}border-red-500/40
|
||||
{% elif gap.severity == 'high' %}border-orange-500/30
|
||||
{% elif gap.severity == 'medium' %}border-yellow-500/20
|
||||
{% else %}border-slate-800{% endif %}
|
||||
p-6 mb-6">
|
||||
|
||||
<div class="flex items-start justify-between gap-4 mb-4">
|
||||
<h1 class="text-2xl font-bold text-white">{{ gap.topic }}</h1>
|
||||
<span class="px-3 py-1 rounded-full text-xs font-bold whitespace-nowrap
|
||||
{% if gap.severity == 'critical' %}bg-red-500/20 text-red-400 ring-1 ring-red-500/30
|
||||
{% elif gap.severity == 'high' %}bg-orange-500/20 text-orange-400 ring-1 ring-orange-500/30
|
||||
{% elif gap.severity == 'medium' %}bg-yellow-500/20 text-yellow-400 ring-1 ring-yellow-500/30
|
||||
{% else %}bg-green-500/20 text-green-400 ring-1 ring-green-500/30{% endif %}">
|
||||
{{ gap.severity | upper }}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{% if gap.category %}
|
||||
<span class="inline-block px-2.5 py-0.5 rounded text-xs bg-slate-800 text-slate-400 mb-4 font-medium">{{ gap.category }}</span>
|
||||
{% endif %}
|
||||
|
||||
<div class="space-y-4">
|
||||
<div>
|
||||
<h3 class="text-xs font-semibold text-slate-500 uppercase tracking-wider mb-2">Description</h3>
|
||||
<p class="text-sm text-slate-300 leading-relaxed">{{ gap.description }}</p>
|
||||
</div>
|
||||
|
||||
{% if gap.evidence %}
|
||||
<div class="bg-slate-800/50 rounded-lg p-4">
|
||||
<h3 class="text-xs font-semibold text-slate-500 uppercase tracking-wider mb-2">Evidence</h3>
|
||||
<p class="text-sm text-slate-400 leading-relaxed">{{ gap.evidence }}</p>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Links -->
|
||||
<div class="mt-4 pt-4 border-t border-slate-800/50 flex flex-wrap gap-3">
|
||||
<a href="/drafts?q={{ gap.topic | urlencode }}" class="inline-flex items-center gap-1.5 text-xs text-blue-400/70 hover:text-blue-400 transition">
|
||||
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M21 21l-6-6m2-5a7 7 0 11-14 0 7 7 0 0114 0z"/></svg>
|
||||
Search related drafts
|
||||
</a>
|
||||
{% if gap.category %}
|
||||
<span class="text-slate-700">|</span>
|
||||
<a href="/drafts?cat={{ gap.category | urlencode }}" class="inline-flex items-center gap-1.5 text-xs text-blue-400/70 hover:text-blue-400 transition">
|
||||
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
|
||||
Browse {{ gap.category }} drafts
|
||||
</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Draft Generation Section -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-6">
|
||||
<div class="flex items-center justify-between mb-4">
|
||||
<div>
|
||||
<h2 class="text-lg font-semibold text-white">Generate Internet-Draft</h2>
|
||||
<p class="text-xs text-slate-500 mt-1">Use AI to generate a full Internet-Draft addressing this gap</p>
|
||||
</div>
|
||||
<button id="generateBtn" onclick="generateDraft({{ gap.id }})"
|
||||
class="inline-flex items-center gap-2 px-4 py-2 bg-blue-600 hover:bg-blue-500 disabled:bg-slate-700 disabled:text-slate-500 text-white text-sm font-medium rounded-lg transition">
|
||||
<svg id="genIcon" class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/></svg>
|
||||
<span id="genText">Generate Draft</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<!-- Status area -->
|
||||
<div id="genStatus" class="hidden mb-4 p-3 rounded-lg bg-blue-500/10 border border-blue-500/20">
|
||||
<div class="flex items-center gap-2 text-sm text-blue-400">
|
||||
<span class="generating-spinner"></span>
|
||||
<span id="statusText">Generating draft... This may take 1-2 minutes.</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Error area -->
|
||||
<div id="genError" class="hidden mb-4 p-3 rounded-lg bg-red-500/10 border border-red-500/20">
|
||||
<p class="text-sm text-red-400" id="errorText"></p>
|
||||
</div>
|
||||
|
||||
<!-- Generated draft output -->
|
||||
<div id="draftOutput" class="hidden">
|
||||
<div class="flex items-center justify-between mb-3">
|
||||
<h3 class="text-sm font-semibold text-slate-300">Generated Draft</h3>
|
||||
<button onclick="downloadDraft()" class="inline-flex items-center gap-1.5 px-3 py-1.5 bg-slate-800 hover:bg-slate-700 text-slate-300 text-xs font-medium rounded-lg transition">
|
||||
<svg class="w-3.5 h-3.5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 10v6m0 0l-3-3m3 3l3-3m2 8H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
|
||||
Download .txt
|
||||
</button>
|
||||
</div>
|
||||
<div class="draft-output bg-slate-950 rounded-lg border border-slate-800 p-4">
|
||||
<pre id="draftText" class="text-xs text-slate-300 font-mono leading-relaxed whitespace-pre-wrap"></pre>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Hint to demo -->
|
||||
<div id="demoHint" class="mt-4 text-xs text-slate-600">
|
||||
Want to see what generated drafts look like without waiting?
|
||||
<a href="/gaps/demo" class="text-blue-500 hover:text-blue-400 transition">View the demo page</a>
|
||||
with {{ generated_drafts | length }} pre-generated examples.
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
let generatedText = '';
|
||||
let generatedFilename = '';
|
||||
|
||||
function generateDraft(gapId) {
|
||||
const btn = document.getElementById('generateBtn');
|
||||
const genIcon = document.getElementById('genIcon');
|
||||
const genText = document.getElementById('genText');
|
||||
const status = document.getElementById('genStatus');
|
||||
const error = document.getElementById('genError');
|
||||
const output = document.getElementById('draftOutput');
|
||||
const hint = document.getElementById('demoHint');
|
||||
|
||||
// Disable button, show spinner
|
||||
btn.disabled = true;
|
||||
genIcon.innerHTML = '';
|
||||
genIcon.classList.add('generating-spinner');
|
||||
genText.textContent = 'Generating...';
|
||||
status.classList.remove('hidden');
|
||||
error.classList.add('hidden');
|
||||
output.classList.add('hidden');
|
||||
hint.classList.add('hidden');
|
||||
|
||||
fetch(`/gaps/${gapId}/generate`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
})
|
||||
.then(r => r.json())
|
||||
.then(data => {
|
||||
status.classList.add('hidden');
|
||||
if (data.error) {
|
||||
error.classList.remove('hidden');
|
||||
document.getElementById('errorText').textContent = data.error;
|
||||
btn.disabled = false;
|
||||
genIcon.classList.remove('generating-spinner');
|
||||
genIcon.innerHTML = '<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/>';
|
||||
genText.textContent = 'Retry';
|
||||
} else {
|
||||
generatedText = data.text;
|
||||
generatedFilename = data.filename || 'generated-draft.txt';
|
||||
document.getElementById('draftText').textContent = data.text;
|
||||
output.classList.remove('hidden');
|
||||
genIcon.classList.remove('generating-spinner');
|
||||
genIcon.innerHTML = '<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M5 13l4 4L19 7"/>';
|
||||
genText.textContent = 'Done';
|
||||
}
|
||||
})
|
||||
.catch(err => {
|
||||
status.classList.add('hidden');
|
||||
error.classList.remove('hidden');
|
||||
document.getElementById('errorText').textContent = 'Network error: ' + err.message;
|
||||
btn.disabled = false;
|
||||
genIcon.classList.remove('generating-spinner');
|
||||
genIcon.innerHTML = '<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z"/>';
|
||||
genText.textContent = 'Retry';
|
||||
});
|
||||
}
|
||||
|
||||
function downloadDraft() {
|
||||
if (!generatedText) return;
|
||||
const blob = new Blob([generatedText], { type: 'text/plain' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = generatedFilename;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
URL.revokeObjectURL(url);
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
89
src/webui/templates/gaps.html
Normal file
89
src/webui/templates/gaps.html
Normal file
@@ -0,0 +1,89 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "gaps" %}
|
||||
|
||||
{% block title %}Gap Explorer — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Gap Explorer</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">{{ gaps | length }} identified gaps in AI/agent standards coverage — click any gap to explore details or generate a draft</p>
|
||||
</div>
|
||||
|
||||
<!-- Action bar -->
|
||||
<div class="mb-6 flex flex-wrap gap-3">
|
||||
<a href="/gaps/demo" class="inline-flex items-center gap-2 px-4 py-2 bg-blue-600 hover:bg-blue-500 text-white text-sm font-medium rounded-lg transition">
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z"/></svg>
|
||||
View Demo Draft
|
||||
</a>
|
||||
{% if generated_drafts %}
|
||||
<span class="inline-flex items-center px-3 py-2 bg-slate-800 text-slate-400 text-sm rounded-lg">
|
||||
{{ generated_drafts | length }} draft{{ 's' if generated_drafts | length != 1 }} already generated
|
||||
</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Severity overview -->
|
||||
{% set ns = namespace(critical=0, high=0, medium=0, low=0) %}
|
||||
{% for gap in gaps %}
|
||||
{% if gap.severity == 'critical' %}{% set ns.critical = ns.critical + 1 %}
|
||||
{% elif gap.severity == 'high' %}{% set ns.high = ns.high + 1 %}
|
||||
{% elif gap.severity == 'medium' %}{% set ns.medium = ns.medium + 1 %}
|
||||
{% else %}{% set ns.low = ns.low + 1 %}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
|
||||
<div class="grid grid-cols-2 md:grid-cols-5 gap-4 mb-6">
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-3xl font-bold text-slate-200">{{ gaps | length }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Total Gaps</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-red-500/30 p-4">
|
||||
<div class="text-3xl font-bold text-red-400">{{ ns.critical }}</div>
|
||||
<div class="text-xs text-red-400/70 mt-1">Critical</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-orange-500/30 p-4">
|
||||
<div class="text-3xl font-bold text-orange-400">{{ ns.high }}</div>
|
||||
<div class="text-xs text-orange-400/70 mt-1">High</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-yellow-500/30 p-4">
|
||||
<div class="text-3xl font-bold text-yellow-400">{{ ns.medium }}</div>
|
||||
<div class="text-xs text-yellow-400/70 mt-1">Medium</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-green-500/30 p-4">
|
||||
<div class="text-3xl font-bold text-green-400">{{ ns.low }}</div>
|
||||
<div class="text-xs text-green-400/70 mt-1">Low</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Gap cards sorted by severity -->
|
||||
<div class="space-y-4">
|
||||
{% for gap in gaps | sort(attribute='severity') %}
|
||||
<a href="/gaps/{{ gap.id }}" class="block bg-slate-900 rounded-xl border
|
||||
{% if gap.severity == 'critical' %}border-red-500/40 hover:border-red-500/60
|
||||
{% elif gap.severity == 'high' %}border-orange-500/30 hover:border-orange-500/50
|
||||
{% elif gap.severity == 'medium' %}border-yellow-500/20 hover:border-yellow-500/40
|
||||
{% else %}border-slate-800 hover:border-slate-700{% endif %}
|
||||
p-5 transition group">
|
||||
<div class="flex items-start justify-between gap-3 mb-3">
|
||||
<h2 class="text-base font-semibold text-white group-hover:text-blue-400 transition">{{ gap.topic }}</h2>
|
||||
<div class="flex items-center gap-2 shrink-0">
|
||||
<span class="px-2.5 py-0.5 rounded-full text-xs font-semibold whitespace-nowrap
|
||||
{% if gap.severity == 'critical' %}bg-red-500/20 text-red-400 ring-1 ring-red-500/30
|
||||
{% elif gap.severity == 'high' %}bg-orange-500/20 text-orange-400 ring-1 ring-orange-500/30
|
||||
{% elif gap.severity == 'medium' %}bg-yellow-500/20 text-yellow-400 ring-1 ring-yellow-500/30
|
||||
{% else %}bg-green-500/20 text-green-400 ring-1 ring-green-500/30{% endif %}">
|
||||
{{ gap.severity | upper }}
|
||||
</span>
|
||||
<svg class="w-4 h-4 text-slate-600 group-hover:text-blue-400 transition" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"/></svg>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% if gap.category %}
|
||||
<span class="inline-block px-2 py-0.5 rounded text-[10px] bg-slate-800 text-slate-400 mb-3 font-medium">{{ gap.category }}</span>
|
||||
{% endif %}
|
||||
|
||||
<p class="text-sm text-slate-400 leading-relaxed">{{ gap.description }}</p>
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% endblock %}
|
||||
200
src/webui/templates/idea_clusters.html
Normal file
200
src/webui/templates/idea_clusters.html
Normal file
@@ -0,0 +1,200 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "idea_clusters" %}
|
||||
|
||||
{% block title %}Idea Clusters — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Idea Clusters</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Extracted ideas grouped by semantic similarity using embedding-based clustering</p>
|
||||
</div>
|
||||
|
||||
<div id="emptyState" class="hidden">
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-12 text-center">
|
||||
<svg class="w-16 h-16 mx-auto text-slate-600 mb-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" d="M4 6a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2V6zm10 0a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2h-2a2 2 0 01-2-2V6zM4 16a2 2 0 012-2h2a2 2 0 012 2v2a2 2 0 01-2 2H6a2 2 0 01-2-2v-2z"/>
|
||||
</svg>
|
||||
<h2 class="text-lg font-semibold text-slate-300 mb-2">No idea embeddings found</h2>
|
||||
<p class="text-slate-500">Run <code class="bg-slate-800 px-2 py-1 rounded text-sm font-mono text-blue-400">ietf embed-ideas</code> to generate embeddings first.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="clusterContent" class="hidden">
|
||||
<!-- Stat cards -->
|
||||
<div class="grid grid-cols-1 md:grid-cols-3 gap-4 mb-6">
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-5">
|
||||
<p class="text-xs text-slate-500 uppercase tracking-wide">Total Ideas Embedded</p>
|
||||
<p class="text-2xl font-bold text-white mt-1" id="statTotal">0</p>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-5">
|
||||
<p class="text-xs text-slate-500 uppercase tracking-wide">Clusters Found</p>
|
||||
<p class="text-2xl font-bold text-white mt-1" id="statClusters">0</p>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-5">
|
||||
<p class="text-xs text-slate-500 uppercase tracking-wide">Avg Cluster Size</p>
|
||||
<p class="text-2xl font-bold text-white mt-1" id="statAvgSize">0</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- t-SNE Scatter -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Idea Embedding Space (t-SNE)</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Each dot is an extracted idea, colored by cluster. Hover for details, click to view the source draft.</p>
|
||||
<div id="scatterPlot" style="height: 560px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Treemap -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Cluster Sizes</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Treemap showing relative sizes of each idea cluster.</p>
|
||||
<div id="treemapPlot" style="height: 450px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Cluster cards grid -->
|
||||
<h2 class="text-lg font-semibold text-white mb-4">Cluster Details</h2>
|
||||
<div id="clusterGrid" class="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-4 mb-6">
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'transparent', plot_bgcolor: 'rgba(15,23,42,0.5)',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 20, r: 20, b: 50, l: 50 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const CFG = { responsive: true, displayModeBar: false };
|
||||
|
||||
const PALETTE = [
|
||||
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
|
||||
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
|
||||
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
|
||||
];
|
||||
|
||||
const data = {{ clusters | tojson }};
|
||||
|
||||
if (data.empty) {
|
||||
document.getElementById('emptyState').classList.remove('hidden');
|
||||
} else {
|
||||
document.getElementById('clusterContent').classList.remove('hidden');
|
||||
|
||||
// Stats
|
||||
const stats = data.stats;
|
||||
document.getElementById('statTotal').textContent = stats.total.toLocaleString();
|
||||
document.getElementById('statClusters').textContent = stats.num_clusters.toLocaleString();
|
||||
document.getElementById('statAvgSize').textContent = stats.num_clusters > 0
|
||||
? (stats.clustered / stats.num_clusters).toFixed(1) : '0';
|
||||
|
||||
// --- t-SNE Scatter ---
|
||||
if (data.scatter.length > 0) {
|
||||
// Group by cluster_id
|
||||
const groups = {};
|
||||
data.scatter.forEach(pt => {
|
||||
if (!groups[pt.cluster_id]) groups[pt.cluster_id] = { x: [], y: [], text: [], names: [] };
|
||||
groups[pt.cluster_id].x.push(pt.x);
|
||||
groups[pt.cluster_id].y.push(pt.y);
|
||||
groups[pt.cluster_id].text.push(pt.title);
|
||||
groups[pt.cluster_id].names.push(pt.draft_name);
|
||||
});
|
||||
|
||||
// Map cluster_id to cluster theme
|
||||
const clusterThemes = {};
|
||||
data.clusters.forEach((c, i) => {
|
||||
// Find the original cluster_id by matching scatter points
|
||||
});
|
||||
|
||||
const clusterIds = Object.keys(groups).sort((a, b) => (groups[b].x.length - groups[a].x.length));
|
||||
const traces = clusterIds.map((cid, i) => {
|
||||
const g = groups[cid];
|
||||
const theme = data.clusters[i] ? data.clusters[i].theme : `Cluster ${cid}`;
|
||||
return {
|
||||
x: g.x, y: g.y, text: g.text, name: theme,
|
||||
customdata: g.names,
|
||||
mode: 'markers', type: 'scatter',
|
||||
marker: {
|
||||
size: 6,
|
||||
color: PALETTE[i % PALETTE.length],
|
||||
opacity: 0.8,
|
||||
line: { width: 0.5, color: 'rgba(255,255,255,0.15)' },
|
||||
},
|
||||
hovertemplate: '<b>%{text}</b><extra>%{customdata}</extra>',
|
||||
};
|
||||
});
|
||||
|
||||
Plotly.newPlot('scatterPlot', traces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
yaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
|
||||
hovermode: 'closest',
|
||||
margin: { t: 10, r: 20, b: 10, l: 20 },
|
||||
}, CFG);
|
||||
|
||||
document.getElementById('scatterPlot').on('plotly_click', function(ev) {
|
||||
const pt = ev.points[0];
|
||||
if (pt.customdata) {
|
||||
window.location.href = '/drafts/' + pt.customdata;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// --- Treemap ---
|
||||
if (data.clusters.length > 0) {
|
||||
const labels = data.clusters.map(c => c.theme);
|
||||
const values = data.clusters.map(c => c.size);
|
||||
const colors = data.clusters.map((_, i) => PALETTE[i % PALETTE.length]);
|
||||
|
||||
Plotly.newPlot('treemapPlot', [{
|
||||
type: 'treemap',
|
||||
labels: labels,
|
||||
parents: labels.map(() => ''),
|
||||
values: values,
|
||||
textinfo: 'label+value',
|
||||
marker: { colors: colors },
|
||||
hovertemplate: '<b>%{label}</b><br>%{value} ideas<extra></extra>',
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
margin: { t: 10, r: 10, b: 10, l: 10 },
|
||||
}, CFG);
|
||||
}
|
||||
|
||||
// --- Cluster Cards ---
|
||||
const grid = document.getElementById('clusterGrid');
|
||||
data.clusters.forEach((cluster, i) => {
|
||||
const color = PALETTE[i % PALETTE.length];
|
||||
const topIdeas = cluster.ideas.slice(0, 3);
|
||||
const ideaListHtml = topIdeas.map(idea =>
|
||||
`<li class="text-xs text-slate-400 truncate" title="${idea.title}">${idea.title}</li>`
|
||||
).join('');
|
||||
const extraCount = cluster.size - topIdeas.length;
|
||||
const extraHtml = extraCount > 0
|
||||
? `<li class="text-xs text-slate-600">+${extraCount} more</li>` : '';
|
||||
|
||||
const draftBadges = cluster.drafts.slice(0, 4).map(d =>
|
||||
`<a href="/drafts/${d}" class="inline-block bg-slate-800 text-slate-400 text-xs px-2 py-0.5 rounded hover:text-blue-400 truncate max-w-[140px]" title="${d}">${d.replace('draft-', '').substring(0, 20)}</a>`
|
||||
).join(' ');
|
||||
const extraDrafts = cluster.drafts.length > 4
|
||||
? `<span class="text-xs text-slate-600">+${cluster.drafts.length - 4}</span>` : '';
|
||||
|
||||
const card = document.createElement('div');
|
||||
card.className = 'bg-slate-900 rounded-xl border border-slate-800 p-5';
|
||||
card.innerHTML = `
|
||||
<div class="flex items-center gap-2 mb-3">
|
||||
<div class="w-3 h-3 rounded-full" style="background: ${color}"></div>
|
||||
<h3 class="text-sm font-semibold text-white">${cluster.theme}</h3>
|
||||
<span class="ml-auto text-xs text-slate-500">${cluster.size} ideas</span>
|
||||
</div>
|
||||
<ul class="space-y-1 mb-3">${ideaListHtml}${extraHtml}</ul>
|
||||
<div class="border-t border-slate-800 pt-3">
|
||||
<p class="text-xs text-slate-500 mb-1">${cluster.drafts.length} source draft${cluster.drafts.length !== 1 ? 's' : ''}</p>
|
||||
<div class="flex flex-wrap gap-1">${draftBadges}${extraDrafts}</div>
|
||||
</div>
|
||||
`;
|
||||
grid.appendChild(card);
|
||||
});
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
124
src/webui/templates/ideas.html
Normal file
124
src/webui/templates/ideas.html
Normal file
@@ -0,0 +1,124 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "ideas" %}
|
||||
|
||||
{% block title %}Ideas — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Extracted Ideas</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">{{ data.total }} technical ideas extracted from rated drafts</p>
|
||||
</div>
|
||||
|
||||
<!-- Stats header -->
|
||||
<div class="grid grid-cols-2 md:grid-cols-4 gap-4 mb-6">
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-3xl font-bold text-blue-400">{{ data.total }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Total Ideas</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-3xl font-bold text-purple-400">{{ data.by_type | length }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Idea Types</div>
|
||||
</div>
|
||||
{% set top_type = data.by_type.keys() | list %}
|
||||
{% if top_type %}
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-lg font-bold text-green-400 truncate">{{ top_type[0] }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Most Common Type</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-3xl font-bold text-amber-400">{{ data.by_type[top_type[0]] }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">{{ top_type[0] }} Count</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Chart -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Ideas by Type</h2>
|
||||
<div id="ideasChart" style="height: 350px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Ideas list -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<div class="p-4 border-b border-slate-800 flex flex-col sm:flex-row gap-3">
|
||||
<input type="text" id="ideaSearch" placeholder="Search ideas by title, description, or draft name..."
|
||||
class="flex-1 bg-slate-800 border border-slate-700 rounded-lg px-4 py-2 text-sm text-slate-200 placeholder-slate-500 focus:outline-none focus:border-blue-500">
|
||||
<select id="typeFilter"
|
||||
class="bg-slate-800 border border-slate-700 rounded-lg px-3 py-2 text-sm text-slate-200 focus:outline-none focus:border-blue-500">
|
||||
<option value="">All Types</option>
|
||||
{% for t in data.by_type %}
|
||||
<option value="{{ t }}">{{ t }} ({{ data.by_type[t] }})</option>
|
||||
{% endfor %}
|
||||
</select>
|
||||
</div>
|
||||
<div class="px-4 py-2 border-b border-slate-800/50 text-xs text-slate-500">
|
||||
<span id="visibleCount">{{ data.ideas | length }}</span> ideas shown
|
||||
</div>
|
||||
<div class="divide-y divide-slate-800/50 max-h-[600px] overflow-y-auto" id="ideaList">
|
||||
{% for idea in data.ideas %}
|
||||
<div class="idea-item px-4 py-3 hover:bg-slate-800/50 transition"
|
||||
data-search="{{ idea.title|lower }} {{ idea.description|lower }} {{ idea.draft_name|lower }}"
|
||||
data-type="{{ idea.type|default('other', true)|lower }}">
|
||||
<div class="flex items-center gap-2 mb-1 flex-wrap">
|
||||
<span class="text-sm font-medium text-slate-200">{{ idea.title }}</span>
|
||||
{% if idea.type %}
|
||||
<span class="px-2 py-0.5 rounded text-[10px] font-medium bg-blue-500/20 text-blue-400 whitespace-nowrap">{{ idea.type }}</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
<p class="text-xs text-slate-500 leading-relaxed">{{ idea.description }}</p>
|
||||
<a href="/drafts/{{ idea.draft_name }}" class="text-[10px] text-slate-600 hover:text-blue-400 transition mt-1 inline-block font-mono">{{ idea.draft_name }}</a>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 10, r: 20, b: 40, l: 140 },
|
||||
xaxis: { gridcolor: '#1e293b', title: 'Count' },
|
||||
yaxis: { gridcolor: '#1e293b' },
|
||||
};
|
||||
|
||||
const byType = {{ data.by_type | tojson }};
|
||||
const types = Object.keys(byType).reverse();
|
||||
const counts = types.map(t => byType[t]);
|
||||
|
||||
// Color gradient from blue to purple
|
||||
const barColors = types.map((_, i) => {
|
||||
const ratio = i / Math.max(types.length - 1, 1);
|
||||
const r = Math.round(59 + ratio * (168 - 59));
|
||||
const g = Math.round(130 + ratio * (85 - 130));
|
||||
const b = Math.round(246 + ratio * (247 - 246));
|
||||
return `rgb(${r},${g},${b})`;
|
||||
});
|
||||
|
||||
Plotly.newPlot('ideasChart', [{
|
||||
y: types, x: counts,
|
||||
type: 'bar', orientation: 'h',
|
||||
marker: { color: barColors },
|
||||
hovertemplate: '<b>%{y}</b>: %{x} ideas<extra></extra>',
|
||||
}], PLOTLY_LAYOUT, { responsive: true, displayModeBar: false });
|
||||
|
||||
// Search and filter
|
||||
function filterIdeas() {
|
||||
const q = document.getElementById('ideaSearch').value.toLowerCase().trim();
|
||||
const typeFilter = document.getElementById('typeFilter').value.toLowerCase();
|
||||
let visible = 0;
|
||||
document.querySelectorAll('.idea-item').forEach(el => {
|
||||
const matchesSearch = !q || el.dataset.search.includes(q);
|
||||
const matchesType = !typeFilter || el.dataset.type === typeFilter;
|
||||
const show = matchesSearch && matchesType;
|
||||
el.style.display = show ? '' : 'none';
|
||||
if (show) visible++;
|
||||
});
|
||||
document.getElementById('visibleCount').textContent = visible;
|
||||
}
|
||||
|
||||
document.getElementById('ideaSearch').addEventListener('input', filterIdeas);
|
||||
document.getElementById('typeFilter').addEventListener('change', filterIdeas);
|
||||
</script>
|
||||
{% endblock %}
|
||||
232
src/webui/templates/landscape.html
Normal file
232
src/webui/templates/landscape.html
Normal file
@@ -0,0 +1,232 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "landscape" %}
|
||||
|
||||
{% block title %}Landscape — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Draft Landscape</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Multi-dimensional visualization of the AI/agent draft space</p>
|
||||
</div>
|
||||
|
||||
<!-- Embedding-based t-SNE map -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6" id="tsneSection">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Embedding Landscape (t-SNE)</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">768-dim embeddings projected to 2D. Color = category, size = composite score. Click for draft detail.</p>
|
||||
<div id="tsneMap" style="height: 560px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Main scatter: Novelty vs Maturity -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Novelty vs Maturity</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Bubble size = composite score, color = category. Hover for details.</p>
|
||||
<div id="mainScatter" style="height: 560px;"></div>
|
||||
</div>
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
|
||||
<!-- Novelty vs Overlap quadrant -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Innovation-Uniqueness Quadrant</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Novelty vs Overlap — find the novel and unique drafts.</p>
|
||||
<div id="quadrantChart" style="height: 450px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Score distributions -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Score Distributions</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Violin plots for each rating dimension.</p>
|
||||
<div id="violinChart" style="height: 450px;"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Category distribution -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Category Distribution</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Number of rated drafts per primary category.</p>
|
||||
<div id="categoryBar" style="height: 400px;"></div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'transparent', plot_bgcolor: 'rgba(15,23,42,0.5)',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 20, r: 20, b: 50, l: 50 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const CFG = { responsive: true, displayModeBar: false };
|
||||
|
||||
const PALETTE = [
|
||||
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
|
||||
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
|
||||
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
|
||||
];
|
||||
|
||||
const dist = {{ dist | tojson }};
|
||||
const tsneData = {{ tsne_data | tojson }};
|
||||
|
||||
// --- 0. t-SNE Embedding Map ---
|
||||
if (tsneData.length > 0) {
|
||||
const tsneCatGroups = {};
|
||||
tsneData.forEach(d => {
|
||||
if (!tsneCatGroups[d.category]) tsneCatGroups[d.category] = { x: [], y: [], size: [], text: [], names: [] };
|
||||
tsneCatGroups[d.category].x.push(d.x);
|
||||
tsneCatGroups[d.category].y.push(d.y);
|
||||
tsneCatGroups[d.category].size.push(Math.max(d.score * 4, 6));
|
||||
tsneCatGroups[d.category].text.push(d.title);
|
||||
tsneCatGroups[d.category].names.push(d.name);
|
||||
});
|
||||
|
||||
const catList = Object.keys(tsneCatGroups).sort((a, b) =>
|
||||
tsneCatGroups[b].x.length - tsneCatGroups[a].x.length
|
||||
);
|
||||
|
||||
const tsneTraces = catList.map((cat, i) => {
|
||||
const g = tsneCatGroups[cat];
|
||||
return {
|
||||
x: g.x, y: g.y, text: g.text, name: cat,
|
||||
customdata: g.names,
|
||||
mode: 'markers', type: 'scatter',
|
||||
marker: {
|
||||
size: g.size,
|
||||
color: PALETTE[i % PALETTE.length],
|
||||
opacity: 0.8,
|
||||
line: { width: 0.5, color: 'rgba(255,255,255,0.15)' },
|
||||
},
|
||||
hovertemplate: '<b>%{text}</b><extra>' + cat + '</extra>',
|
||||
};
|
||||
});
|
||||
|
||||
const tsnePlot = Plotly.newPlot('tsneMap', tsneTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
yaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
|
||||
hovermode: 'closest',
|
||||
margin: { t: 10, r: 20, b: 10, l: 20 },
|
||||
}, CFG);
|
||||
|
||||
// Click to navigate to draft detail
|
||||
document.getElementById('tsneMap').on('plotly_click', function(data) {
|
||||
const pt = data.points[0];
|
||||
if (pt.customdata) {
|
||||
window.location.href = '/drafts/' + pt.customdata;
|
||||
}
|
||||
});
|
||||
} else {
|
||||
document.getElementById('tsneSection').style.display = 'none';
|
||||
}
|
||||
|
||||
// --- Group by category for rating-based charts ---
|
||||
const catGroups = {};
|
||||
dist.names.forEach((name, i) => {
|
||||
const cat = dist.categories[i];
|
||||
if (!catGroups[cat]) catGroups[cat] = { x: [], y: [], nov: [], ovl: [], size: [], text: [], scores: [] };
|
||||
catGroups[cat].x.push(dist.novelty[i] + (Math.random() - 0.5) * 0.25);
|
||||
catGroups[cat].y.push(dist.maturity[i] + (Math.random() - 0.5) * 0.25);
|
||||
catGroups[cat].nov.push(dist.novelty[i]);
|
||||
catGroups[cat].ovl.push(dist.overlap[i]);
|
||||
catGroups[cat].size.push(Math.max(dist.scores[i] * 4, 5));
|
||||
catGroups[cat].text.push(name);
|
||||
catGroups[cat].scores.push(dist.scores[i]);
|
||||
});
|
||||
|
||||
// --- 1. Main Scatter: Novelty vs Maturity ---
|
||||
const mainTraces = Object.entries(catGroups).map(([cat, d]) => ({
|
||||
x: d.x, y: d.y, text: d.text, name: cat,
|
||||
customdata: d.scores.map((s, i) => [s, d.nov[i], d.ovl[i]]),
|
||||
mode: 'markers', type: 'scatter',
|
||||
marker: { size: d.size, opacity: 0.75, line: { width: 0.5, color: 'rgba(255,255,255,0.15)' } },
|
||||
hovertemplate: '<b>%{text}</b><br>Novelty: %{customdata[1]}<br>Maturity: %{y:.0f}<br>Score: %{customdata[0]:.2f}<br>Overlap: %{customdata[2]}<extra>' + cat + '</extra>',
|
||||
}));
|
||||
Plotly.newPlot('mainScatter', mainTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Novelty', range: [0.3, 5.7], dtick: 1 },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Maturity', range: [0.3, 5.7], dtick: 1 },
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
|
||||
}, CFG);
|
||||
|
||||
// Click to navigate to draft detail
|
||||
document.getElementById('mainScatter').on('plotly_click', function(data) {
|
||||
const pt = data.points[0];
|
||||
if (pt.text) {
|
||||
window.location.href = '/drafts/' + pt.text;
|
||||
}
|
||||
});
|
||||
|
||||
// --- 2. Novelty vs Overlap Quadrant ---
|
||||
const quadTraces = Object.entries(catGroups).map(([cat, d]) => ({
|
||||
x: d.nov.map(v => v + (Math.random() - 0.5) * 0.25),
|
||||
y: d.ovl.map(v => v + (Math.random() - 0.5) * 0.25),
|
||||
text: d.text, name: cat,
|
||||
customdata: d.scores,
|
||||
mode: 'markers', type: 'scatter',
|
||||
marker: { size: 7, opacity: 0.7 },
|
||||
hovertemplate: '<b>%{text}</b><br>Novelty: %{x:.0f}<br>Overlap: %{y:.0f}<br>Score: %{customdata:.2f}<extra>' + cat + '</extra>',
|
||||
showlegend: false,
|
||||
}));
|
||||
Plotly.newPlot('quadrantChart', quadTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Novelty', range: [0.3, 5.7], dtick: 1 },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Overlap', range: [0.3, 5.7], dtick: 1 },
|
||||
shapes: [
|
||||
{ type: 'line', x0: 3, x1: 3, y0: 0, y1: 6, line: { color: '#334155', width: 1, dash: 'dash' } },
|
||||
{ type: 'line', x0: 0, x1: 6, y0: 3, y1: 3, line: { color: '#334155', width: 1, dash: 'dash' } },
|
||||
],
|
||||
annotations: [
|
||||
{ x: 4.5, y: 1.2, text: 'Novel & Unique', showarrow: false, font: { size: 11, color: '#4ade80' } },
|
||||
{ x: 4.5, y: 5.0, text: 'Novel & Overlapping', showarrow: false, font: { size: 11, color: '#facc15' } },
|
||||
{ x: 1.5, y: 1.2, text: 'Mature & Unique', showarrow: false, font: { size: 11, color: '#60a5fa' } },
|
||||
{ x: 1.5, y: 5.0, text: 'Crowded', showarrow: false, font: { size: 11, color: '#f87171' } },
|
||||
],
|
||||
}, CFG);
|
||||
|
||||
// --- 3. Violin / Box Plots ---
|
||||
const dims = ['novelty', 'maturity', 'overlap', 'momentum', 'relevance'];
|
||||
const dimColors = ['#3b82f6', '#22c55e', '#ef4444', '#f59e0b', '#a855f7'];
|
||||
const violinTraces = dims.map((d, i) => ({
|
||||
y: dist[d],
|
||||
name: d.charAt(0).toUpperCase() + d.slice(1),
|
||||
type: 'violin',
|
||||
box: { visible: true },
|
||||
meanline: { visible: true },
|
||||
line: { color: dimColors[i] },
|
||||
fillcolor: dimColors[i] + '30',
|
||||
opacity: 0.85,
|
||||
}));
|
||||
Plotly.newPlot('violinChart', violinTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
showlegend: false,
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, range: [0.3, 5.7], dtick: 1, title: 'Score' },
|
||||
}, CFG);
|
||||
|
||||
// --- 4. Category Distribution ---
|
||||
const catCounts = {};
|
||||
dist.categories.forEach(c => { catCounts[c] = (catCounts[c] || 0) + 1; });
|
||||
const sorted = Object.entries(catCounts).sort((a, b) => b[1] - a[1]);
|
||||
const catNames = sorted.map(s => s[0]).reverse();
|
||||
const catValues = sorted.map(s => s[1]).reverse();
|
||||
|
||||
Plotly.newPlot('categoryBar', [{
|
||||
y: catNames, x: catValues,
|
||||
type: 'bar', orientation: 'h',
|
||||
marker: {
|
||||
color: catValues.map((_, i) => {
|
||||
const pct = i / Math.max(catValues.length - 1, 1);
|
||||
return `hsl(${210 + pct * 120}, 70%, 55%)`;
|
||||
}),
|
||||
},
|
||||
text: catValues.map(v => v.toString()),
|
||||
textposition: 'outside',
|
||||
textfont: { color: '#94a3b8', size: 11 },
|
||||
hovertemplate: '<b>%{y}</b><br>%{x} drafts<extra></extra>',
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
margin: { t: 10, r: 60, b: 40, l: 220 },
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Number of Drafts' },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, automargin: true },
|
||||
}, CFG);
|
||||
</script>
|
||||
{% endblock %}
|
||||
191
src/webui/templates/monitor.html
Normal file
191
src/webui/templates/monitor.html
Normal file
@@ -0,0 +1,191 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "monitor" %}
|
||||
|
||||
{% block title %}Monitor — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Live Monitor</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Track automated monitoring runs and pipeline status</p>
|
||||
</div>
|
||||
|
||||
<div id="monitor-app"></div>
|
||||
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'transparent',
|
||||
plot_bgcolor: 'transparent',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 30, r: 20, b: 40, l: 40 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const PLOTLY_CONFIG = { responsive: true, displayModeBar: false };
|
||||
const PALETTE = [
|
||||
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
|
||||
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
|
||||
];
|
||||
|
||||
const data = {{ status | tojson }};
|
||||
const app = document.getElementById('monitor-app');
|
||||
|
||||
// Status banner
|
||||
const lastRun = data.last_run;
|
||||
let bannerColor, bannerBorder, bannerText;
|
||||
if (!lastRun) {
|
||||
bannerColor = 'text-slate-400';
|
||||
bannerBorder = 'border-slate-700';
|
||||
bannerText = 'No monitoring runs recorded yet. Run <code class="text-slate-300">ietf monitor run</code> to start.';
|
||||
} else if (lastRun.status === 'completed') {
|
||||
bannerColor = 'text-green-400';
|
||||
bannerBorder = 'border-green-500/30';
|
||||
bannerText = 'Last run completed successfully';
|
||||
} else if (lastRun.status === 'failed') {
|
||||
bannerColor = 'text-red-400';
|
||||
bannerBorder = 'border-red-500/30';
|
||||
bannerText = 'Last run failed: ' + (lastRun.error_message || 'unknown error');
|
||||
} else {
|
||||
bannerColor = 'text-yellow-400';
|
||||
bannerBorder = 'border-yellow-500/30';
|
||||
bannerText = 'A monitoring run is currently in progress...';
|
||||
}
|
||||
|
||||
let html = `
|
||||
<div class="stat-card rounded-xl border ${bannerBorder} p-4 mb-6">
|
||||
<div class="flex items-center gap-3">
|
||||
<div class="w-3 h-3 rounded-full ${lastRun && lastRun.status === 'completed' ? 'bg-green-500' : lastRun && lastRun.status === 'failed' ? 'bg-red-500' : lastRun && lastRun.status === 'running' ? 'bg-yellow-500 animate-pulse' : 'bg-slate-600'}"></div>
|
||||
<span class="${bannerColor} font-medium">${bannerText}</span>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// Stat cards row
|
||||
const lastTime = lastRun ? (lastRun.started_at || '').replace('T', ' ').slice(0, 19) : '-';
|
||||
const lastDuration = lastRun && lastRun.duration_seconds ? lastRun.duration_seconds.toFixed(1) + 's' : '-';
|
||||
const lastNew = lastRun ? lastRun.new_drafts_found : 0;
|
||||
const totalRuns = data.total_runs;
|
||||
|
||||
html += `
|
||||
<div class="grid grid-cols-2 md:grid-cols-4 gap-4 mb-6">
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-2xl font-bold text-slate-200">${totalRuns}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Total Runs</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-sm font-mono font-bold text-slate-200 truncate">${lastTime}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Last Run</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-2xl font-bold text-slate-200">${lastDuration}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Duration</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4">
|
||||
<div class="text-2xl font-bold text-blue-400">${lastNew}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">New Drafts (Last Run)</div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// Unprocessed counts
|
||||
const up = data.unprocessed;
|
||||
function warnColor(n) { return n > 0 ? 'text-yellow-400 border-yellow-500/30' : 'text-green-400 border-green-500/30'; }
|
||||
|
||||
html += `
|
||||
<h2 class="text-lg font-semibold text-white mb-3">Unprocessed Drafts</h2>
|
||||
<div class="grid grid-cols-3 gap-4 mb-8">
|
||||
<div class="stat-card rounded-xl border ${up.unrated > 0 ? 'border-yellow-500/30' : 'border-green-500/30'} p-4">
|
||||
<div class="text-2xl font-bold ${up.unrated > 0 ? 'text-yellow-400' : 'text-green-400'}">${up.unrated}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Unrated</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border ${up.unembedded > 0 ? 'border-yellow-500/30' : 'border-green-500/30'} p-4">
|
||||
<div class="text-2xl font-bold ${up.unembedded > 0 ? 'text-yellow-400' : 'text-green-400'}">${up.unembedded}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">Un-embedded</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border ${up.no_ideas > 0 ? 'border-yellow-500/30' : 'border-green-500/30'} p-4">
|
||||
<div class="text-2xl font-bold ${up.no_ideas > 0 ? 'text-yellow-400' : 'text-green-400'}">${up.no_ideas}</div>
|
||||
<div class="text-xs text-slate-400 mt-1">No Ideas</div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// New drafts over time chart
|
||||
const runs = data.runs.slice().reverse(); // chronological order
|
||||
if (runs.length > 1) {
|
||||
html += `
|
||||
<h2 class="text-lg font-semibold text-white mb-3">New Drafts Found Over Time</h2>
|
||||
<div id="monitor-chart" class="bg-slate-900/50 rounded-xl border border-slate-800 p-4 mb-8" style="height:300px"></div>
|
||||
`;
|
||||
}
|
||||
|
||||
// Run history table
|
||||
if (data.runs.length > 0) {
|
||||
html += `
|
||||
<h2 class="text-lg font-semibold text-white mb-3">Run History</h2>
|
||||
<div class="bg-slate-900/50 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<table class="w-full text-sm">
|
||||
<thead>
|
||||
<tr class="border-b border-slate-800 text-slate-400 text-xs uppercase">
|
||||
<th class="px-4 py-3 text-left">#</th>
|
||||
<th class="px-4 py-3 text-left">Started</th>
|
||||
<th class="px-4 py-3 text-right">Duration</th>
|
||||
<th class="px-4 py-3 text-center">Status</th>
|
||||
<th class="px-4 py-3 text-right">New Drafts</th>
|
||||
<th class="px-4 py-3 text-right">Analyzed</th>
|
||||
<th class="px-4 py-3 text-right">Embedded</th>
|
||||
<th class="px-4 py-3 text-right">Ideas</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>`;
|
||||
|
||||
for (const r of data.runs) {
|
||||
const statusBadge = r.status === 'completed'
|
||||
? '<span class="px-2 py-0.5 rounded-full text-xs font-semibold bg-green-500/20 text-green-400">completed</span>'
|
||||
: r.status === 'failed'
|
||||
? '<span class="px-2 py-0.5 rounded-full text-xs font-semibold bg-red-500/20 text-red-400">failed</span>'
|
||||
: '<span class="px-2 py-0.5 rounded-full text-xs font-semibold bg-yellow-500/20 text-yellow-400">running</span>';
|
||||
const started = (r.started_at || '').replace('T', ' ').slice(0, 19);
|
||||
const dur = r.duration_seconds ? r.duration_seconds.toFixed(1) + 's' : '-';
|
||||
html += `
|
||||
<tr class="border-b border-slate-800/50 hover:bg-slate-800/30">
|
||||
<td class="px-4 py-2.5 text-slate-500">${r.id}</td>
|
||||
<td class="px-4 py-2.5 font-mono text-xs text-slate-300">${started}</td>
|
||||
<td class="px-4 py-2.5 text-right text-slate-400">${dur}</td>
|
||||
<td class="px-4 py-2.5 text-center">${statusBadge}</td>
|
||||
<td class="px-4 py-2.5 text-right text-slate-300">${r.new_drafts_found}</td>
|
||||
<td class="px-4 py-2.5 text-right text-slate-300">${r.drafts_analyzed}</td>
|
||||
<td class="px-4 py-2.5 text-right text-slate-300">${r.drafts_embedded}</td>
|
||||
<td class="px-4 py-2.5 text-right text-slate-300">${r.ideas_extracted}</td>
|
||||
</tr>`;
|
||||
}
|
||||
html += `
|
||||
</tbody>
|
||||
</table>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
app.innerHTML = html;
|
||||
|
||||
// Render chart
|
||||
if (runs.length > 1) {
|
||||
const x = runs.map(r => (r.started_at || '').slice(0, 19));
|
||||
const y = runs.map(r => r.new_drafts_found || 0);
|
||||
Plotly.newPlot('monitor-chart', [{
|
||||
x: x,
|
||||
y: y,
|
||||
type: 'scatter',
|
||||
mode: 'lines+markers',
|
||||
fill: 'tozeroy',
|
||||
line: { color: PALETTE[0], width: 2 },
|
||||
marker: { color: PALETTE[0], size: 6 },
|
||||
fillcolor: PALETTE[0] + '30',
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Run Date', font: { size: 11 } } },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'New Drafts', font: { size: 11 } } },
|
||||
}, PLOTLY_CONFIG);
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
205
src/webui/templates/overview.html
Normal file
205
src/webui/templates/overview.html
Normal file
@@ -0,0 +1,205 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "overview" %}
|
||||
|
||||
{% block title %}Overview — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-8">
|
||||
<h1 class="text-2xl font-bold text-white">Dashboard Overview</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">IETF AI/Agent Internet-Drafts at a glance</p>
|
||||
</div>
|
||||
|
||||
<!-- Stat cards -->
|
||||
<div class="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-5 gap-4 mb-8">
|
||||
<a href="/drafts" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-blue-500/40 transition group">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
|
||||
<div class="text-3xl font-bold text-blue-400">{{ stats.total_drafts }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-blue-400/70 transition">Total Drafts →</div>
|
||||
</a>
|
||||
<a href="/ratings" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-emerald-500/40 transition group">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
|
||||
<div class="text-3xl font-bold text-emerald-400">{{ stats.rated_count }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-emerald-400/70 transition">Rated Drafts →</div>
|
||||
</a>
|
||||
<a href="/authors" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-purple-500/40 transition group">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
|
||||
<div class="text-3xl font-bold text-purple-400">{{ stats.author_count }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-purple-400/70 transition">Authors →</div>
|
||||
</a>
|
||||
<a href="/ideas" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-amber-500/40 transition group">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-amber-500 to-amber-400"></div>
|
||||
<div class="text-3xl font-bold text-amber-400">{{ stats.idea_count }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-amber-400/70 transition">Ideas →</div>
|
||||
</a>
|
||||
<a href="/gaps" class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden hover:border-red-500/40 transition group">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-red-500 to-red-400"></div>
|
||||
<div class="text-3xl font-bold text-red-400">{{ stats.gap_count }}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider group-hover:text-red-400/70 transition">Gaps Found →</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<!-- Charts row 1: Score distribution + Category donut -->
|
||||
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Composite Score Distribution</h2>
|
||||
<div id="scoreHist" style="height: 300px;"></div>
|
||||
</div>
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Drafts by Category</h2>
|
||||
<div id="categoryPie" style="height: 300px;"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Timeline (full width) -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Submissions Over Time</h2>
|
||||
<div id="timeline" style="height: 350px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Category radar -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Category Rating Profiles</h2>
|
||||
<div id="radar" style="height: 420px;"></div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
// Shared Plotly config
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'transparent',
|
||||
plot_bgcolor: 'transparent',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 30, r: 20, b: 40, l: 40 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const PLOTLY_CONFIG = { responsive: true, displayModeBar: false };
|
||||
|
||||
const PALETTE = [
|
||||
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
|
||||
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
|
||||
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
|
||||
];
|
||||
|
||||
// --- Score histogram ---
|
||||
const scores = {{ scores | tojson }};
|
||||
if (scores.length > 0) {
|
||||
Plotly.newPlot('scoreHist', [{
|
||||
x: scores,
|
||||
type: 'histogram',
|
||||
nbinsx: 20,
|
||||
marker: {
|
||||
color: 'rgba(59, 130, 246, 0.7)',
|
||||
line: { color: '#3b82f6', width: 1 },
|
||||
},
|
||||
hovertemplate: 'Score: %{x}<br>Count: %{y}<extra></extra>',
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Composite Score', font: { size: 11 } } },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'Count', font: { size: 11 } } },
|
||||
}, PLOTLY_CONFIG);
|
||||
} else {
|
||||
document.getElementById('scoreHist').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No score data available</p>';
|
||||
}
|
||||
|
||||
// --- Category donut ---
|
||||
const categories = {{ categories | tojson }};
|
||||
const catNames = Object.keys(categories);
|
||||
const catVals = Object.values(categories);
|
||||
if (catNames.length > 0) {
|
||||
Plotly.newPlot('categoryPie', [{
|
||||
labels: catNames,
|
||||
values: catVals,
|
||||
type: 'pie',
|
||||
hole: 0.45,
|
||||
textinfo: 'label+percent',
|
||||
textposition: 'outside',
|
||||
textfont: { size: 10, color: '#94a3b8' },
|
||||
hovertemplate: '%{label}<br>%{value} drafts (%{percent})<extra></extra>',
|
||||
marker: { colors: PALETTE },
|
||||
pull: catVals.map((_, i) => i === 0 ? 0.03 : 0),
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
showlegend: false,
|
||||
margin: { t: 10, r: 10, b: 10, l: 10 },
|
||||
}, PLOTLY_CONFIG);
|
||||
// Click category to filter drafts
|
||||
document.getElementById('categoryPie').on('plotly_click', function(data) {
|
||||
const cat = data.points[0].label;
|
||||
if (cat) window.location.href = '/drafts?cat=' + encodeURIComponent(cat);
|
||||
});
|
||||
} else {
|
||||
document.getElementById('categoryPie').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No category data available</p>';
|
||||
}
|
||||
|
||||
// --- Timeline (stacked area) ---
|
||||
const timeline = {{ timeline | tojson }};
|
||||
if (timeline.months && timeline.months.length > 0) {
|
||||
const timeTraces = timeline.categories.map((cat, i) => ({
|
||||
x: timeline.months,
|
||||
y: timeline.series[cat],
|
||||
name: cat,
|
||||
type: 'scatter',
|
||||
mode: 'lines',
|
||||
stackgroup: 'one',
|
||||
line: { width: 0.5, color: PALETTE[i % PALETTE.length] },
|
||||
fillcolor: PALETTE[i % PALETTE.length] + '80',
|
||||
hovertemplate: '%{x}<br>' + cat + ': %{y}<extra></extra>',
|
||||
}));
|
||||
Plotly.newPlot('timeline', timeTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Month', font: { size: 11 } } },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'Drafts', font: { size: 11 } } },
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, orientation: 'h', y: -0.2, x: 0.5, xanchor: 'center' },
|
||||
hovermode: 'x unified',
|
||||
}, PLOTLY_CONFIG);
|
||||
} else {
|
||||
document.getElementById('timeline').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No timeline data available</p>';
|
||||
}
|
||||
|
||||
// --- Category radar ---
|
||||
const radar = {{ radar | tojson }};
|
||||
const dims = ['novelty', 'maturity', 'relevance', 'momentum', 'low_overlap'];
|
||||
const dimLabels = ['Novelty', 'Maturity', 'Relevance', 'Momentum', 'Low Overlap'];
|
||||
const radarCats = Object.keys(radar);
|
||||
if (radarCats.length > 0) {
|
||||
const radarTraces = radarCats.map((cat, i) => {
|
||||
const vals = radar[cat];
|
||||
return {
|
||||
type: 'scatterpolar',
|
||||
r: dims.map(d => vals[d]).concat([vals[dims[0]]]),
|
||||
theta: dimLabels.concat([dimLabels[0]]),
|
||||
fill: 'toself',
|
||||
fillcolor: PALETTE[i % PALETTE.length] + '20',
|
||||
line: { color: PALETTE[i % PALETTE.length], width: 2 },
|
||||
name: cat + ' (' + vals.count + ')',
|
||||
opacity: 0.85,
|
||||
hovertemplate: cat + '<br>%{theta}: %{r:.1f}<extra></extra>',
|
||||
};
|
||||
});
|
||||
Plotly.newPlot('radar', radarTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
polar: {
|
||||
bgcolor: 'transparent',
|
||||
radialaxis: {
|
||||
visible: true,
|
||||
range: [0, 5],
|
||||
gridcolor: '#1e293b',
|
||||
color: '#64748b',
|
||||
tickfont: { size: 10 },
|
||||
},
|
||||
angularaxis: {
|
||||
gridcolor: '#1e293b',
|
||||
color: '#94a3b8',
|
||||
tickfont: { size: 11 },
|
||||
},
|
||||
},
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, x: 1.05, y: 0.5 },
|
||||
margin: { t: 30, r: 120, b: 30, l: 60 },
|
||||
}, PLOTLY_CONFIG);
|
||||
} else {
|
||||
document.getElementById('radar').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No radar data available</p>';
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
211
src/webui/templates/ratings.html
Normal file
211
src/webui/templates/ratings.html
Normal file
@@ -0,0 +1,211 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "ratings" %}
|
||||
|
||||
{% block title %}Ratings — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Rating Analytics</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Distribution and analysis of AI-generated ratings</p>
|
||||
</div>
|
||||
|
||||
<!-- Score Distribution -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Composite Score Distribution</h2>
|
||||
<div id="scoreHist" style="height: 300px;"></div>
|
||||
</div>
|
||||
|
||||
<div class="grid grid-cols-1 lg:grid-cols-2 gap-6 mb-6">
|
||||
<!-- Dimension Box Plots -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Score Distributions by Dimension</h2>
|
||||
<div id="dimDist" style="height: 350px;"></div>
|
||||
</div>
|
||||
<!-- Category Radar -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Category Rating Profiles</h2>
|
||||
<div id="radar" style="height: 350px;"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Scatter: novelty vs maturity -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-3">Novelty vs Maturity (bubble = relevance)</h2>
|
||||
<div id="scatter" style="height: 450px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Top 20 Leaderboard -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 overflow-hidden">
|
||||
<div class="p-4 border-b border-slate-800">
|
||||
<h2 class="text-sm font-semibold text-slate-300">Top 20 Drafts by Composite Score</h2>
|
||||
</div>
|
||||
<div class="overflow-x-auto">
|
||||
<table class="w-full text-sm">
|
||||
<thead>
|
||||
<tr class="border-b border-slate-800 text-left text-xs text-slate-500">
|
||||
<th class="px-4 py-3 font-medium">#</th>
|
||||
<th class="px-4 py-3 font-medium">Draft</th>
|
||||
<th class="px-4 py-3 font-medium text-center">Score</th>
|
||||
<th class="px-4 py-3 font-medium text-center">Novelty</th>
|
||||
<th class="px-4 py-3 font-medium text-center">Maturity</th>
|
||||
<th class="px-4 py-3 font-medium text-center">Relevance</th>
|
||||
<th class="px-4 py-3 font-medium text-center">Momentum</th>
|
||||
<th class="px-4 py-3 font-medium text-center">Overlap</th>
|
||||
<th class="px-4 py-3 font-medium">Category</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="leaderboard" class="divide-y divide-slate-800/50">
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 20, r: 20, b: 40, l: 50 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const CFG = { responsive: true, displayModeBar: false };
|
||||
|
||||
const dist = {{ dist | tojson }};
|
||||
const radar = {{ radar | tojson }};
|
||||
|
||||
// Score Histogram
|
||||
Plotly.newPlot('scoreHist', [{
|
||||
x: dist.scores,
|
||||
type: 'histogram',
|
||||
nbinsx: 25,
|
||||
marker: { color: '#3b82f6', line: { color: '#1e40af', width: 1 } },
|
||||
hovertemplate: 'Score: %{x:.1f}<br>Count: %{y}<extra></extra>',
|
||||
}], {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Composite Score' },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Count' },
|
||||
}, CFG);
|
||||
|
||||
// Box plots for each dimension
|
||||
const dims = ['novelty', 'maturity', 'overlap', 'momentum', 'relevance'];
|
||||
const dimLabelsBox = ['Novelty', 'Maturity', 'Overlap', 'Momentum', 'Relevance'];
|
||||
const colors = ['#3b82f6', '#22c55e', '#ef4444', '#f59e0b', '#a855f7'];
|
||||
const boxTraces = dims.map((d, i) => ({
|
||||
y: dist[d], name: dimLabelsBox[i],
|
||||
type: 'box', marker: { color: colors[i] }, boxmean: true,
|
||||
}));
|
||||
Plotly.newPlot('dimDist', boxTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
showlegend: false,
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, range: [0.5, 5.5], dtick: 1 },
|
||||
}, CFG);
|
||||
|
||||
// Radar
|
||||
const radarDims = ['novelty', 'maturity', 'relevance', 'momentum', 'low_overlap'];
|
||||
const radarLabels = ['Novelty', 'Maturity', 'Relevance', 'Momentum', 'Low Overlap'];
|
||||
const radarTraces = Object.entries(radar).map(([cat, vals]) => ({
|
||||
type: 'scatterpolar',
|
||||
r: radarDims.map(d => vals[d]).concat([vals[radarDims[0]]]),
|
||||
theta: radarLabels.concat([radarLabels[0]]),
|
||||
fill: 'toself', name: `${cat} (${vals.count})`, opacity: 0.4,
|
||||
}));
|
||||
Plotly.newPlot('radar', radarTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
polar: {
|
||||
bgcolor: 'rgba(0,0,0,0)',
|
||||
radialaxis: { visible: true, range: [0, 5], gridcolor: '#1e293b', color: '#64748b' },
|
||||
angularaxis: { gridcolor: '#1e293b', color: '#94a3b8' },
|
||||
},
|
||||
legend: { font: { size: 10, color: '#94a3b8' } },
|
||||
margin: { t: 30, r: 60, b: 30, l: 60 },
|
||||
}, CFG);
|
||||
|
||||
// Scatter: novelty vs maturity
|
||||
const catGroups = {};
|
||||
dist.names.forEach((name, i) => {
|
||||
const cat = dist.categories[i];
|
||||
if (!catGroups[cat]) catGroups[cat] = { x: [], y: [], size: [], text: [] };
|
||||
catGroups[cat].x.push(dist.novelty[i] + (Math.random() - 0.5) * 0.3);
|
||||
catGroups[cat].y.push(dist.maturity[i] + (Math.random() - 0.5) * 0.3);
|
||||
catGroups[cat].size.push(Math.max(dist.relevance[i] * 4, 6));
|
||||
catGroups[cat].text.push(name);
|
||||
});
|
||||
const scatterTraces = Object.entries(catGroups).map(([cat, d]) => ({
|
||||
x: d.x, y: d.y, text: d.text, name: cat,
|
||||
mode: 'markers', type: 'scatter',
|
||||
marker: { size: d.size, opacity: 0.7 },
|
||||
hovertemplate: '<b>%{text}</b><br>Novelty: %{x:.1f}<br>Maturity: %{y:.1f}<extra>' + cat + '</extra>',
|
||||
}));
|
||||
Plotly.newPlot('scatter', scatterTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: 'Novelty', range: [0.5, 5.5], dtick: 1 },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: 'Maturity', range: [0.5, 5.5], dtick: 1 },
|
||||
legend: { font: { size: 10, color: '#94a3b8' } },
|
||||
hovermode: 'closest',
|
||||
}, CFG);
|
||||
|
||||
// Click scatter points to navigate to draft detail
|
||||
document.getElementById('scatter').on('plotly_click', function(data) {
|
||||
const pt = data.points[0];
|
||||
if (pt.text) {
|
||||
window.location.href = '/drafts/' + pt.text;
|
||||
}
|
||||
});
|
||||
|
||||
// Top 20 Leaderboard
|
||||
(function buildLeaderboard() {
|
||||
// Combine arrays into objects and sort by score descending
|
||||
const drafts = dist.names.map((name, i) => ({
|
||||
name,
|
||||
score: dist.scores[i],
|
||||
novelty: dist.novelty[i],
|
||||
maturity: dist.maturity[i],
|
||||
relevance: dist.relevance[i],
|
||||
momentum: dist.momentum[i],
|
||||
overlap: dist.overlap[i],
|
||||
category: dist.categories[i],
|
||||
}));
|
||||
drafts.sort((a, b) => b.score - a.score);
|
||||
|
||||
const tbody = document.getElementById('leaderboard');
|
||||
const top20 = drafts.slice(0, 20);
|
||||
|
||||
function scoreClass(score) {
|
||||
if (score >= 3.5) return 'score-high';
|
||||
if (score >= 2.5) return 'score-mid';
|
||||
return 'score-low';
|
||||
}
|
||||
|
||||
function dimBadge(val) {
|
||||
const cls = val >= 4 ? 'text-green-400' : val >= 3 ? 'text-yellow-400' : 'text-slate-500';
|
||||
return `<span class="${cls}">${val}</span>`;
|
||||
}
|
||||
|
||||
top20.forEach((d, i) => {
|
||||
const shortName = d.name.replace('draft-', '').substring(0, 40);
|
||||
const row = document.createElement('tr');
|
||||
row.className = 'hover:bg-slate-800/50 transition';
|
||||
row.innerHTML = `
|
||||
<td class="px-4 py-3 text-slate-500 font-mono text-xs">${i + 1}</td>
|
||||
<td class="px-4 py-3">
|
||||
<a href="/drafts/${d.name}" class="text-blue-400 hover:text-blue-300 transition text-xs font-mono">${shortName}</a>
|
||||
</td>
|
||||
<td class="px-4 py-3 text-center">
|
||||
<span class="score-badge ${scoreClass(d.score)}">${d.score.toFixed(2)}</span>
|
||||
</td>
|
||||
<td class="px-4 py-3 text-center">${dimBadge(d.novelty)}</td>
|
||||
<td class="px-4 py-3 text-center">${dimBadge(d.maturity)}</td>
|
||||
<td class="px-4 py-3 text-center">${dimBadge(d.relevance)}</td>
|
||||
<td class="px-4 py-3 text-center">${dimBadge(d.momentum)}</td>
|
||||
<td class="px-4 py-3 text-center">${dimBadge(d.overlap)}</td>
|
||||
<td class="px-4 py-3">
|
||||
<span class="px-2 py-0.5 rounded text-[10px] bg-slate-800 text-slate-400">${d.category}</span>
|
||||
</td>
|
||||
`;
|
||||
tbody.appendChild(row);
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
{% endblock %}
|
||||
249
src/webui/templates/similarity.html
Normal file
249
src/webui/templates/similarity.html
Normal file
@@ -0,0 +1,249 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "similarity" %}
|
||||
|
||||
{% block title %}Similarity — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Draft Similarity Graph</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Force-directed graph of draft-to-draft semantic similarity based on embeddings</p>
|
||||
</div>
|
||||
|
||||
<!-- Summary stats -->
|
||||
<div class="grid grid-cols-2 md:grid-cols-3 gap-4 mb-6">
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Connected Drafts</div>
|
||||
<div class="text-2xl font-bold text-white mt-1" id="statNodes">0</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Similarity Links</div>
|
||||
<div class="text-2xl font-bold text-white mt-1" id="statEdges">0</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-4 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
|
||||
<div class="text-xs text-slate-500 uppercase tracking-wider">Avg Similarity</div>
|
||||
<div class="text-2xl font-bold text-white mt-1" id="statAvgSim">0</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Threshold slider -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-4 mb-6">
|
||||
<div class="flex items-center gap-4 flex-wrap">
|
||||
<label class="text-sm text-slate-300 font-medium">Similarity Threshold:</label>
|
||||
<input type="range" id="thresholdSlider" min="0.50" max="0.99" step="0.01" value="0.75"
|
||||
class="w-48 h-2 bg-slate-700 rounded-lg appearance-none cursor-pointer accent-blue-500">
|
||||
<span class="text-sm font-mono text-blue-400" id="thresholdLabel">0.75</span>
|
||||
<span class="text-xs text-slate-500 ml-2">(<span id="visibleEdges">0</span> edges visible)</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Force-directed graph -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Similarity Network</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Node size = composite score, color = category. Edge opacity = similarity strength. Click a node to view draft detail.</p>
|
||||
<div id="simGraph" style="height: 640px;"></div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'transparent',
|
||||
plot_bgcolor: 'transparent',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 20, r: 20, b: 40, l: 50 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const CFG = { responsive: true, displayModeBar: false };
|
||||
|
||||
const PALETTE = [
|
||||
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
|
||||
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
|
||||
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
|
||||
];
|
||||
|
||||
const fullNetwork = {{ network | tojson }};
|
||||
|
||||
// Assign color per category
|
||||
const catSet = [...new Set(fullNetwork.nodes.map(n => n.category))];
|
||||
const catColor = {};
|
||||
catSet.forEach((c, i) => { catColor[c] = PALETTE[i % PALETTE.length]; });
|
||||
|
||||
// Update stat cards
|
||||
document.getElementById('statNodes').textContent = fullNetwork.stats.node_count;
|
||||
document.getElementById('statEdges').textContent = fullNetwork.stats.edge_count;
|
||||
document.getElementById('statAvgSim').textContent = fullNetwork.stats.avg_similarity.toFixed(3);
|
||||
|
||||
function renderGraph(threshold) {
|
||||
const edges = fullNetwork.edges.filter(e => e.similarity >= threshold);
|
||||
|
||||
// Only show nodes that are connected at current threshold
|
||||
const connectedNames = new Set();
|
||||
edges.forEach(e => { connectedNames.add(e.source); connectedNames.add(e.target); });
|
||||
const nodes = fullNetwork.nodes.filter(n => connectedNames.has(n.name));
|
||||
|
||||
document.getElementById('visibleEdges').textContent = edges.length;
|
||||
|
||||
if (nodes.length === 0) {
|
||||
document.getElementById('simGraph').innerHTML = '<p class="text-slate-500 text-sm text-center mt-20">No connections at this threshold. Try lowering it.</p>';
|
||||
return;
|
||||
}
|
||||
|
||||
// Build index
|
||||
const N = nodes.length;
|
||||
const nodeIndex = {};
|
||||
const pos = [];
|
||||
nodes.forEach((n, i) => {
|
||||
nodeIndex[n.name] = i;
|
||||
pos.push({
|
||||
x: Math.cos(i * 2 * Math.PI / N) * 3 + (Math.random() - 0.5),
|
||||
y: Math.sin(i * 2 * Math.PI / N) * 3 + (Math.random() - 0.5)
|
||||
});
|
||||
});
|
||||
|
||||
// Force-directed spring layout
|
||||
const k = Math.sqrt(80.0 / Math.max(N, 1));
|
||||
for (let iter = 0; iter < 150; iter++) {
|
||||
const disp = pos.map(() => ({ x: 0, y: 0 }));
|
||||
const temp = 3.0 * (1 - iter / 150);
|
||||
|
||||
// Repulsion between all pairs
|
||||
for (let i = 0; i < N; i++) {
|
||||
for (let j = i + 1; j < N; j++) {
|
||||
let dx = pos[i].x - pos[j].x;
|
||||
let dy = pos[i].y - pos[j].y;
|
||||
let dist = Math.sqrt(dx * dx + dy * dy) || 0.01;
|
||||
let force = k * k / dist;
|
||||
disp[i].x += (dx / dist) * force;
|
||||
disp[i].y += (dy / dist) * force;
|
||||
disp[j].x -= (dx / dist) * force;
|
||||
disp[j].y -= (dy / dist) * force;
|
||||
}
|
||||
}
|
||||
|
||||
// Attraction along edges
|
||||
for (const e of edges) {
|
||||
const si = nodeIndex[e.source];
|
||||
const ti = nodeIndex[e.target];
|
||||
if (si === undefined || ti === undefined) continue;
|
||||
let dx = pos[si].x - pos[ti].x;
|
||||
let dy = pos[si].y - pos[ti].y;
|
||||
let dist = Math.sqrt(dx * dx + dy * dy) || 0.01;
|
||||
let force = dist * dist / k * e.similarity;
|
||||
disp[si].x -= (dx / dist) * force;
|
||||
disp[si].y -= (dy / dist) * force;
|
||||
disp[ti].x += (dx / dist) * force;
|
||||
disp[ti].y += (dy / dist) * force;
|
||||
}
|
||||
|
||||
// Apply with temperature
|
||||
for (let i = 0; i < N; i++) {
|
||||
let len = Math.sqrt(disp[i].x * disp[i].x + disp[i].y * disp[i].y) || 0.01;
|
||||
pos[i].x += (disp[i].x / len) * Math.min(len, temp);
|
||||
pos[i].y += (disp[i].y / len) * Math.min(len, temp);
|
||||
}
|
||||
}
|
||||
|
||||
// Count connections per node for hover
|
||||
const connCount = {};
|
||||
edges.forEach(e => {
|
||||
connCount[e.source] = (connCount[e.source] || 0) + 1;
|
||||
connCount[e.target] = (connCount[e.target] || 0) + 1;
|
||||
});
|
||||
|
||||
// Build edge traces — group by opacity bands for performance
|
||||
const edgeX = [];
|
||||
const edgeY = [];
|
||||
for (const e of edges) {
|
||||
const si = nodeIndex[e.source];
|
||||
const ti = nodeIndex[e.target];
|
||||
if (si === undefined || ti === undefined) continue;
|
||||
edgeX.push(pos[si].x, pos[ti].x, null);
|
||||
edgeY.push(pos[si].y, pos[ti].y, null);
|
||||
}
|
||||
|
||||
// Compute per-segment opacity based on similarity
|
||||
// Plotly lines don't support per-segment opacity easily, so we use a base color
|
||||
const minSim = Math.min(...edges.map(e => e.similarity));
|
||||
const maxSim = Math.max(...edges.map(e => e.similarity));
|
||||
const avgOpacity = edges.length > 0 ? 0.15 + 0.35 * ((maxSim + minSim) / 2 - threshold) / Math.max(1 - threshold, 0.01) : 0.2;
|
||||
|
||||
const edgeTrace = {
|
||||
x: edgeX, y: edgeY,
|
||||
mode: 'lines',
|
||||
type: 'scatter',
|
||||
line: { color: `rgba(100, 116, 139, ${Math.min(avgOpacity, 0.4).toFixed(2)})`, width: 0.8 },
|
||||
hoverinfo: 'skip',
|
||||
showlegend: false,
|
||||
};
|
||||
|
||||
// Build node trace grouped by category for legend
|
||||
const catGroups = {};
|
||||
nodes.forEach((n, i) => {
|
||||
if (!catGroups[n.category]) catGroups[n.category] = { x: [], y: [], size: [], text: [], names: [] };
|
||||
catGroups[n.category].x.push(pos[i].x);
|
||||
catGroups[n.category].y.push(pos[i].y);
|
||||
catGroups[n.category].size.push(Math.max(n.score * 4, 6));
|
||||
catGroups[n.category].text.push(
|
||||
`<b>${n.title}</b><br>Category: ${n.category}<br>Score: ${n.score}<br>Connections: ${connCount[n.name] || 0}`
|
||||
);
|
||||
catGroups[n.category].names.push(n.name);
|
||||
});
|
||||
|
||||
const catList = Object.keys(catGroups).sort((a, b) =>
|
||||
catGroups[b].x.length - catGroups[a].x.length
|
||||
);
|
||||
|
||||
const nodeTraces = catList.map((cat, i) => {
|
||||
const g = catGroups[cat];
|
||||
return {
|
||||
x: g.x, y: g.y,
|
||||
customdata: g.names,
|
||||
mode: 'markers',
|
||||
type: 'scatter',
|
||||
name: cat,
|
||||
marker: {
|
||||
size: g.size,
|
||||
color: catColor[cat] || '#64748b',
|
||||
opacity: 0.85,
|
||||
line: { color: 'rgba(255,255,255,0.15)', width: 1 },
|
||||
},
|
||||
hovertext: g.text,
|
||||
hoverinfo: 'text',
|
||||
};
|
||||
});
|
||||
|
||||
Plotly.newPlot('simGraph', [edgeTrace, ...nodeTraces], {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
yaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent', x: 1.02, y: 0.5 },
|
||||
margin: { t: 10, r: 140, b: 10, l: 10 },
|
||||
hovermode: 'closest',
|
||||
}, CFG);
|
||||
|
||||
// Click to navigate to draft detail
|
||||
document.getElementById('simGraph').on('plotly_click', function(data) {
|
||||
const pt = data.points[0];
|
||||
if (pt.customdata) {
|
||||
window.location.href = '/drafts/' + pt.customdata;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Initial render
|
||||
renderGraph(0.75);
|
||||
|
||||
// Threshold slider
|
||||
const slider = document.getElementById('thresholdSlider');
|
||||
const label = document.getElementById('thresholdLabel');
|
||||
slider.addEventListener('input', function() {
|
||||
const val = parseFloat(this.value);
|
||||
label.textContent = val.toFixed(2);
|
||||
renderGraph(val);
|
||||
});
|
||||
</script>
|
||||
{% endblock %}
|
||||
241
src/webui/templates/timeline.html
Normal file
241
src/webui/templates/timeline.html
Normal file
@@ -0,0 +1,241 @@
|
||||
{% extends "base.html" %}
|
||||
{% set active_page = "timeline" %}
|
||||
|
||||
{% block title %}Timeline — IETF Draft Analyzer{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="mb-6">
|
||||
<h1 class="text-2xl font-bold text-white">Timeline Animation</h1>
|
||||
<p class="text-slate-400 text-sm mt-1">Watch the AI/agent draft landscape evolve month by month</p>
|
||||
</div>
|
||||
|
||||
<!-- Stats summary -->
|
||||
<div class="grid grid-cols-3 gap-4 mb-6" id="statCards">
|
||||
</div>
|
||||
|
||||
<!-- Animated t-SNE map -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6" id="tsneSection">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Animated Embedding Landscape</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">t-SNE projection with cumulative drafts per month. Color = category, size = composite score. Press Play to animate.</p>
|
||||
<div id="monthBadge" class="text-center mb-2">
|
||||
<span class="inline-block bg-slate-800 border border-slate-700 rounded-lg px-4 py-1.5 text-sm font-mono text-blue-400"></span>
|
||||
</div>
|
||||
<div id="tsneAnim" style="height: 560px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Stacked area chart -->
|
||||
<div class="bg-slate-900 rounded-xl border border-slate-800 p-5 mb-6">
|
||||
<h2 class="text-sm font-semibold text-slate-300 mb-1">Category Submissions Over Time</h2>
|
||||
<p class="text-xs text-slate-500 mb-3">Stacked area chart showing draft submissions by category per month.</p>
|
||||
<div id="stackedArea" style="height: 400px;"></div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block extra_scripts %}
|
||||
<script>
|
||||
const PLOTLY_LAYOUT = {
|
||||
paper_bgcolor: 'transparent', plot_bgcolor: 'rgba(15,23,42,0.5)',
|
||||
font: { color: '#94a3b8', family: 'Inter, system-ui, sans-serif', size: 12 },
|
||||
margin: { t: 20, r: 20, b: 50, l: 50 },
|
||||
xaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
yaxis: { gridcolor: '#1e293b', zerolinecolor: '#334155' },
|
||||
};
|
||||
const CFG = { responsive: true, displayModeBar: false };
|
||||
|
||||
const PALETTE = [
|
||||
'#3b82f6', '#ef4444', '#22c55e', '#a855f7', '#f59e0b',
|
||||
'#06b6d4', '#ec4899', '#84cc16', '#f97316', '#8b5cf6',
|
||||
'#14b8a6', '#e11d48', '#64748b', '#eab308', '#6366f1',
|
||||
];
|
||||
|
||||
const animData = {{ animation | tojson }};
|
||||
const points = animData.points;
|
||||
const months = animData.months;
|
||||
const catMonthly = animData.category_monthly;
|
||||
|
||||
if (points.length > 0 && months.length > 0) {
|
||||
|
||||
// --- Stat cards ---
|
||||
const firstMonth = months[0];
|
||||
const lastMonth = months[months.length - 1];
|
||||
const allCats = [...new Set(points.map(p => p.category))];
|
||||
document.getElementById('statCards').innerHTML = `
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-blue-500 to-blue-400"></div>
|
||||
<div class="text-3xl font-bold text-blue-400">${months.length}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider">Months Span</div>
|
||||
<div class="text-xs text-slate-500 mt-0.5">${firstMonth} to ${lastMonth}</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-emerald-500 to-emerald-400"></div>
|
||||
<div class="text-3xl font-bold text-emerald-400">${points.length}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider">Total Drafts</div>
|
||||
</div>
|
||||
<div class="stat-card rounded-xl border border-slate-800 p-5 relative overflow-hidden">
|
||||
<div class="absolute top-0 left-0 w-full h-1 bg-gradient-to-r from-purple-500 to-purple-400"></div>
|
||||
<div class="text-3xl font-bold text-purple-400">${allCats.length}</div>
|
||||
<div class="text-xs text-slate-400 mt-1 uppercase tracking-wider">Categories</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// --- Build category list sorted by frequency ---
|
||||
const catCounts = {};
|
||||
points.forEach(p => { catCounts[p.category] = (catCounts[p.category] || 0) + 1; });
|
||||
const catList = Object.keys(catCounts).sort((a, b) => catCounts[b] - catCounts[a]);
|
||||
const catColor = {};
|
||||
catList.forEach((c, i) => { catColor[c] = PALETTE[i % PALETTE.length]; });
|
||||
|
||||
// --- Helper: build traces for points up to a given month ---
|
||||
function buildTraces(upToMonth) {
|
||||
const filtered = points.filter(p => p.month <= upToMonth);
|
||||
const groups = {};
|
||||
filtered.forEach(p => {
|
||||
if (!groups[p.category]) groups[p.category] = { x: [], y: [], size: [], text: [], names: [] };
|
||||
groups[p.category].x.push(p.x);
|
||||
groups[p.category].y.push(p.y);
|
||||
groups[p.category].size.push(Math.max(p.score * 4, 6));
|
||||
groups[p.category].text.push(p.title);
|
||||
groups[p.category].names.push(p.name);
|
||||
});
|
||||
return catList.map(cat => {
|
||||
const g = groups[cat] || { x: [], y: [], size: [], text: [], names: [] };
|
||||
return {
|
||||
x: g.x, y: g.y, text: g.text, name: cat,
|
||||
customdata: g.names,
|
||||
mode: 'markers', type: 'scatter',
|
||||
marker: {
|
||||
size: g.size,
|
||||
color: catColor[cat],
|
||||
opacity: 0.8,
|
||||
line: { width: 0.5, color: 'rgba(255,255,255,0.15)' },
|
||||
},
|
||||
hovertemplate: '<b>%{text}</b><extra>' + cat + '</extra>',
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
// --- Build frames ---
|
||||
const frames = months.map(month => {
|
||||
const cumCount = points.filter(p => p.month <= month).length;
|
||||
return {
|
||||
name: month,
|
||||
data: buildTraces(month),
|
||||
};
|
||||
});
|
||||
|
||||
// --- Initial plot (first month) ---
|
||||
const firstTraces = buildTraces(months[0]);
|
||||
const firstCount = points.filter(p => p.month <= months[0]).length;
|
||||
|
||||
// Slider steps
|
||||
const sliderSteps = months.map(month => ({
|
||||
method: 'animate',
|
||||
label: month,
|
||||
args: [[month], { frame: { duration: 500, redraw: true }, transition: { duration: 300 }, mode: 'immediate' }],
|
||||
}));
|
||||
|
||||
const layout = {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
yaxis: { visible: false, showgrid: false, zeroline: false },
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, bgcolor: 'transparent' },
|
||||
hovermode: 'closest',
|
||||
margin: { t: 40, r: 20, b: 60, l: 20 },
|
||||
updatemenus: [{
|
||||
type: 'buttons', showactive: false, x: 0.05, y: 1.08,
|
||||
buttons: [
|
||||
{
|
||||
label: '▶ Play',
|
||||
method: 'animate',
|
||||
args: [null, { frame: { duration: 500, redraw: true }, transition: { duration: 300 }, fromcurrent: true }]
|
||||
},
|
||||
{
|
||||
label: '◼ Pause',
|
||||
method: 'animate',
|
||||
args: [[null], { frame: { duration: 0, redraw: true }, mode: 'immediate' }]
|
||||
}
|
||||
]
|
||||
}],
|
||||
sliders: [{
|
||||
active: 0,
|
||||
steps: sliderSteps,
|
||||
x: 0.05, len: 0.9,
|
||||
xanchor: 'left',
|
||||
y: -0.02,
|
||||
yanchor: 'top',
|
||||
pad: { t: 30, b: 10 },
|
||||
currentvalue: { visible: false },
|
||||
transition: { duration: 300 },
|
||||
font: { size: 9, color: '#64748b' },
|
||||
bgcolor: '#1e293b',
|
||||
activebgcolor: '#3b82f6',
|
||||
bordercolor: '#334155',
|
||||
borderwidth: 1,
|
||||
ticklen: 4,
|
||||
tickcolor: '#475569',
|
||||
}],
|
||||
};
|
||||
|
||||
Plotly.newPlot('tsneAnim', firstTraces, layout, CFG).then(() => {
|
||||
Plotly.addFrames('tsneAnim', frames);
|
||||
});
|
||||
|
||||
// Update badge on animation frame
|
||||
const badge = document.querySelector('#monthBadge span');
|
||||
badge.textContent = `Month: ${months[0]} (${firstCount} drafts)`;
|
||||
|
||||
document.getElementById('tsneAnim').on('plotly_animatingframe', function(ev) {
|
||||
const month = ev.name;
|
||||
const cumCount = points.filter(p => p.month <= month).length;
|
||||
badge.textContent = `Month: ${month} (${cumCount} drafts)`;
|
||||
});
|
||||
|
||||
// Click to navigate
|
||||
document.getElementById('tsneAnim').on('plotly_click', function(data) {
|
||||
const pt = data.points[0];
|
||||
if (pt.customdata) {
|
||||
window.location.href = '/drafts/' + pt.customdata;
|
||||
}
|
||||
});
|
||||
|
||||
// --- Stacked area chart ---
|
||||
// Collect all categories across all months
|
||||
const areaCats = {};
|
||||
Object.values(catMonthly).forEach(mc => {
|
||||
Object.keys(mc).forEach(c => { areaCats[c] = true; });
|
||||
});
|
||||
// Sort by total count
|
||||
const areaCatList = Object.keys(areaCats).sort((a, b) => {
|
||||
const totalA = months.reduce((s, m) => s + ((catMonthly[m] || {})[a] || 0), 0);
|
||||
const totalB = months.reduce((s, m) => s + ((catMonthly[m] || {})[b] || 0), 0);
|
||||
return totalB - totalA;
|
||||
});
|
||||
|
||||
const areaTraces = areaCatList.map((cat, i) => ({
|
||||
x: months,
|
||||
y: months.map(m => (catMonthly[m] || {})[cat] || 0),
|
||||
name: cat,
|
||||
type: 'scatter',
|
||||
mode: 'lines',
|
||||
stackgroup: 'one',
|
||||
line: { width: 0.5, color: catColor[cat] || PALETTE[i % PALETTE.length] },
|
||||
fillcolor: (catColor[cat] || PALETTE[i % PALETTE.length]) + '80',
|
||||
hovertemplate: '%{x}<br>' + cat + ': %{y}<extra></extra>',
|
||||
}));
|
||||
|
||||
Plotly.newPlot('stackedArea', areaTraces, {
|
||||
...PLOTLY_LAYOUT,
|
||||
xaxis: { ...PLOTLY_LAYOUT.xaxis, title: { text: 'Month', font: { size: 11 } } },
|
||||
yaxis: { ...PLOTLY_LAYOUT.yaxis, title: { text: 'Drafts', font: { size: 11 } } },
|
||||
legend: { font: { size: 10, color: '#94a3b8' }, orientation: 'h', y: -0.25, x: 0.5, xanchor: 'center' },
|
||||
hovermode: 'x unified',
|
||||
margin: { t: 20, r: 20, b: 80, l: 50 },
|
||||
}, CFG);
|
||||
|
||||
} else {
|
||||
document.getElementById('tsneSection').innerHTML = '<p class="text-slate-500 text-sm text-center py-20">No timeline animation data available. Run the analysis pipeline first.</p>';
|
||||
document.getElementById('stackedArea').innerHTML = '<p class="text-slate-500 text-sm text-center py-20">No data available.</p>';
|
||||
document.getElementById('statCards').style.display = 'none';
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
Reference in New Issue
Block a user