Commit Graph

12 Commits

Author SHA1 Message Date
3e36802500 Fix author network graph: tune force sim, reduce node count
- Raise co-authorship threshold from 1 to 2 shared drafts (498→156 nodes)
- Tune D3 force parameters for large graphs: capped link strength,
  wider distance, adaptive charge, lower velocity decay
- Add initial circular layout to prevent explosion on load
- Fix cluster highlighting with fallback name matching and
  position validation before zoom-to-fit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:04:18 +01:00
757b781c67 Platform upgrade: semantic search, citations, readiness, tests, Docker
Major features added by 5 parallel agent teams:
- Semantic "Ask" (NL queries via FTS5 + embeddings + Claude synthesis)
- Global search across drafts, ideas, authors, gaps
- REST API expansion (14 endpoints, up from 3) with CSV/JSON export
- Citation graph visualization (D3.js, 440 nodes, 2422 edges)
- Standards readiness scoring (0-100 composite from 6 factors)
- Side-by-side draft comparison view with shared/unique analysis
- Annotation system (notes + tags per draft, DB-persisted)
- Docker deployment (Dockerfile + docker-compose with Ollama)
- Scheduled updates (cron script with log rotation)
- Pipeline health dashboard (stage progress bars, cost tracking)
- Test suite foundation (54 pytest tests covering DB, models, web data)

Fixes: compare_drafts() stubbed→working, get_authors_for_draft() bug,
source-aware analysis prompts, config env var overrides + validation,
resilient batch error handling with --retry-failed, observatory --dry-run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 20:52:56 +01:00
da2a989744 GDPR compliance: self-host all assets, add Impressum + Datenschutz
- Self-host Tailwind, Plotly, D3.js (no more CDN requests)
- Self-host Inter font (no more Google Fonts requests)
- Replace JetBrains Mono with system monospace font stack
- Zero external requests when visiting the site (GDPR-safe)
- Add /impressum page (§5 TMG, §18 MStV)
- Add /datenschutz page (DSGVO Art. 13/14)
- Hosting: Hetzner, no cookies, no tracking, no analytics
- Add Impressum/Datenschutz links to sidebar footer
- Configure Flask static_folder for local asset serving

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:20:52 +01:00
75c4da72e0 Fix broken reference links and web UI bugs
- Fix RFC URLs with leading zeros (rfc0020 -> rfc20) via int filter
- Draft refs: internal link for drafts in our DB, Datatracker for external
- BCP refs: link to rfc-editor.org/info/bcpN
- Add DB connection teardown (@app.teardown_appcontext)
- Fix JS syntax error in gap_demo.html (HTML-escaped string in script tag)
- Add URL encoding to all query params in drafts.html and draft_detail.html
- Fix variable shadowing of Flask's g import in gaps_demo()
- Add None safety for ideas search data attribute

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:59:45 +01:00
6e3a387778 Idea quality pipeline, web UI features, academic paper
- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing
  1,907 ideas to 468 across 434 drafts (78% reduction)
- Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity
- Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter)
  using Claude to rate ideas 1-5, removing 49 generic building blocks
- Final count: 419 high-quality ideas (avg 1.1/draft)
- Web UI: gap explorer with live draft generation and pre-generated demos
- Web UI: D3.js author collaboration network (498 nodes, 1142 edges,
  68 clusters, org filtering, interactive zoom/pan)
- Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft
  AI agent standards landscape
- Save improvement ideas backlog to data/reports/improvement-ideas.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:17:57 +01:00
3c3d7e649f Add ASCII art figures to all 6 generated Internet-Drafts
Each draft gets 2 illustrative figures:
- ABVP: architecture components + verification workflow
- ATD: example DAG structure + execution state transitions
- HITL: primitive framework overview + approval workflow sequence
- AEM/PPALP: federated learning architecture + aggregation flow
- RARP: cross-domain architecture + two-phase rollback protocol
- APAE: layered architecture + cross-domain provenance tracking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 02:20:39 +01:00
404092b938 Generate 5-draft ecosystem family, fix formatter markdown stripping
Pipeline output:
- ABVP: Agent Behavior Verification Protocol (quality 3.0/5)
- AEM: Privacy-Preserving Agent Learning Protocol (quality 2.1/5)
- ATD: Agent Task DAG Framework (quality 2.5/5)
- HITL: Human-in-the-Loop Primitives (quality 2.4/5)
- AEPB: Real-Time Agent Rollback Protocol (quality 2.5/5)
- APAE: Agent Provenance Assurance Ecosystem (quality 2.5/5)

Quality gates: all pass novelty + references, format gate improved
with markdown stripping (_strip_markdown) and dynamic header padding.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 01:42:30 +01:00
7a1aa346b9 Observatory update: 434 docs, fix W3C fetcher, regenerate dashboard
- Fixed W3C fetcher to paginate /specifications endpoint (group
  endpoints use type prefixes like cg/, wg/ that weren't in config)
- Fetched 72 new IETF drafts + 1 W3C spec, all analyzed and embedded
- Regenerated dashboard with updated data
- Total: 434 docs, 11 gaps, 1907 ideas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 01:09:30 +01:00
d6beb9c0a0 v0.3.0: Gap-to-Draft pipeline, Living Standards Observatory, blog series
Gap-to-Draft Pipeline (ietf pipeline):
- Context builder assembles ideas, RFC foundations, similar drafts, ecosystem vision
- Generator produces outlines + sections using rich context with Claude
- Quality gates: novelty (embedding similarity), references, format, self-rating
- Family coordinator generates 5-draft ecosystem (AEM/ATD/HITL/AEPB/APAE)
- I-D formatter with proper headers, references, 72-char wrapping

Living Standards Observatory (ietf observatory):
- Source abstraction with IETF + W3C fetchers
- 7-step update pipeline: snapshot, fetch, analyze, embed, ideas, gaps, record
- Static GitHub Pages dashboard (explorer, gap tracker, timeline)
- Weekly CI/CD automation via GitHub Actions

Also includes:
- 361 drafts (expanded from 260 with 6 new keywords), 403 authors, 1,262 ideas, 12 gaps
- Blog series (8 posts planned), reports, arXiv paper figures
- Agent team infrastructure (CLAUDE.md, scripts, dev journal)
- 5 new DB tables, schema migration, ~15 new query methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 00:48:57 +01:00
be9cf9c5d9 v0.2.0: visualizations, interactive browser, arXiv paper, gap analysis
New features:
- 12 interactive visualizations (ietf viz): t-SNE landscape, similarity
  heatmap, score distributions, timeline, bubble explorer, radar charts,
  author network graph, category treemap, quality vs overlap, org bar chart,
  ideas chart, and interactive draft browser
- Interactive draft browser (browser.html): filterable by category, keyword,
  score sliders with sortable table and expandable detail rows
- arXiv paper (paper/main.tex): 13-page manuscript with all findings
- Gap analysis: 12 identified under-addressed areas
- Author network: collaboration graph, org contributions, cross-org analysis
- Draft generation from gaps (ietf draft-gen)
- Auto-load .env for API keys (python-dotenv)

New modules: visualize.py, authors.py, draftgen.py
New reports: timeline, overlap-matrix, authors, gaps
New deps: plotly, matplotlib, seaborn, scipy, scikit-learn, networkx, python-dotenv

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 13:37:55 +01:00
f44f9265bd Add SQLite database with 260 analyzed drafts
Includes all draft metadata, full text, Claude ratings (cached),
and nomic-embed-text embeddings. This is the expensive data —
~114k tokens of Claude analysis + 260 Ollama embeddings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 00:49:18 +01:00
6771a4c235 IETF Draft Analyzer v0.1.0 — track, categorize, and rate AI/agent drafts
Python CLI tool that fetches AI/agent-related Internet-Drafts from the IETF
Datatracker, rates them using Claude, generates embeddings via Ollama for
similarity/clustering, and produces markdown reports.

Features:
- Fetch drafts by keyword from Datatracker API with full text download
- Batch analysis with Claude (token-optimized, responses cached in SQLite)
- Embedding-based similarity search and overlap cluster detection
- Reports: overview, landscape by category, overlap clusters, weekly digest
- SQLite with FTS5 for full-text search across 260 tracked drafts

Initial analysis of 260 drafts reveals OAuth agent auth (13 drafts) and
agent gateway/collaboration (10 drafts) as the most crowded clusters,
while AI safety/alignment is underserved with the highest quality scores.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 00:36:45 +01:00