Commit Graph

11 Commits

Author SHA1 Message Date
a46a01bd8c Add auto-heal pipeline command and fix multi-source draft processing
- Add `ietf auto` command: fetches, analyzes, embeds, extracts ideas,
  and refreshes gaps across all sources with cost-based auto-approval
- Fix SourceDocument→Draft conversion in auto fetch step
- Fix gap_analysis method name in auto command
- Process all 270 unrated ETSI/ISO/ITU/NIST drafts (761 total, all rated)
- Update web UI templates and data layer for multi-source support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 18:41:42 +01:00
1ec1f69bee v0.3.0: Publication-ready release with blog site, paper update, and polish
Release prep:
- Version bump to 0.3.0 (pyproject.toml, cli.py)
- Rewrite README.md with current stats (475 drafts, 713 authors, 501 ideas)
- Add CONTRIBUTING.md with dev setup and code conventions

Blog site:
- Add scripts/build-site.py (markdown → HTML with clean CSS, dark mode, nav)
- Generate static site in docs/blog/ (10 pages)
- Ready for GitHub Pages deployment

Academic paper (paper/main.tex):
- Update all counts: 474→475 drafts, 557→710 authors, 1907→462 ideas, 11→12 gaps
- Add false-positive filtering methodology (113 excluded, 361 relevant)
- Add cross-org convergence analysis (132 ideas, 33% rate)
- Add GDPR compliance gap to gap table
- Add LLM-as-judge caveats to rating methodology and limitations
- Add FIPA, IEEE P3394, W3C WoT to related work with bibliography entries
- Fix safety ratio to show monthly variation (1.5:1 to 21:1)

Pipeline:
- Fetch 1 new draft (475 total), 3 new authors (713 total)
- Fix 16 ruff lint errors across test files
- All 106 tests pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 17:54:43 +01:00
e7527ad68e Fix remaining critical, high, and medium issues from 4-perspective review
Critical fixes:
- Fix rating clamp range 1-10 → 1-5 (actual scale)
- Add `ietf ideas convergence` command (SequenceMatcher at 0.75 threshold)
- Fix "628 cross-org ideas" → 130 (verified from current DB) across 8 files

Security fixes:
- Sanitize FTS5 query input (strip special chars + boolean operators)
- Add rate limiting (10 req/min/IP) on Claude-calling endpoints
- Change <path:name> → <string:name> on draft routes

Codebase fixes:
- Add Database context manager (__enter__/__exit__)
- Wire false_positive filtering into queries (exclude by default in web UI)
- Fix Post 3 arithmetic ("~300" → "~409" distinct proposals)

Content & licensing:
- Add MIT LICENSE file
- Add IPR/FRAND notes (BCP 79, RFC 8179) to Posts 03 and 07
- Qualify "4:1 safety ratio" with monthly variation in 6 remaining files
- Add "Data as of March 2026" freeze-date headers to all 10 blog posts
- Hedge causal language in Post 04

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 12:47:47 +01:00
a386d0bb1a Add explanation of how author clusters are formed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:27:23 +01:00
02049c37a8 Show related drafts in author cluster cards
Each cluster card now shows draft count badge and up to 5 linked draft
titles with clickable links to draft detail pages. Data collected from
member nodes' draft lists during cluster detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:23:13 +01:00
34c36f81f1 Make /ask free by default, Claude synthesis is opt-in
Search results (FTS5 + Ollama embeddings) are shown immediately at no
cost. AI synthesis via Claude is behind a "Synthesize" button that the
user must explicitly click. Results are cached permanently so repeat
visitors never trigger API calls.

- Split ask into search_only() (free) and ask() (paid, cached)
- GET /ask now uses search_only — no Claude tokens spent
- POST /api/ask/synthesize triggers Claude (Haiku, ~$0.001)
- Cached answers shown with "cached" badge, no re-generation
- Template shows sources immediately + optional synthesize button

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:15:07 +01:00
3e36802500 Fix author network graph: tune force sim, reduce node count
- Raise co-authorship threshold from 1 to 2 shared drafts (498→156 nodes)
- Tune D3 force parameters for large graphs: capped link strength,
  wider distance, adaptive charge, lower velocity decay
- Add initial circular layout to prevent explosion on load
- Fix cluster highlighting with fallback name matching and
  position validation before zoom-to-fit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:04:18 +01:00
757b781c67 Platform upgrade: semantic search, citations, readiness, tests, Docker
Major features added by 5 parallel agent teams:
- Semantic "Ask" (NL queries via FTS5 + embeddings + Claude synthesis)
- Global search across drafts, ideas, authors, gaps
- REST API expansion (14 endpoints, up from 3) with CSV/JSON export
- Citation graph visualization (D3.js, 440 nodes, 2422 edges)
- Standards readiness scoring (0-100 composite from 6 factors)
- Side-by-side draft comparison view with shared/unique analysis
- Annotation system (notes + tags per draft, DB-persisted)
- Docker deployment (Dockerfile + docker-compose with Ollama)
- Scheduled updates (cron script with log rotation)
- Pipeline health dashboard (stage progress bars, cost tracking)
- Test suite foundation (54 pytest tests covering DB, models, web data)

Fixes: compare_drafts() stubbed→working, get_authors_for_draft() bug,
source-aware analysis prompts, config env var overrides + validation,
resilient batch error handling with --retry-failed, observatory --dry-run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 20:52:56 +01:00
da2a989744 GDPR compliance: self-host all assets, add Impressum + Datenschutz
- Self-host Tailwind, Plotly, D3.js (no more CDN requests)
- Self-host Inter font (no more Google Fonts requests)
- Replace JetBrains Mono with system monospace font stack
- Zero external requests when visiting the site (GDPR-safe)
- Add /impressum page (§5 TMG, §18 MStV)
- Add /datenschutz page (DSGVO Art. 13/14)
- Hosting: Hetzner, no cookies, no tracking, no analytics
- Add Impressum/Datenschutz links to sidebar footer
- Configure Flask static_folder for local asset serving

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:20:52 +01:00
75c4da72e0 Fix broken reference links and web UI bugs
- Fix RFC URLs with leading zeros (rfc0020 -> rfc20) via int filter
- Draft refs: internal link for drafts in our DB, Datatracker for external
- BCP refs: link to rfc-editor.org/info/bcpN
- Add DB connection teardown (@app.teardown_appcontext)
- Fix JS syntax error in gap_demo.html (HTML-escaped string in script tag)
- Add URL encoding to all query params in drafts.html and draft_detail.html
- Fix variable shadowing of Flask's g import in gaps_demo()
- Add None safety for ideas search data attribute

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:59:45 +01:00
6e3a387778 Idea quality pipeline, web UI features, academic paper
- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing
  1,907 ideas to 468 across 434 drafts (78% reduction)
- Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity
- Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter)
  using Claude to rate ideas 1-5, removing 49 generic building blocks
- Final count: 419 high-quality ideas (avg 1.1/draft)
- Web UI: gap explorer with live draft generation and pre-generated demos
- Web UI: D3.js author collaboration network (498 nodes, 1142 edges,
  68 clusters, org filtering, interactive zoom/pan)
- Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft
  AI agent standards landscape
- Save improvement ideas backlog to data/reports/improvement-ideas.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:17:57 +01:00