Commit Graph

9 Commits

Author SHA1 Message Date
42b4546ded feat: refresh pipeline, reorganize draft generation, polish public pages
Some checks failed
CI / test (3.11) (push) Failing after 1m39s
CI / test (3.12) (push) Failing after 1m1s
Pipeline refresh:
- Extract ideas from 46 remaining drafts (844 total ideas now)
- Clear stale llm_cache entries blocking re-extraction
- Re-run gap analysis with expanded corpus: 10 gaps (was 12), fresh IDs #1-#10
- Re-link 3 proposals to new gap IDs
- Add scripts/pipeline-refresh.sh for reproducible runs

Draft generation moved from gaps to proposals:
- Remove "Generate Internet-Draft" section from gap detail page
- Add it to proposal detail page instead (proposals → generate I-D flow)
- New route POST /proposals/<id>/generate with richer context
  (proposal title + description + linked gap topics)
- Remove misleading "Search related drafts" link from gap page
  (related drafts already shown inline)

Public page polish:
- Overview: update subtitle to mention all 6 standards bodies
- About: describe multi-source scope (IETF, ISO, ITU-T, ETSI, NIST, W3C)
- About: add guiding question ("Where is the AI agent standards race heading?")
- Obsidian export button hidden in production mode (prior commit)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 05:39:13 +01:00
d1a20fa02e feat: add blog draft generator and fix broken routes
Blog drafting section (dev-only):
- BlogDraftGenerator gathers project data (gaps, proposals, stats) as
  context and calls Claude to produce Medium-style blog posts
- DB schema: blog_drafts table with title, content, tags, cost tracking
- Web UI: list, generate (async with live preview), detail (rendered +
  source toggle), edit, and export routes
- 6 writing styles: deep-dive, overview, opinion, listicle, comparison,
  series-post
- Nav link added to sidebar under Proposals

Bug fixes found via route testing (scripts/test_all_routes.py):
- /authors/<id>: Draft.status → Draft.states (correct attribute name)
- /false-positives: add missing `import re` in ratings.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:39:16 +01:00
4710668419 chore: commit current DB and add SQL dump export/import scripts
Include data/drafts.db so other machines don't need to re-run
expensive Claude API calls (~$3+ of analysis, 474 drafts, 403 authors,
1262 ideas, 12 gaps). Add scripts/db-export.sh and scripts/db-import.sh
for portable compressed SQL dump sharing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:21:00 +01:00
5ec7410b89 feat: proposal intake pipeline with AI-powered generation on /proposals/new
Add full proposal system: DB schema (proposals + proposal_gaps tables),
CLI `ietf intake` command, and web UI with Quick Generate on /proposals/new.
The new page merges AI intake (paste URL/text → Haiku generates multiple
proposals auto-linked to gaps) with manual form entry. Generated proposals
are clickable cards that fill the editor below for refinement.

Uses claude_model_cheap (Haiku) for cost-efficient web intake. Includes
CaML-inspired draft proposals from arXiv:2503.18813 analysis.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:15:11 +01:00
a46a01bd8c Add auto-heal pipeline command and fix multi-source draft processing
- Add `ietf auto` command: fetches, analyzes, embeds, extracts ideas,
  and refreshes gaps across all sources with cost-based auto-approval
- Fix SourceDocument→Draft conversion in auto fetch step
- Fix gap_analysis method name in auto command
- Process all 270 unrated ETSI/ISO/ITU/NIST drafts (761 total, all rated)
- Update web UI templates and data layer for multi-source support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 18:41:42 +01:00
1ec1f69bee v0.3.0: Publication-ready release with blog site, paper update, and polish
Release prep:
- Version bump to 0.3.0 (pyproject.toml, cli.py)
- Rewrite README.md with current stats (475 drafts, 713 authors, 501 ideas)
- Add CONTRIBUTING.md with dev setup and code conventions

Blog site:
- Add scripts/build-site.py (markdown → HTML with clean CSS, dark mode, nav)
- Generate static site in docs/blog/ (10 pages)
- Ready for GitHub Pages deployment

Academic paper (paper/main.tex):
- Update all counts: 474→475 drafts, 557→710 authors, 1907→462 ideas, 11→12 gaps
- Add false-positive filtering methodology (113 excluded, 361 relevant)
- Add cross-org convergence analysis (132 ideas, 33% rate)
- Add GDPR compliance gap to gap table
- Add LLM-as-judge caveats to rating methodology and limitations
- Add FIPA, IEEE P3394, W3C WoT to related work with bibliography entries
- Fix safety ratio to show monthly variation (1.5:1 to 21:1)

Pipeline:
- Fetch 1 new draft (475 total), 3 new authors (713 total)
- Fix 16 ruff lint errors across test files
- All 106 tests pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 17:54:43 +01:00
e247bfef8f Run pipeline, write Post 08, commit untracked files
Pipeline:
- Extract ideas for 38 new drafts → 462 ideas total
- Convergence analysis: 132 cross-org convergent ideas (33% rate)
- Fetch authors for 102 drafts → 709 authors (up from 403)
- Refresh gap analysis: 12 gaps across full 474-draft corpus
- Update verified counts with new totals

Post 08:
- Complete rewrite of "Agents Building the Agent Analysis" (2,953 words)
- Covers 3 phases: writing team → review cycle → fix cycle
- Meta-irony table mapping team coordination to IETF gap names
- Specific examples from dev journal (SQL injection, consent conflation, ideas mismatch)

Untracked files committed:
- scripts/: backfill-wg-names, classify-unrated, compare-classifiers, download-relevant-text, run-webui
- src/ietf_analyzer/classifier.py: two-stage Ollama classifier
- src/webui/: analytics (GDPR-compliant), auth, obsidian_export
- tests/test_obsidian_export.py (10 tests)
- data/reports/: wg-analysis, generated draft for gap #37

Housekeeping:
- .gitignore: exclude LaTeX artifacts, stale DBs, analytics.db

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:31:30 +01:00
757b781c67 Platform upgrade: semantic search, citations, readiness, tests, Docker
Major features added by 5 parallel agent teams:
- Semantic "Ask" (NL queries via FTS5 + embeddings + Claude synthesis)
- Global search across drafts, ideas, authors, gaps
- REST API expansion (14 endpoints, up from 3) with CSV/JSON export
- Citation graph visualization (D3.js, 440 nodes, 2422 edges)
- Standards readiness scoring (0-100 composite from 6 factors)
- Side-by-side draft comparison view with shared/unique analysis
- Annotation system (notes + tags per draft, DB-persisted)
- Docker deployment (Dockerfile + docker-compose with Ollama)
- Scheduled updates (cron script with log rotation)
- Pipeline health dashboard (stage progress bars, cost tracking)
- Test suite foundation (54 pytest tests covering DB, models, web data)

Fixes: compare_drafts() stubbed→working, get_authors_for_draft() bug,
source-aware analysis prompts, config env var overrides + validation,
resilient batch error handling with --retry-failed, observatory --dry-run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 20:52:56 +01:00
d6beb9c0a0 v0.3.0: Gap-to-Draft pipeline, Living Standards Observatory, blog series
Gap-to-Draft Pipeline (ietf pipeline):
- Context builder assembles ideas, RFC foundations, similar drafts, ecosystem vision
- Generator produces outlines + sections using rich context with Claude
- Quality gates: novelty (embedding similarity), references, format, self-rating
- Family coordinator generates 5-draft ecosystem (AEM/ATD/HITL/AEPB/APAE)
- I-D formatter with proper headers, references, 72-char wrapping

Living Standards Observatory (ietf observatory):
- Source abstraction with IETF + W3C fetchers
- 7-step update pipeline: snapshot, fetch, analyze, embed, ideas, gaps, record
- Static GitHub Pages dashboard (explorer, gap tracker, timeline)
- Weekly CI/CD automation via GitHub Actions

Also includes:
- 361 drafts (expanded from 260 with 6 new keywords), 403 authors, 1,262 ideas, 12 gaps
- Blog series (8 posts planned), reports, arXiv paper figures
- Agent team infrastructure (CLAUDE.md, scripts, dev journal)
- 5 new DB tables, schema migration, ~15 new query methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 00:48:57 +01:00