Commit Graph

46 Commits

Author SHA1 Message Date
56f2ce669c feat: unified drafts/ structure with PDF outputs for ACT and ECT
Both drafts now live in workspace/drafts/ as siblings:
  drafts/
  ├── act/                       (ACT -01, native to parent repo)
  │   ├── draft-nennemann-act-01.md     kramdown-rfc source
  │   ├── draft-nennemann-act-01.{xml,txt,html,pdf}
  │   ├── .refcache/             bibxml cache
  │   └── build.sh
  ├── ietf-wimse-ect/            (ECT -02, submodule, PDF added)
  │   └── ...
  └── README-pdf.md              PDF toolchain docs

ACT kramdown-rfc conversion:
- full YAML frontmatter (title, author, refs)
- section structure matches kramdown-rfc conventions
- {{REF}} citation syntax, auto-numbered sections
- references auto-built from normative/informative blocks
- removed manual TOC (kramdown-rfc generates)
- builds cleanly: 133K XML, 89K TXT, 208K HTML, 167K PDF

PDF toolchain:
- xml2rfc --pdf via weasyprint<60 + pydyf<0.10 injected into xml2rfc pipx venv
- both build.sh scripts now produce PDF as Step 4
- README-pdf.md documents the setup for new machines

Submodule: bump ietf-wimse-ect pointer for build.sh PDF step
2026-04-12 14:01:57 +02:00
37859beef6 feat: interop test package + session handoff doc
Cross-spec interop validation between ietf-act and ietf-ect:
- new packages/interop/ sibling package (ietf-act-ect-interop)
- 32 tests pass: shared claims, algorithm matrix, DAG structure,
  divergence handling, anti-goals
- documents ES256 raw signature wire-compatibility
- documents airtight typ separation (act+jwt vs exec+jwt)

Hazards surfaced:
- ACTLedger.append() silently accepts ECT Payload via duck-typing
  (both have .jti) — documented in interop README as a production
  hazard requiring external isinstance checks

Session handoff:
- SESSION-2026-04-12.md — snapshot of decisions, artifacts, open
  actions, and next-session starting points

Also: session-end commit of hash-format fix propagation to
packages/ect/ (the fix was applied to the old refimpl location
but did not propagate through the parallel package-move agent).
2026-04-12 07:39:41 +02:00
3a139dfc7e feat: ACT/ECT strategy, package restructure, draft -01/-02 prep
Strategic work for IETF submission of draft-nennemann-act-01 and
draft-nennemann-wimse-ect-02:

Package restructure:
- move ACT and ECT refimpls to workspace/packages/{act,ect}/
- ietf-act and ietf-ect distribution names (sibling packages)
- cross-spec interop test plan (INTEROP-TEST-PLAN.md)

ACT draft -01 revisions:
- rename 'par' claim to 'pred' (align with ECT)
- rename 'Agent Compact Token' to 'Agent Context Token' (semantic
  alignment with ECT family)
- add Applicability section (MCP, OpenAI, LangGraph, A2A, CrewAI)
- add DAG vs Linear Delegation Chains section (differentiator vs
  txn-tokens-for-agents actchain, Agentic JWT, AIP/IBCTs)
- add Related Work: AIP, SentinelAgent, Agentic JWT, txn-tokens-for-agents,
  HDP, SCITT-AI-agent-execution
- pin SCITT arch to -22, note AUTH48 status

Outreach drafts:
- Emirdag liaison email (SCITT-AI coordination)
- OAuth ML response on txn-tokens-for-agents-06

Strategy document:
- STRATEGY.md with phased action plan, risk register, timeline

Submodule:
- update workspace/drafts/ietf-wimse-ect pointer to -02 commit
2026-04-12 07:33:08 +02:00
b38747ad92 fix: refimpl hash format aligned to -01 spec, draft rebuilt
Some checks failed
CI / test (3.11) (push) Failing after 53s
CI / test (3.12) (push) Failing after 41s
Updated Go ValidateHashFormat to plain base64url (no algorithm prefix)
per -01 spec and RFC 9449. Python already fixed. All tests pass.
2026-04-11 17:51:50 +02:00
2506b6325a feat: add draft data, gap analysis report, and workspace config
Some checks failed
CI / test (3.11) (push) Failing after 1m37s
CI / test (3.12) (push) Failing after 57s
2026-04-06 18:47:15 +02:00
4f310407b0 docs: assess ECT draft submission readiness, fix -00/-01 discrepancies
Some checks failed
CI / test (3.11) (push) Failing after 8s
CI / test (3.12) (push) Failing after 9s
Observatory Update / update (push) Failing after 10s
- Add docs/status.md with full draft quality assessment
- Fix blog post: update outdated claim names (par -> pred, ext -> ect_ext)
- Update refimpl README: add -00 vs -01 migration note with claim mapping
- Update IMPROVEMENTS.md: add section 6 with all -01 migration tasks
2026-04-03 07:46:45 +02:00
42b4546ded feat: refresh pipeline, reorganize draft generation, polish public pages
Some checks failed
CI / test (3.11) (push) Failing after 1m39s
CI / test (3.12) (push) Failing after 1m1s
Pipeline refresh:
- Extract ideas from 46 remaining drafts (844 total ideas now)
- Clear stale llm_cache entries blocking re-extraction
- Re-run gap analysis with expanded corpus: 10 gaps (was 12), fresh IDs #1-#10
- Re-link 3 proposals to new gap IDs
- Add scripts/pipeline-refresh.sh for reproducible runs

Draft generation moved from gaps to proposals:
- Remove "Generate Internet-Draft" section from gap detail page
- Add it to proposal detail page instead (proposals → generate I-D flow)
- New route POST /proposals/<id>/generate with richer context
  (proposal title + description + linked gap topics)
- Remove misleading "Search related drafts" link from gap page
  (related drafts already shown inline)

Public page polish:
- Overview: update subtitle to mention all 6 standards bodies
- About: describe multi-source scope (IETF, ISO, ITU-T, ETSI, NIST, W3C)
- About: add guiding question ("Where is the AI agent standards race heading?")
- Obsidian export button hidden in production mode (prior commit)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 05:39:13 +01:00
f8ed2b83e9 fix: security hardening — self-hosted JS, XSS protection, SSRF blocking
- Replace all CDN script tags (marked, plotly) with self-hosted static files
- Add DOMPurify for sanitizing markdown-rendered HTML
- Add escapeHtml() helper to base.html for all innerHTML operations
- Sanitize dynamic data in innerHTML across 13 templates
- Add security headers (X-Content-Type-Options, X-Frame-Options, Referrer-Policy)
- Add SSRF protection to proposal intake URL fetcher (block private/loopback IPs)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:47:32 +01:00
d1a20fa02e feat: add blog draft generator and fix broken routes
Blog drafting section (dev-only):
- BlogDraftGenerator gathers project data (gaps, proposals, stats) as
  context and calls Claude to produce Medium-style blog posts
- DB schema: blog_drafts table with title, content, tags, cost tracking
- Web UI: list, generate (async with live preview), detail (rendered +
  source toggle), edit, and export routes
- 6 writing styles: deep-dive, overview, opinion, listicle, comparison,
  series-post
- Nav link added to sidebar under Proposals

Bug fixes found via route testing (scripts/test_all_routes.py):
- /authors/<id>: Draft.status → Draft.states (correct attribute name)
- /false-positives: add missing `import re` in ratings.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:39:16 +01:00
2229e70c73 fix: deduplicate draft revisions in search results
Different revisions of the same draft (e.g. draft-foo-bar-00, -01, -02)
were showing up as separate results. Now keeps only the highest-scoring
revision per base draft name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:07:38 +01:00
cdb2e505c6 fix: restore SAFETY/CAPABILITY_CATEGORIES lost in data.py split
Constants were defined in the old monolithic data.py but not carried
over to data/analysis.py during the split, breaking /trends page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:04:19 +01:00
61cdab16b9 fix: dev mode auth regression from blueprint refactor
The _initialized singleton in auth.py prevented hooks from registering
on the correct app instance when create_app() was called twice (once
eagerly at import, once from __main__). Removed the guard and made
the module-level app lazy. Also adds feature backlog and architecture
assessment from the review team.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:52:02 +01:00
dea36e931a Add test coverage for CLI commands, Flask routes, and shared DB methods
74 new tests across 3 files:
- test_cli.py: CLI help, version, config, report generation, wg/viz/pipeline subcommands
- test_web.py: All public pages, admin pages (dev/prod), API JSON endpoints, CSV export, 404s
- test_db_shared.py: rated_count, gap_count, false_positive_names, category_counts, source_counts, draft_author_count_map, search_gaps, search_authors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:49:09 +01:00
c755b2bbf3 Add author detail, idea detail, and gap-draft reverse link pages
- Author detail page (/authors/<person_id>): shows author info, all drafts
  with ratings, and co-authors with shared draft counts. Public route.
- Idea detail page (/ideas/<idea_id>): shows idea metadata, source draft,
  and top-5 most similar ideas via embedding cosine similarity. Admin route.
- Gap detail page: added "Related Drafts" section that finds drafts by
  extracting draft names from evidence text and searching by topic keywords.
- Updated author links across templates to use /authors/<person_id> URLs.
- Added DB methods: get_author_by_id, get_author_drafts, get_coauthors.
- Extended top_authors to include person_id (5th tuple element).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:45:00 +01:00
4a368bde62 fix: update test_top_authors for new person_id return field
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:41:57 +01:00
b10572c628 Split cli.py (3,438 LOC) into modular command packages
Move 98 CLI commands from monolithic cli.py into organized modules:
- commands/common.py: shared utilities (console, pass_cfg_db, _get_config)
- commands/fetch.py: fetch, search, list, show, annotate, classify, authors, network
- commands/analysis.py: analyze, ask, compare, embed, ideas, gaps, refs, trends, etc.
- commands/reports.py: report group, viz group, wg group, export
- commands/admin.py: config, pipeline, observatory, monitor, auto-heal
- commands/proposals.py: draft-gen, intake

cli.py is now a slim 30-line entry point that registers all modules.
All command names, options, and behavior preserved exactly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:40:57 +01:00
3fb17100d7 Split webui into Flask blueprints and data domain modules
- Split app.py (66 routes) into 3 blueprints: pages (public), api (JSON), admin (@admin_required)
- Split data.py (4,360 LOC) into 7 domain modules: drafts, authors, ratings, gaps, analysis, search, proposals
- Add data/__init__.py re-exporting all public functions for backward compatibility
- Add custom 404/500 error pages matching dark theme
- Add request timing logging via before_request/after_request hooks
- Refactor app.py into create_app() factory pattern
- All 106 tests pass, all 66 routes preserved

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:37:15 +01:00
c066b04d74 Add DB indexes and extract shared query methods to Database class
Add missing indexes on ratings(false_positive), drafts(source), and
draft_authors(person_id) for faster filtering. Extract 12 shared query
methods (false_positive_drafts_raw, non_false_positive_ratings_raw,
false_positive_names, rated_count, gap_count, search_gaps, search_authors,
draft_affiliation_pairs, all_persons_info, category_counts,
draft_author_count_map, source_counts) to eliminate duplicated SQL across
cli.py, data.py, and reports.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:33:18 +01:00
4710668419 chore: commit current DB and add SQL dump export/import scripts
Include data/drafts.db so other machines don't need to re-run
expensive Claude API calls (~$3+ of analysis, 474 drafts, 403 authors,
1262 ideas, 12 gaps). Add scripts/db-export.sh and scripts/db-import.sh
for portable compressed SQL dump sharing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:21:00 +01:00
5ec7410b89 feat: proposal intake pipeline with AI-powered generation on /proposals/new
Add full proposal system: DB schema (proposals + proposal_gaps tables),
CLI `ietf intake` command, and web UI with Quick Generate on /proposals/new.
The new page merges AI intake (paste URL/text → Haiku generates multiple
proposals auto-linked to gaps) with manual form entry. Generated proposals
are clickable cards that fill the editor below for refinement.

Uses claude_model_cheap (Haiku) for cost-efficient web intake. Includes
CaML-inspired draft proposals from arXiv:2503.18813 analysis.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:15:11 +01:00
ae5e5f8cbf Enforce public/private visibility for web UI pages
Dev-only pages (sources, trends, complexity, idea-analysis, false-positives,
similarity, landscape, export) now require @admin_required and are hidden
from nav in production mode. Citations page keeps the graph public but
hides influence/BCP tabs behind --dev flag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:52:43 +01:00
dec8667193 Add 6 new analysis pages and 5 CLI reports
New web UI pages with Plotly charts:
- /sources: cross-source comparison (ratings, categories by standards body)
- /false-positives: profiling of 73 false positives (box plots, terms)
- /trends: temporal evolution (submissions, ratings, safety ratio over time)
- /complexity: draft complexity matrix (correlations, scatter plots)
- /idea-analysis: idea novelty deep dive (sunburst, distribution, shared ideas)
- /citations: enhanced with influence analysis and BCP dependency tabs

New CLI reports (ietf report <name>):
- sources, false-positives, citations, complexity, idea-analysis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:35:32 +01:00
8515e46d5d Architecture designer, author cluster names, FP filtering, new pages
- Add /architecture page: system-of-systems view with 8 layers, component
  cards, gap markers, source coverage chart, and clickable detail sidebar
- Give author clusters meaningful names from orgs + draft topic keywords
- Filter false positives (73 drafts, 54 ideas) from idea clusters,
  architecture, ideas listing, and search results
- Add NIST source fetcher with curated catalog of 11 AI publications
- New pages: trends, complexity, sources, false positives, idea analysis
- Clickable gap cards with full details (evidence, priority, nearby work)
- Component detail panel with linked drafts and top ideas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 19:58:40 +01:00
a46a01bd8c Add auto-heal pipeline command and fix multi-source draft processing
- Add `ietf auto` command: fetches, analyzes, embeds, extracts ideas,
  and refreshes gaps across all sources with cost-based auto-approval
- Fix SourceDocument→Draft conversion in auto fetch step
- Fix gap_analysis method name in auto command
- Process all 270 unrated ETSI/ISO/ITU/NIST drafts (761 total, all rated)
- Update web UI templates and data layer for multi-source support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 18:41:42 +01:00
1ec1f69bee v0.3.0: Publication-ready release with blog site, paper update, and polish
Release prep:
- Version bump to 0.3.0 (pyproject.toml, cli.py)
- Rewrite README.md with current stats (475 drafts, 713 authors, 501 ideas)
- Add CONTRIBUTING.md with dev setup and code conventions

Blog site:
- Add scripts/build-site.py (markdown → HTML with clean CSS, dark mode, nav)
- Generate static site in docs/blog/ (10 pages)
- Ready for GitHub Pages deployment

Academic paper (paper/main.tex):
- Update all counts: 474→475 drafts, 557→710 authors, 1907→462 ideas, 11→12 gaps
- Add false-positive filtering methodology (113 excluded, 361 relevant)
- Add cross-org convergence analysis (132 ideas, 33% rate)
- Add GDPR compliance gap to gap table
- Add LLM-as-judge caveats to rating methodology and limitations
- Add FIPA, IEEE P3394, W3C WoT to related work with bibliography entries
- Fix safety ratio to show monthly variation (1.5:1 to 21:1)

Pipeline:
- Fetch 1 new draft (475 total), 3 new authors (713 total)
- Fix 16 ruff lint errors across test files
- All 106 tests pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 17:54:43 +01:00
e247bfef8f Run pipeline, write Post 08, commit untracked files
Pipeline:
- Extract ideas for 38 new drafts → 462 ideas total
- Convergence analysis: 132 cross-org convergent ideas (33% rate)
- Fetch authors for 102 drafts → 709 authors (up from 403)
- Refresh gap analysis: 12 gaps across full 474-draft corpus
- Update verified counts with new totals

Post 08:
- Complete rewrite of "Agents Building the Agent Analysis" (2,953 words)
- Covers 3 phases: writing team → review cycle → fix cycle
- Meta-irony table mapping team coordination to IETF gap names
- Specific examples from dev journal (SQL injection, consent conflation, ideas mismatch)

Untracked files committed:
- scripts/: backfill-wg-names, classify-unrated, compare-classifiers, download-relevant-text, run-webui
- src/ietf_analyzer/classifier.py: two-stage Ollama classifier
- src/webui/: analytics (GDPR-compliant), auth, obsidian_export
- tests/test_obsidian_export.py (10 tests)
- data/reports/: wg-analysis, generated draft for gap #37

Housekeeping:
- .gitignore: exclude LaTeX artifacts, stale DBs, analytics.db

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:31:30 +01:00
20c45a7eba Complete remaining medium/low issues: performance, CLI, types, CI, tests
Performance:
- Batch readiness computation (~200 queries → ~6 per page)
- Batch draft lookup in author network (N+1 → single query)
- File-based similarity matrix cache (.npy + metadata sidecar)
- 5-minute TTL embedding cache for search queries

CLI quality:
- Add pass_cfg_db decorator, convert ~30 commands to shared config/db lifecycle
- Add --dry-run to analyze, embed, embed-ideas, ideas, gaps commands
- Move 15+ in-function imports to top of data.py

Types & documentation:
- Add 16 TypedDicts to data.py, annotate 12 function return types
- Add ethics section to Post 06 (premature standardization, power asymmetry)
- Add EU AI Act Article 43 conformity mapping to Post 06
- Add NIS2 and CRA references to Post 04

CI & testing:
- Add GitHub Actions CI workflow (Python 3.11+3.12, ruff, pytest)
- Add API documentation for all 20 endpoints (data/reports/api-docs.md)
- Add 41 new tests (test_analyzer.py, test_search.py) — 64 total pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:06:54 +01:00
e7527ad68e Fix remaining critical, high, and medium issues from 4-perspective review
Critical fixes:
- Fix rating clamp range 1-10 → 1-5 (actual scale)
- Add `ietf ideas convergence` command (SequenceMatcher at 0.75 threshold)
- Fix "628 cross-org ideas" → 130 (verified from current DB) across 8 files

Security fixes:
- Sanitize FTS5 query input (strip special chars + boolean operators)
- Add rate limiting (10 req/min/IP) on Claude-calling endpoints
- Change <path:name> → <string:name> on draft routes

Codebase fixes:
- Add Database context manager (__enter__/__exit__)
- Wire false_positive filtering into queries (exclude by default in web UI)
- Fix Post 3 arithmetic ("~300" → "~409" distinct proposals)

Content & licensing:
- Add MIT LICENSE file
- Add IPR/FRAND notes (BCP 79, RFC 8179) to Posts 03 and 07
- Qualify "4:1 safety ratio" with monthly variation in 6 remaining files
- Add "Data as of March 2026" freeze-date headers to all 10 blog posts
- Hedge causal language in Post 04

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 12:47:47 +01:00
f1a0b0264c Fix blog accuracy and add methodology documentation
Blog posts (all 10 files updated):
- Update all counts to match DB: 434 drafts, 557 authors, 419 ideas, 11 gaps
- Fix EU AI Act timeline to August 2026 (5 months, not 18)
- Reframe growth claim from "36x" to actual monthly figures (5→61→85)
- Add safety ratio nuance (1.5:1 to 21:1 monthly variation)
- Fix composite scores (4.8→4.75, 4.6→4.5)
- Add OAuth/GDPR consent distinction (Art. 6(1)(a), Art. 28)
- Add EU AI Act Annex III + MDR context to hospital scenario
- Add FIPA, IEEE P3394, eIDAS 2.0 references
- Add GDPR gap paragraph (DPIA, erasure, portability, purpose limitation)
- Rewrite Post 04 gap table to match actual DB gap names

Methodology:
- Expand methodology.md: pipeline docs, limitations, related work
- Add LLM-as-judge caveats and explicit rating rubric to analyzer.py
- Add clustering threshold rationale to embeddings.py
- Add gap analysis grounding notes to analyzer.py
- Add Limitations section to Post 07

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 11:04:40 +01:00
439424bd04 Fix security, data integrity, and accuracy issues from 4-perspective review
Security fixes:
- Fix SQL injection in db.py:update_generation_run (column name whitelist)
- Flask SECRET_KEY from env var instead of hardcoded
- Add LLM rating bounds validation (_clamp_rating, 1-10)
- Fix JSON extraction trailing whitespace handling

Data integrity:
- Normalize 21 legacy category names to 11 canonical short forms
- Add false_positive column, flag 73 non-AI drafts (361 relevant remain)
- Document verified counts: 434 total/361 relevant drafts, 557 authors, 419 ideas, 11 gaps

Code quality:
- Fix version string 0.1.0 → 0.2.0
- Add close()/context manager to Embedder class
- Dynamic matrix size instead of hardcoded "260x260"

Blog accuracy:
- Fix EU AI Act timeline (enforcement Aug 2026, not "18 months")
- Distinguish OAuth consent from GDPR Einwilligung
- Add EU AI Act Annex III context to hospital scenario
- Add FIPA, eIDAS 2.0 references where relevant

Methodology:
- Add methodology.md documenting pipeline, limitations, rating rubric
- Add LLM-as-judge caveats to analyzer.py
- Document clustering threshold rationale

Reviews from: legal (German/EU law), statistics, development, science perspectives.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 10:52:33 +01:00
a386d0bb1a Add explanation of how author clusters are formed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:27:23 +01:00
02049c37a8 Show related drafts in author cluster cards
Each cluster card now shows draft count badge and up to 5 linked draft
titles with clickable links to draft detail pages. Data collected from
member nodes' draft lists during cluster detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:23:13 +01:00
34c36f81f1 Make /ask free by default, Claude synthesis is opt-in
Search results (FTS5 + Ollama embeddings) are shown immediately at no
cost. AI synthesis via Claude is behind a "Synthesize" button that the
user must explicitly click. Results are cached permanently so repeat
visitors never trigger API calls.

- Split ask into search_only() (free) and ask() (paid, cached)
- GET /ask now uses search_only — no Claude tokens spent
- POST /api/ask/synthesize triggers Claude (Haiku, ~$0.001)
- Cached answers shown with "cached" badge, no re-generation
- Template shows sources immediately + optional synthesize button

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:15:07 +01:00
e65d1cfacd Fix drafts search to include author names
Clicking an author in the network graph opens /drafts?q=AuthorName
but search only checked draft name/title/summary. Now also matches
against author names via a JOIN on draft_authors + authors tables.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:06:26 +01:00
3e36802500 Fix author network graph: tune force sim, reduce node count
- Raise co-authorship threshold from 1 to 2 shared drafts (498→156 nodes)
- Tune D3 force parameters for large graphs: capped link strength,
  wider distance, adaptive charge, lower velocity decay
- Add initial circular layout to prevent explosion on load
- Fix cluster highlighting with fallback name matching and
  position validation before zoom-to-fit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:04:18 +01:00
757b781c67 Platform upgrade: semantic search, citations, readiness, tests, Docker
Major features added by 5 parallel agent teams:
- Semantic "Ask" (NL queries via FTS5 + embeddings + Claude synthesis)
- Global search across drafts, ideas, authors, gaps
- REST API expansion (14 endpoints, up from 3) with CSV/JSON export
- Citation graph visualization (D3.js, 440 nodes, 2422 edges)
- Standards readiness scoring (0-100 composite from 6 factors)
- Side-by-side draft comparison view with shared/unique analysis
- Annotation system (notes + tags per draft, DB-persisted)
- Docker deployment (Dockerfile + docker-compose with Ollama)
- Scheduled updates (cron script with log rotation)
- Pipeline health dashboard (stage progress bars, cost tracking)
- Test suite foundation (54 pytest tests covering DB, models, web data)

Fixes: compare_drafts() stubbed→working, get_authors_for_draft() bug,
source-aware analysis prompts, config env var overrides + validation,
resilient batch error handling with --retry-failed, observatory --dry-run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 20:52:56 +01:00
da2a989744 GDPR compliance: self-host all assets, add Impressum + Datenschutz
- Self-host Tailwind, Plotly, D3.js (no more CDN requests)
- Self-host Inter font (no more Google Fonts requests)
- Replace JetBrains Mono with system monospace font stack
- Zero external requests when visiting the site (GDPR-safe)
- Add /impressum page (§5 TMG, §18 MStV)
- Add /datenschutz page (DSGVO Art. 13/14)
- Hosting: Hetzner, no cookies, no tracking, no analytics
- Add Impressum/Datenschutz links to sidebar footer
- Configure Flask static_folder for local asset serving

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:20:52 +01:00
75c4da72e0 Fix broken reference links and web UI bugs
- Fix RFC URLs with leading zeros (rfc0020 -> rfc20) via int filter
- Draft refs: internal link for drafts in our DB, Datatracker for external
- BCP refs: link to rfc-editor.org/info/bcpN
- Add DB connection teardown (@app.teardown_appcontext)
- Fix JS syntax error in gap_demo.html (HTML-escaped string in script tag)
- Add URL encoding to all query params in drafts.html and draft_detail.html
- Fix variable shadowing of Flask's g import in gaps_demo()
- Add None safety for ideas search data attribute

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:59:45 +01:00
6e3a387778 Idea quality pipeline, web UI features, academic paper
- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing
  1,907 ideas to 468 across 434 drafts (78% reduction)
- Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity
- Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter)
  using Claude to rate ideas 1-5, removing 49 generic building blocks
- Final count: 419 high-quality ideas (avg 1.1/draft)
- Web UI: gap explorer with live draft generation and pre-generated demos
- Web UI: D3.js author collaboration network (498 nodes, 1142 edges,
  68 clusters, org filtering, interactive zoom/pan)
- Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft
  AI agent standards landscape
- Save improvement ideas backlog to data/reports/improvement-ideas.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:17:57 +01:00
3c3d7e649f Add ASCII art figures to all 6 generated Internet-Drafts
Each draft gets 2 illustrative figures:
- ABVP: architecture components + verification workflow
- ATD: example DAG structure + execution state transitions
- HITL: primitive framework overview + approval workflow sequence
- AEM/PPALP: federated learning architecture + aggregation flow
- RARP: cross-domain architecture + two-phase rollback protocol
- APAE: layered architecture + cross-domain provenance tracking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 02:20:39 +01:00
404092b938 Generate 5-draft ecosystem family, fix formatter markdown stripping
Pipeline output:
- ABVP: Agent Behavior Verification Protocol (quality 3.0/5)
- AEM: Privacy-Preserving Agent Learning Protocol (quality 2.1/5)
- ATD: Agent Task DAG Framework (quality 2.5/5)
- HITL: Human-in-the-Loop Primitives (quality 2.4/5)
- AEPB: Real-Time Agent Rollback Protocol (quality 2.5/5)
- APAE: Agent Provenance Assurance Ecosystem (quality 2.5/5)

Quality gates: all pass novelty + references, format gate improved
with markdown stripping (_strip_markdown) and dynamic header padding.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 01:42:30 +01:00
7a1aa346b9 Observatory update: 434 docs, fix W3C fetcher, regenerate dashboard
- Fixed W3C fetcher to paginate /specifications endpoint (group
  endpoints use type prefixes like cg/, wg/ that weren't in config)
- Fetched 72 new IETF drafts + 1 W3C spec, all analyzed and embedded
- Regenerated dashboard with updated data
- Total: 434 docs, 11 gaps, 1907 ideas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 01:09:30 +01:00
d6beb9c0a0 v0.3.0: Gap-to-Draft pipeline, Living Standards Observatory, blog series
Gap-to-Draft Pipeline (ietf pipeline):
- Context builder assembles ideas, RFC foundations, similar drafts, ecosystem vision
- Generator produces outlines + sections using rich context with Claude
- Quality gates: novelty (embedding similarity), references, format, self-rating
- Family coordinator generates 5-draft ecosystem (AEM/ATD/HITL/AEPB/APAE)
- I-D formatter with proper headers, references, 72-char wrapping

Living Standards Observatory (ietf observatory):
- Source abstraction with IETF + W3C fetchers
- 7-step update pipeline: snapshot, fetch, analyze, embed, ideas, gaps, record
- Static GitHub Pages dashboard (explorer, gap tracker, timeline)
- Weekly CI/CD automation via GitHub Actions

Also includes:
- 361 drafts (expanded from 260 with 6 new keywords), 403 authors, 1,262 ideas, 12 gaps
- Blog series (8 posts planned), reports, arXiv paper figures
- Agent team infrastructure (CLAUDE.md, scripts, dev journal)
- 5 new DB tables, schema migration, ~15 new query methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 00:48:57 +01:00
be9cf9c5d9 v0.2.0: visualizations, interactive browser, arXiv paper, gap analysis
New features:
- 12 interactive visualizations (ietf viz): t-SNE landscape, similarity
  heatmap, score distributions, timeline, bubble explorer, radar charts,
  author network graph, category treemap, quality vs overlap, org bar chart,
  ideas chart, and interactive draft browser
- Interactive draft browser (browser.html): filterable by category, keyword,
  score sliders with sortable table and expandable detail rows
- arXiv paper (paper/main.tex): 13-page manuscript with all findings
- Gap analysis: 12 identified under-addressed areas
- Author network: collaboration graph, org contributions, cross-org analysis
- Draft generation from gaps (ietf draft-gen)
- Auto-load .env for API keys (python-dotenv)

New modules: visualize.py, authors.py, draftgen.py
New reports: timeline, overlap-matrix, authors, gaps
New deps: plotly, matplotlib, seaborn, scipy, scikit-learn, networkx, python-dotenv

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 13:37:55 +01:00
f44f9265bd Add SQLite database with 260 analyzed drafts
Includes all draft metadata, full text, Claude ratings (cached),
and nomic-embed-text embeddings. This is the expensive data —
~114k tokens of Claude analysis + 260 Ollama embeddings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 00:49:18 +01:00
6771a4c235 IETF Draft Analyzer v0.1.0 — track, categorize, and rate AI/agent drafts
Python CLI tool that fetches AI/agent-related Internet-Drafts from the IETF
Datatracker, rates them using Claude, generates embeddings via Ollama for
similarity/clustering, and produces markdown reports.

Features:
- Fetch drafts by keyword from Datatracker API with full text download
- Batch analysis with Claude (token-optimized, responses cached in SQLite)
- Embedding-based similarity search and overlap cluster detection
- Reports: overview, landscape by category, overlap clusters, weekly digest
- SQLite with FTS5 for full-text search across 260 tracked drafts

Initial analysis of 260 drafts reveals OAuth agent auth (13 drafts) and
agent gateway/collaboration (10 drafts) as the most crowded clusters,
while AI safety/alignment is underserved with the highest quality scores.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 00:36:45 +01:00