Commit Graph

49 Commits

Author SHA1 Message Date
316fdefcd7 fix: rename ACT from "Compact" to "Context" token, add pyproject.toml and PoC plan
Corrects the ACT acronym expansion to "Agent Context Token" in the
reference implementation. Adds proper pyproject.toml for the act package
and the MCP+LangGraph PoC planning document.
2026-04-12 12:43:31 +00:00
9a0dc899a8 feat: add ACT+ECT over MCP demo with LangGraph agent
End-to-end PoC demonstrating Agent Context Token authorization and
Execution Context Token accountability over MCP tool calls, using a
LangGraph agent with ES256-signed JWT tokens and DAG verification.
2026-04-12 12:43:22 +00:00
45cb13fbe8 feat: add IETF landscape paper source (LaTeX + BibTeX + Makefile)
New LaTeX paper analyzing the AI-agent standardization landscape across
IETF Internet-Drafts. Includes bibliography, updated Makefile for
pdflatex+bibtex build, and gitignore entries for build artifacts.
2026-04-12 12:43:15 +00:00
56f2ce669c feat: unified drafts/ structure with PDF outputs for ACT and ECT
Both drafts now live in workspace/drafts/ as siblings:
  drafts/
  ├── act/                       (ACT -01, native to parent repo)
  │   ├── draft-nennemann-act-01.md     kramdown-rfc source
  │   ├── draft-nennemann-act-01.{xml,txt,html,pdf}
  │   ├── .refcache/             bibxml cache
  │   └── build.sh
  ├── ietf-wimse-ect/            (ECT -02, submodule, PDF added)
  │   └── ...
  └── README-pdf.md              PDF toolchain docs

ACT kramdown-rfc conversion:
- full YAML frontmatter (title, author, refs)
- section structure matches kramdown-rfc conventions
- {{REF}} citation syntax, auto-numbered sections
- references auto-built from normative/informative blocks
- removed manual TOC (kramdown-rfc generates)
- builds cleanly: 133K XML, 89K TXT, 208K HTML, 167K PDF

PDF toolchain:
- xml2rfc --pdf via weasyprint<60 + pydyf<0.10 injected into xml2rfc pipx venv
- both build.sh scripts now produce PDF as Step 4
- README-pdf.md documents the setup for new machines

Submodule: bump ietf-wimse-ect pointer for build.sh PDF step
2026-04-12 14:01:57 +02:00
37859beef6 feat: interop test package + session handoff doc
Cross-spec interop validation between ietf-act and ietf-ect:
- new packages/interop/ sibling package (ietf-act-ect-interop)
- 32 tests pass: shared claims, algorithm matrix, DAG structure,
  divergence handling, anti-goals
- documents ES256 raw signature wire-compatibility
- documents airtight typ separation (act+jwt vs exec+jwt)

Hazards surfaced:
- ACTLedger.append() silently accepts ECT Payload via duck-typing
  (both have .jti) — documented in interop README as a production
  hazard requiring external isinstance checks

Session handoff:
- SESSION-2026-04-12.md — snapshot of decisions, artifacts, open
  actions, and next-session starting points

Also: session-end commit of hash-format fix propagation to
packages/ect/ (the fix was applied to the old refimpl location
but did not propagate through the parallel package-move agent).
2026-04-12 07:39:41 +02:00
3a139dfc7e feat: ACT/ECT strategy, package restructure, draft -01/-02 prep
Strategic work for IETF submission of draft-nennemann-act-01 and
draft-nennemann-wimse-ect-02:

Package restructure:
- move ACT and ECT refimpls to workspace/packages/{act,ect}/
- ietf-act and ietf-ect distribution names (sibling packages)
- cross-spec interop test plan (INTEROP-TEST-PLAN.md)

ACT draft -01 revisions:
- rename 'par' claim to 'pred' (align with ECT)
- rename 'Agent Compact Token' to 'Agent Context Token' (semantic
  alignment with ECT family)
- add Applicability section (MCP, OpenAI, LangGraph, A2A, CrewAI)
- add DAG vs Linear Delegation Chains section (differentiator vs
  txn-tokens-for-agents actchain, Agentic JWT, AIP/IBCTs)
- add Related Work: AIP, SentinelAgent, Agentic JWT, txn-tokens-for-agents,
  HDP, SCITT-AI-agent-execution
- pin SCITT arch to -22, note AUTH48 status

Outreach drafts:
- Emirdag liaison email (SCITT-AI coordination)
- OAuth ML response on txn-tokens-for-agents-06

Strategy document:
- STRATEGY.md with phased action plan, risk register, timeline

Submodule:
- update workspace/drafts/ietf-wimse-ect pointer to -02 commit
2026-04-12 07:33:08 +02:00
b38747ad92 fix: refimpl hash format aligned to -01 spec, draft rebuilt
Some checks failed
CI / test (3.11) (push) Failing after 53s
CI / test (3.12) (push) Failing after 41s
Updated Go ValidateHashFormat to plain base64url (no algorithm prefix)
per -01 spec and RFC 9449. Python already fixed. All tests pass.
2026-04-11 17:51:50 +02:00
2506b6325a feat: add draft data, gap analysis report, and workspace config
Some checks failed
CI / test (3.11) (push) Failing after 1m37s
CI / test (3.12) (push) Failing after 57s
2026-04-06 18:47:15 +02:00
4f310407b0 docs: assess ECT draft submission readiness, fix -00/-01 discrepancies
Some checks failed
CI / test (3.11) (push) Failing after 8s
CI / test (3.12) (push) Failing after 9s
Observatory Update / update (push) Failing after 10s
- Add docs/status.md with full draft quality assessment
- Fix blog post: update outdated claim names (par -> pred, ext -> ect_ext)
- Update refimpl README: add -00 vs -01 migration note with claim mapping
- Update IMPROVEMENTS.md: add section 6 with all -01 migration tasks
2026-04-03 07:46:45 +02:00
42b4546ded feat: refresh pipeline, reorganize draft generation, polish public pages
Some checks failed
CI / test (3.11) (push) Failing after 1m39s
CI / test (3.12) (push) Failing after 1m1s
Pipeline refresh:
- Extract ideas from 46 remaining drafts (844 total ideas now)
- Clear stale llm_cache entries blocking re-extraction
- Re-run gap analysis with expanded corpus: 10 gaps (was 12), fresh IDs #1-#10
- Re-link 3 proposals to new gap IDs
- Add scripts/pipeline-refresh.sh for reproducible runs

Draft generation moved from gaps to proposals:
- Remove "Generate Internet-Draft" section from gap detail page
- Add it to proposal detail page instead (proposals → generate I-D flow)
- New route POST /proposals/<id>/generate with richer context
  (proposal title + description + linked gap topics)
- Remove misleading "Search related drafts" link from gap page
  (related drafts already shown inline)

Public page polish:
- Overview: update subtitle to mention all 6 standards bodies
- About: describe multi-source scope (IETF, ISO, ITU-T, ETSI, NIST, W3C)
- About: add guiding question ("Where is the AI agent standards race heading?")
- Obsidian export button hidden in production mode (prior commit)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 05:39:13 +01:00
f8ed2b83e9 fix: security hardening — self-hosted JS, XSS protection, SSRF blocking
- Replace all CDN script tags (marked, plotly) with self-hosted static files
- Add DOMPurify for sanitizing markdown-rendered HTML
- Add escapeHtml() helper to base.html for all innerHTML operations
- Sanitize dynamic data in innerHTML across 13 templates
- Add security headers (X-Content-Type-Options, X-Frame-Options, Referrer-Policy)
- Add SSRF protection to proposal intake URL fetcher (block private/loopback IPs)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:47:32 +01:00
d1a20fa02e feat: add blog draft generator and fix broken routes
Blog drafting section (dev-only):
- BlogDraftGenerator gathers project data (gaps, proposals, stats) as
  context and calls Claude to produce Medium-style blog posts
- DB schema: blog_drafts table with title, content, tags, cost tracking
- Web UI: list, generate (async with live preview), detail (rendered +
  source toggle), edit, and export routes
- 6 writing styles: deep-dive, overview, opinion, listicle, comparison,
  series-post
- Nav link added to sidebar under Proposals

Bug fixes found via route testing (scripts/test_all_routes.py):
- /authors/<id>: Draft.status → Draft.states (correct attribute name)
- /false-positives: add missing `import re` in ratings.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:39:16 +01:00
2229e70c73 fix: deduplicate draft revisions in search results
Different revisions of the same draft (e.g. draft-foo-bar-00, -01, -02)
were showing up as separate results. Now keeps only the highest-scoring
revision per base draft name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:07:38 +01:00
cdb2e505c6 fix: restore SAFETY/CAPABILITY_CATEGORIES lost in data.py split
Constants were defined in the old monolithic data.py but not carried
over to data/analysis.py during the split, breaking /trends page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 04:04:19 +01:00
61cdab16b9 fix: dev mode auth regression from blueprint refactor
The _initialized singleton in auth.py prevented hooks from registering
on the correct app instance when create_app() was called twice (once
eagerly at import, once from __main__). Removed the guard and made
the module-level app lazy. Also adds feature backlog and architecture
assessment from the review team.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:52:02 +01:00
dea36e931a Add test coverage for CLI commands, Flask routes, and shared DB methods
74 new tests across 3 files:
- test_cli.py: CLI help, version, config, report generation, wg/viz/pipeline subcommands
- test_web.py: All public pages, admin pages (dev/prod), API JSON endpoints, CSV export, 404s
- test_db_shared.py: rated_count, gap_count, false_positive_names, category_counts, source_counts, draft_author_count_map, search_gaps, search_authors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:49:09 +01:00
c755b2bbf3 Add author detail, idea detail, and gap-draft reverse link pages
- Author detail page (/authors/<person_id>): shows author info, all drafts
  with ratings, and co-authors with shared draft counts. Public route.
- Idea detail page (/ideas/<idea_id>): shows idea metadata, source draft,
  and top-5 most similar ideas via embedding cosine similarity. Admin route.
- Gap detail page: added "Related Drafts" section that finds drafts by
  extracting draft names from evidence text and searching by topic keywords.
- Updated author links across templates to use /authors/<person_id> URLs.
- Added DB methods: get_author_by_id, get_author_drafts, get_coauthors.
- Extended top_authors to include person_id (5th tuple element).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:45:00 +01:00
4a368bde62 fix: update test_top_authors for new person_id return field
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:41:57 +01:00
b10572c628 Split cli.py (3,438 LOC) into modular command packages
Move 98 CLI commands from monolithic cli.py into organized modules:
- commands/common.py: shared utilities (console, pass_cfg_db, _get_config)
- commands/fetch.py: fetch, search, list, show, annotate, classify, authors, network
- commands/analysis.py: analyze, ask, compare, embed, ideas, gaps, refs, trends, etc.
- commands/reports.py: report group, viz group, wg group, export
- commands/admin.py: config, pipeline, observatory, monitor, auto-heal
- commands/proposals.py: draft-gen, intake

cli.py is now a slim 30-line entry point that registers all modules.
All command names, options, and behavior preserved exactly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:40:57 +01:00
3fb17100d7 Split webui into Flask blueprints and data domain modules
- Split app.py (66 routes) into 3 blueprints: pages (public), api (JSON), admin (@admin_required)
- Split data.py (4,360 LOC) into 7 domain modules: drafts, authors, ratings, gaps, analysis, search, proposals
- Add data/__init__.py re-exporting all public functions for backward compatibility
- Add custom 404/500 error pages matching dark theme
- Add request timing logging via before_request/after_request hooks
- Refactor app.py into create_app() factory pattern
- All 106 tests pass, all 66 routes preserved

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:37:15 +01:00
c066b04d74 Add DB indexes and extract shared query methods to Database class
Add missing indexes on ratings(false_positive), drafts(source), and
draft_authors(person_id) for faster filtering. Extract 12 shared query
methods (false_positive_drafts_raw, non_false_positive_ratings_raw,
false_positive_names, rated_count, gap_count, search_gaps, search_authors,
draft_affiliation_pairs, all_persons_info, category_counts,
draft_author_count_map, source_counts) to eliminate duplicated SQL across
cli.py, data.py, and reports.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:33:18 +01:00
4710668419 chore: commit current DB and add SQL dump export/import scripts
Include data/drafts.db so other machines don't need to re-run
expensive Claude API calls (~$3+ of analysis, 474 drafts, 403 authors,
1262 ideas, 12 gaps). Add scripts/db-export.sh and scripts/db-import.sh
for portable compressed SQL dump sharing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:21:00 +01:00
5ec7410b89 feat: proposal intake pipeline with AI-powered generation on /proposals/new
Add full proposal system: DB schema (proposals + proposal_gaps tables),
CLI `ietf intake` command, and web UI with Quick Generate on /proposals/new.
The new page merges AI intake (paste URL/text → Haiku generates multiple
proposals auto-linked to gaps) with manual form entry. Generated proposals
are clickable cards that fill the editor below for refinement.

Uses claude_model_cheap (Haiku) for cost-efficient web intake. Includes
CaML-inspired draft proposals from arXiv:2503.18813 analysis.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:15:11 +01:00
ae5e5f8cbf Enforce public/private visibility for web UI pages
Dev-only pages (sources, trends, complexity, idea-analysis, false-positives,
similarity, landscape, export) now require @admin_required and are hidden
from nav in production mode. Citations page keeps the graph public but
hides influence/BCP tabs behind --dev flag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:52:43 +01:00
dec8667193 Add 6 new analysis pages and 5 CLI reports
New web UI pages with Plotly charts:
- /sources: cross-source comparison (ratings, categories by standards body)
- /false-positives: profiling of 73 false positives (box plots, terms)
- /trends: temporal evolution (submissions, ratings, safety ratio over time)
- /complexity: draft complexity matrix (correlations, scatter plots)
- /idea-analysis: idea novelty deep dive (sunburst, distribution, shared ideas)
- /citations: enhanced with influence analysis and BCP dependency tabs

New CLI reports (ietf report <name>):
- sources, false-positives, citations, complexity, idea-analysis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 20:35:32 +01:00
8515e46d5d Architecture designer, author cluster names, FP filtering, new pages
- Add /architecture page: system-of-systems view with 8 layers, component
  cards, gap markers, source coverage chart, and clickable detail sidebar
- Give author clusters meaningful names from orgs + draft topic keywords
- Filter false positives (73 drafts, 54 ideas) from idea clusters,
  architecture, ideas listing, and search results
- Add NIST source fetcher with curated catalog of 11 AI publications
- New pages: trends, complexity, sources, false positives, idea analysis
- Clickable gap cards with full details (evidence, priority, nearby work)
- Component detail panel with linked drafts and top ideas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 19:58:40 +01:00
a46a01bd8c Add auto-heal pipeline command and fix multi-source draft processing
- Add `ietf auto` command: fetches, analyzes, embeds, extracts ideas,
  and refreshes gaps across all sources with cost-based auto-approval
- Fix SourceDocument→Draft conversion in auto fetch step
- Fix gap_analysis method name in auto command
- Process all 270 unrated ETSI/ISO/ITU/NIST drafts (761 total, all rated)
- Update web UI templates and data layer for multi-source support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 18:41:42 +01:00
1ec1f69bee v0.3.0: Publication-ready release with blog site, paper update, and polish
Release prep:
- Version bump to 0.3.0 (pyproject.toml, cli.py)
- Rewrite README.md with current stats (475 drafts, 713 authors, 501 ideas)
- Add CONTRIBUTING.md with dev setup and code conventions

Blog site:
- Add scripts/build-site.py (markdown → HTML with clean CSS, dark mode, nav)
- Generate static site in docs/blog/ (10 pages)
- Ready for GitHub Pages deployment

Academic paper (paper/main.tex):
- Update all counts: 474→475 drafts, 557→710 authors, 1907→462 ideas, 11→12 gaps
- Add false-positive filtering methodology (113 excluded, 361 relevant)
- Add cross-org convergence analysis (132 ideas, 33% rate)
- Add GDPR compliance gap to gap table
- Add LLM-as-judge caveats to rating methodology and limitations
- Add FIPA, IEEE P3394, W3C WoT to related work with bibliography entries
- Fix safety ratio to show monthly variation (1.5:1 to 21:1)

Pipeline:
- Fetch 1 new draft (475 total), 3 new authors (713 total)
- Fix 16 ruff lint errors across test files
- All 106 tests pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 17:54:43 +01:00
e247bfef8f Run pipeline, write Post 08, commit untracked files
Pipeline:
- Extract ideas for 38 new drafts → 462 ideas total
- Convergence analysis: 132 cross-org convergent ideas (33% rate)
- Fetch authors for 102 drafts → 709 authors (up from 403)
- Refresh gap analysis: 12 gaps across full 474-draft corpus
- Update verified counts with new totals

Post 08:
- Complete rewrite of "Agents Building the Agent Analysis" (2,953 words)
- Covers 3 phases: writing team → review cycle → fix cycle
- Meta-irony table mapping team coordination to IETF gap names
- Specific examples from dev journal (SQL injection, consent conflation, ideas mismatch)

Untracked files committed:
- scripts/: backfill-wg-names, classify-unrated, compare-classifiers, download-relevant-text, run-webui
- src/ietf_analyzer/classifier.py: two-stage Ollama classifier
- src/webui/: analytics (GDPR-compliant), auth, obsidian_export
- tests/test_obsidian_export.py (10 tests)
- data/reports/: wg-analysis, generated draft for gap #37

Housekeeping:
- .gitignore: exclude LaTeX artifacts, stale DBs, analytics.db

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:31:30 +01:00
20c45a7eba Complete remaining medium/low issues: performance, CLI, types, CI, tests
Performance:
- Batch readiness computation (~200 queries → ~6 per page)
- Batch draft lookup in author network (N+1 → single query)
- File-based similarity matrix cache (.npy + metadata sidecar)
- 5-minute TTL embedding cache for search queries

CLI quality:
- Add pass_cfg_db decorator, convert ~30 commands to shared config/db lifecycle
- Add --dry-run to analyze, embed, embed-ideas, ideas, gaps commands
- Move 15+ in-function imports to top of data.py

Types & documentation:
- Add 16 TypedDicts to data.py, annotate 12 function return types
- Add ethics section to Post 06 (premature standardization, power asymmetry)
- Add EU AI Act Article 43 conformity mapping to Post 06
- Add NIS2 and CRA references to Post 04

CI & testing:
- Add GitHub Actions CI workflow (Python 3.11+3.12, ruff, pytest)
- Add API documentation for all 20 endpoints (data/reports/api-docs.md)
- Add 41 new tests (test_analyzer.py, test_search.py) — 64 total pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:06:54 +01:00
e7527ad68e Fix remaining critical, high, and medium issues from 4-perspective review
Critical fixes:
- Fix rating clamp range 1-10 → 1-5 (actual scale)
- Add `ietf ideas convergence` command (SequenceMatcher at 0.75 threshold)
- Fix "628 cross-org ideas" → 130 (verified from current DB) across 8 files

Security fixes:
- Sanitize FTS5 query input (strip special chars + boolean operators)
- Add rate limiting (10 req/min/IP) on Claude-calling endpoints
- Change <path:name> → <string:name> on draft routes

Codebase fixes:
- Add Database context manager (__enter__/__exit__)
- Wire false_positive filtering into queries (exclude by default in web UI)
- Fix Post 3 arithmetic ("~300" → "~409" distinct proposals)

Content & licensing:
- Add MIT LICENSE file
- Add IPR/FRAND notes (BCP 79, RFC 8179) to Posts 03 and 07
- Qualify "4:1 safety ratio" with monthly variation in 6 remaining files
- Add "Data as of March 2026" freeze-date headers to all 10 blog posts
- Hedge causal language in Post 04

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 12:47:47 +01:00
f1a0b0264c Fix blog accuracy and add methodology documentation
Blog posts (all 10 files updated):
- Update all counts to match DB: 434 drafts, 557 authors, 419 ideas, 11 gaps
- Fix EU AI Act timeline to August 2026 (5 months, not 18)
- Reframe growth claim from "36x" to actual monthly figures (5→61→85)
- Add safety ratio nuance (1.5:1 to 21:1 monthly variation)
- Fix composite scores (4.8→4.75, 4.6→4.5)
- Add OAuth/GDPR consent distinction (Art. 6(1)(a), Art. 28)
- Add EU AI Act Annex III + MDR context to hospital scenario
- Add FIPA, IEEE P3394, eIDAS 2.0 references
- Add GDPR gap paragraph (DPIA, erasure, portability, purpose limitation)
- Rewrite Post 04 gap table to match actual DB gap names

Methodology:
- Expand methodology.md: pipeline docs, limitations, related work
- Add LLM-as-judge caveats and explicit rating rubric to analyzer.py
- Add clustering threshold rationale to embeddings.py
- Add gap analysis grounding notes to analyzer.py
- Add Limitations section to Post 07

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 11:04:40 +01:00
439424bd04 Fix security, data integrity, and accuracy issues from 4-perspective review
Security fixes:
- Fix SQL injection in db.py:update_generation_run (column name whitelist)
- Flask SECRET_KEY from env var instead of hardcoded
- Add LLM rating bounds validation (_clamp_rating, 1-10)
- Fix JSON extraction trailing whitespace handling

Data integrity:
- Normalize 21 legacy category names to 11 canonical short forms
- Add false_positive column, flag 73 non-AI drafts (361 relevant remain)
- Document verified counts: 434 total/361 relevant drafts, 557 authors, 419 ideas, 11 gaps

Code quality:
- Fix version string 0.1.0 → 0.2.0
- Add close()/context manager to Embedder class
- Dynamic matrix size instead of hardcoded "260x260"

Blog accuracy:
- Fix EU AI Act timeline (enforcement Aug 2026, not "18 months")
- Distinguish OAuth consent from GDPR Einwilligung
- Add EU AI Act Annex III context to hospital scenario
- Add FIPA, eIDAS 2.0 references where relevant

Methodology:
- Add methodology.md documenting pipeline, limitations, rating rubric
- Add LLM-as-judge caveats to analyzer.py
- Document clustering threshold rationale

Reviews from: legal (German/EU law), statistics, development, science perspectives.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 10:52:33 +01:00
a386d0bb1a Add explanation of how author clusters are formed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:27:23 +01:00
02049c37a8 Show related drafts in author cluster cards
Each cluster card now shows draft count badge and up to 5 linked draft
titles with clickable links to draft detail pages. Data collected from
member nodes' draft lists during cluster detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:23:13 +01:00
34c36f81f1 Make /ask free by default, Claude synthesis is opt-in
Search results (FTS5 + Ollama embeddings) are shown immediately at no
cost. AI synthesis via Claude is behind a "Synthesize" button that the
user must explicitly click. Results are cached permanently so repeat
visitors never trigger API calls.

- Split ask into search_only() (free) and ask() (paid, cached)
- GET /ask now uses search_only — no Claude tokens spent
- POST /api/ask/synthesize triggers Claude (Haiku, ~$0.001)
- Cached answers shown with "cached" badge, no re-generation
- Template shows sources immediately + optional synthesize button

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:15:07 +01:00
e65d1cfacd Fix drafts search to include author names
Clicking an author in the network graph opens /drafts?q=AuthorName
but search only checked draft name/title/summary. Now also matches
against author names via a JOIN on draft_authors + authors tables.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:06:26 +01:00
3e36802500 Fix author network graph: tune force sim, reduce node count
- Raise co-authorship threshold from 1 to 2 shared drafts (498→156 nodes)
- Tune D3 force parameters for large graphs: capped link strength,
  wider distance, adaptive charge, lower velocity decay
- Add initial circular layout to prevent explosion on load
- Fix cluster highlighting with fallback name matching and
  position validation before zoom-to-fit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 21:04:18 +01:00
757b781c67 Platform upgrade: semantic search, citations, readiness, tests, Docker
Major features added by 5 parallel agent teams:
- Semantic "Ask" (NL queries via FTS5 + embeddings + Claude synthesis)
- Global search across drafts, ideas, authors, gaps
- REST API expansion (14 endpoints, up from 3) with CSV/JSON export
- Citation graph visualization (D3.js, 440 nodes, 2422 edges)
- Standards readiness scoring (0-100 composite from 6 factors)
- Side-by-side draft comparison view with shared/unique analysis
- Annotation system (notes + tags per draft, DB-persisted)
- Docker deployment (Dockerfile + docker-compose with Ollama)
- Scheduled updates (cron script with log rotation)
- Pipeline health dashboard (stage progress bars, cost tracking)
- Test suite foundation (54 pytest tests covering DB, models, web data)

Fixes: compare_drafts() stubbed→working, get_authors_for_draft() bug,
source-aware analysis prompts, config env var overrides + validation,
resilient batch error handling with --retry-failed, observatory --dry-run

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 20:52:56 +01:00
da2a989744 GDPR compliance: self-host all assets, add Impressum + Datenschutz
- Self-host Tailwind, Plotly, D3.js (no more CDN requests)
- Self-host Inter font (no more Google Fonts requests)
- Replace JetBrains Mono with system monospace font stack
- Zero external requests when visiting the site (GDPR-safe)
- Add /impressum page (§5 TMG, §18 MStV)
- Add /datenschutz page (DSGVO Art. 13/14)
- Hosting: Hetzner, no cookies, no tracking, no analytics
- Add Impressum/Datenschutz links to sidebar footer
- Configure Flask static_folder for local asset serving

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:20:52 +01:00
75c4da72e0 Fix broken reference links and web UI bugs
- Fix RFC URLs with leading zeros (rfc0020 -> rfc20) via int filter
- Draft refs: internal link for drafts in our DB, Datatracker for external
- BCP refs: link to rfc-editor.org/info/bcpN
- Add DB connection teardown (@app.teardown_appcontext)
- Fix JS syntax error in gap_demo.html (HTML-escaped string in script tag)
- Add URL encoding to all query params in drafts.html and draft_detail.html
- Fix variable shadowing of Flask's g import in gaps_demo()
- Add None safety for ideas search data attribute

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 07:59:45 +01:00
6e3a387778 Idea quality pipeline, web UI features, academic paper
- Tighten idea extraction prompts (1-4 ideas, no sub-features) reducing
  1,907 ideas to 468 across 434 drafts (78% reduction)
- Add embedding-based dedup (ietf dedup-ideas) for same-draft similarity
- Add novelty scoring (ietf ideas score) and filtering (ietf ideas filter)
  using Claude to rate ideas 1-5, removing 49 generic building blocks
- Final count: 419 high-quality ideas (avg 1.1/draft)
- Web UI: gap explorer with live draft generation and pre-generated demos
- Web UI: D3.js author collaboration network (498 nodes, 1142 edges,
  68 clusters, org filtering, interactive zoom/pan)
- Academic paper: 15-page LaTeX workshop paper analyzing the 434-draft
  AI agent standards landscape
- Save improvement ideas backlog to data/reports/improvement-ideas.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:17:57 +01:00
3c3d7e649f Add ASCII art figures to all 6 generated Internet-Drafts
Each draft gets 2 illustrative figures:
- ABVP: architecture components + verification workflow
- ATD: example DAG structure + execution state transitions
- HITL: primitive framework overview + approval workflow sequence
- AEM/PPALP: federated learning architecture + aggregation flow
- RARP: cross-domain architecture + two-phase rollback protocol
- APAE: layered architecture + cross-domain provenance tracking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 02:20:39 +01:00
404092b938 Generate 5-draft ecosystem family, fix formatter markdown stripping
Pipeline output:
- ABVP: Agent Behavior Verification Protocol (quality 3.0/5)
- AEM: Privacy-Preserving Agent Learning Protocol (quality 2.1/5)
- ATD: Agent Task DAG Framework (quality 2.5/5)
- HITL: Human-in-the-Loop Primitives (quality 2.4/5)
- AEPB: Real-Time Agent Rollback Protocol (quality 2.5/5)
- APAE: Agent Provenance Assurance Ecosystem (quality 2.5/5)

Quality gates: all pass novelty + references, format gate improved
with markdown stripping (_strip_markdown) and dynamic header padding.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 01:42:30 +01:00
7a1aa346b9 Observatory update: 434 docs, fix W3C fetcher, regenerate dashboard
- Fixed W3C fetcher to paginate /specifications endpoint (group
  endpoints use type prefixes like cg/, wg/ that weren't in config)
- Fetched 72 new IETF drafts + 1 W3C spec, all analyzed and embedded
- Regenerated dashboard with updated data
- Total: 434 docs, 11 gaps, 1907 ideas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 01:09:30 +01:00
d6beb9c0a0 v0.3.0: Gap-to-Draft pipeline, Living Standards Observatory, blog series
Gap-to-Draft Pipeline (ietf pipeline):
- Context builder assembles ideas, RFC foundations, similar drafts, ecosystem vision
- Generator produces outlines + sections using rich context with Claude
- Quality gates: novelty (embedding similarity), references, format, self-rating
- Family coordinator generates 5-draft ecosystem (AEM/ATD/HITL/AEPB/APAE)
- I-D formatter with proper headers, references, 72-char wrapping

Living Standards Observatory (ietf observatory):
- Source abstraction with IETF + W3C fetchers
- 7-step update pipeline: snapshot, fetch, analyze, embed, ideas, gaps, record
- Static GitHub Pages dashboard (explorer, gap tracker, timeline)
- Weekly CI/CD automation via GitHub Actions

Also includes:
- 361 drafts (expanded from 260 with 6 new keywords), 403 authors, 1,262 ideas, 12 gaps
- Blog series (8 posts planned), reports, arXiv paper figures
- Agent team infrastructure (CLAUDE.md, scripts, dev journal)
- 5 new DB tables, schema migration, ~15 new query methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 00:48:57 +01:00
be9cf9c5d9 v0.2.0: visualizations, interactive browser, arXiv paper, gap analysis
New features:
- 12 interactive visualizations (ietf viz): t-SNE landscape, similarity
  heatmap, score distributions, timeline, bubble explorer, radar charts,
  author network graph, category treemap, quality vs overlap, org bar chart,
  ideas chart, and interactive draft browser
- Interactive draft browser (browser.html): filterable by category, keyword,
  score sliders with sortable table and expandable detail rows
- arXiv paper (paper/main.tex): 13-page manuscript with all findings
- Gap analysis: 12 identified under-addressed areas
- Author network: collaboration graph, org contributions, cross-org analysis
- Draft generation from gaps (ietf draft-gen)
- Auto-load .env for API keys (python-dotenv)

New modules: visualize.py, authors.py, draftgen.py
New reports: timeline, overlap-matrix, authors, gaps
New deps: plotly, matplotlib, seaborn, scipy, scikit-learn, networkx, python-dotenv

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 13:37:55 +01:00
f44f9265bd Add SQLite database with 260 analyzed drafts
Includes all draft metadata, full text, Claude ratings (cached),
and nomic-embed-text embeddings. This is the expensive data —
~114k tokens of Claude analysis + 260 Ollama embeddings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 00:49:18 +01:00
6771a4c235 IETF Draft Analyzer v0.1.0 — track, categorize, and rate AI/agent drafts
Python CLI tool that fetches AI/agent-related Internet-Drafts from the IETF
Datatracker, rates them using Claude, generates embeddings via Ollama for
similarity/clustering, and produces markdown reports.

Features:
- Fetch drafts by keyword from Datatracker API with full text download
- Batch analysis with Claude (token-optimized, responses cached in SQLite)
- Embedding-based similarity search and overlap cluster detection
- Reports: overview, landscape by category, overlap clusters, weekly digest
- SQLite with FTS5 for full-text search across 260 tracked drafts

Initial analysis of 260 drafts reveals OAuth agent auth (13 drafts) and
agent gateway/collaboration (10 drafts) as the most crowded clusters,
while AI safety/alignment is underserved with the highest quality scores.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 00:36:45 +01:00