Run pipeline, write Post 08, commit untracked files

Pipeline:
- Extract ideas for 38 new drafts → 462 ideas total
- Convergence analysis: 132 cross-org convergent ideas (33% rate)
- Fetch authors for 102 drafts → 709 authors (up from 403)
- Refresh gap analysis: 12 gaps across full 474-draft corpus
- Update verified counts with new totals

Post 08:
- Complete rewrite of "Agents Building the Agent Analysis" (2,953 words)
- Covers 3 phases: writing team → review cycle → fix cycle
- Meta-irony table mapping team coordination to IETF gap names
- Specific examples from dev journal (SQL injection, consent conflation, ideas mismatch)

Untracked files committed:
- scripts/: backfill-wg-names, classify-unrated, compare-classifiers, download-relevant-text, run-webui
- src/ietf_analyzer/classifier.py: two-stage Ollama classifier
- src/webui/: analytics (GDPR-compliant), auth, obsidian_export
- tests/test_obsidian_export.py (10 tests)
- data/reports/: wg-analysis, generated draft for gap #37

Housekeeping:
- .gitignore: exclude LaTeX artifacts, stale DBs, analytics.db

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-08 15:31:30 +01:00
parent 20c45a7eba
commit e247bfef8f
19 changed files with 2758 additions and 586 deletions

View File

@@ -4,6 +4,41 @@
---
### 2026-03-08 ANALYST — Pipeline run: authors + gaps refresh
**What**: Ran the processing pipeline on 474-draft corpus. Fetched authors for 102 previously-unlinked drafts (113 were missing, 11 had Datatracker issues). Re-ran gap analysis with --refresh on the full corpus. Checked idea extraction status.
**Why**: After corpus expansion to 474 drafts, 113 drafts lacked author data and gap analysis needed refreshing against the full set.
**Result**: Author coverage: 463/474 drafts now have authors (up from ~350), 709 unique authors (up from 403). Gap analysis: 12 gaps identified (same count, refreshed against full corpus). All 474 drafts already rated. Idea extraction: 59 drafts have no ideas but are in the LLM cache (previously processed, yielded nothing -- 25 rated relevance 4-5, so may warrant individual re-extraction with --reextract).
**Surprise**: The `drafts_without_ideas` query checks both the ideas table AND the llm_cache table, so drafts that were batch-processed but yielded no ideas won't be retried by `--all`. To force re-extraction for high-relevance drafts without ideas, use `ietf ideas --reextract --draft <name>` individually.
---
### 2026-03-08 WRITER — Post 08 Rewrite: "Agents Building the Agent Analysis"
**What**: Complete rewrite of Post 08, the meta post about using Claude Code agent teams to build the project. The previous draft (~3,500 words, written before the review cycle) covered only Phase 1 (the writing team). The new version (~2,800 words) covers all three phases: the 4-agent writing team, the 4-agent review cycle, and the 3-round fix cycle.
**Why**: The review cycle was the most consequential phase of the entire project -- 4 parallel reviewers found 36 issues including a SQL injection, consent conflation with EU law, a 76% ideas count mismatch, and uncalibrated methodology. This material was missing from the previous draft entirely. The post needed to tell the complete story.
**Result**: New structure: Phase 1 (writing team + parallel execution + Architect's impact + human "so what?" intervention), Phase 2 (4 parallel reviewers, specific findings per reviewer), Phase 3 (3-round fix cycle by severity), then analysis sections (specialized perspectives, review-fix-verify pattern, shared-state coordination problem, cost breakdown, meta-irony table). The meta-irony table now maps 7 team coordination needs to specific IETF gap names from the database.
**Surprise**: The post's strongest structural element is the review cycle section -- the specific examples (consent conflation, HPKE false positive, silent revision failure) are more vivid and demonstrable than the writing-phase anecdotes. The review cycle essentially proved the thesis: agents analyzing agents need the same coordination standards the agents are analyzing.
---
### 2026-03-08 CODER — Track untracked files, update .gitignore
**What**: Cleaned up untracked files in the repo. Updated `.gitignore` to exclude LaTeX build artifacts (`paper/*.aux`, `paper/*.log`, `paper/*.out`), `data/analytics.db`, and `data/ietf_drafts.db` (stale DB). Staged 12 new files for commit: 5 scripts (`backfill-wg-names.py`, `classify-unrated.py`, `compare-classifiers.py`, `download-relevant-text.py`, `run-webui.sh`), 4 source modules (`classifier.py`, `analytics.py`, `auth.py`, `obsidian_export.py`), 1 test (`test_obsidian_export.py`), 2 reports (`wg-analysis.md`, generated draft).
**Why**: These files had accumulated as untracked over several sessions. Production code, utility scripts, and analysis reports all belong in version control. Build artifacts and local DBs do not.
**Result**: 12 files staged, .gitignore updated with 6 new patterns. No commit made yet (deferred to parent process).
---
### 2026-03-08 ANALYST — Re-extract ideas and convergence analysis
**What**: Ran idea extraction pipeline for 38 drafts that were missing ideas (out of 97 initially missing — 59 remain without ideas, likely false positives or drafts without sufficient content). Then ran cross-organization convergence analysis on the full idea set.
**Why**: Ideas count was stale at 419 across 377 drafts after the DB expanded to 474 drafts. Convergence analysis needed to understand which technical ideas are independently emerging across multiple organizations.
**Result**: 462 ideas across 415 drafts. Convergence analysis found 132 cross-org convergent ideas out of 398 unique clusters (33% convergence rate). Top convergent idea: "Fully Adaptive Routing Ethernet for AI" with 14 contributing organizations. Notable: "AI Agent Protocol Framework" converges across 7 orgs and 3 separate drafts. Updated `data/reports/reviews/verified-counts.md` with new counts and convergence results.
**Cost**: 654,377 tokens in + 335,984 tokens out (Haiku, cheap mode), 8 batches of 5 drafts.
---
### 2026-03-08 CODER — TypedDicts for data layer, ethics + regulatory content in blog series
**What**: Four improvements across typing and content: