IETF Draft Analyzer

Track, categorize, rate, and map AI/agent-related IETF Internet-Drafts.

475 drafts analyzed (361 relevant after false-positive filtering) with 713 authors, 501 extracted ideas, 132 cross-org convergent ideas, and 12 identified gaps — spanning 2024 to March 2026.

What This Does

The IETF is experiencing an unprecedented wave of standardization activity around AI agents. This tool provides a quantitative lens on that activity:

• Fetches draft metadata and full text from the IETF Datatracker API
• Rates each draft on 5 dimensions (novelty, maturity, overlap, momentum, relevance) using Claude
• Embeds drafts with Ollama for pairwise similarity and clustering
• Extracts discrete technical ideas and identifies landscape gaps
• Analyzes cross-organizational convergence (SequenceMatcher at 0.75 threshold)
• Maps the author collaboration network and organizational affiliations
• Generates markdown reports and a full web dashboard
• Filters false positives automatically (relevance-based + manual flagging)

Quick Start

# Install
pip install -e .

# Set your API key (or add to .env file)
export ANTHROPIC_API_KEY=sk-ant-...

# Fetch drafts from IETF Datatracker
ietf fetch

# Rate all unrated drafts (--cheap uses Haiku for lower cost)
ietf analyze --all
ietf analyze --all --cheap    # ~10x cheaper with Haiku

# Generate embeddings (requires Ollama running locally)
ietf embed

# Extract technical ideas
ietf ideas --all --cheap --batch 5

# Analyze cross-org convergence
ietf ideas convergence

# Identify gaps in the landscape
ietf gaps

# Fetch author data
ietf authors --fetch

# Generate reports
ietf report overview
ietf report landscape
ietf report authors

# Launch the web dashboard
./scripts/run-webui.sh

Web Dashboard

A full interactive dashboard at http://127.0.0.1:5000:

./scripts/run-webui.sh
# or: FLASK_APP=src/webui/app.py flask run

Page	What it shows
Overview	Stat cards, score histogram, category radar, submission timeline
Draft Explorer	Searchable/filterable/sortable table with category pills and score badges
Draft Detail	Score ring, dimension bars, ideas, references, linked authors
Ratings	Score distributions, box plots, category radar, novelty vs maturity scatter
Landscape	t-SNE embedding map, quality quadrants
Authors	Co-authorship force-directed graph, organization charts
Ideas	Extracted ideas grouped by type with search
Gaps	Gap cards sorted by severity with related drafts
Citations	RFC cross-reference graph
Similarity	Draft similarity network
Timeline	Submission trends over time
Monitor	Pipeline health, API costs, processing status

Charts are interactive (Plotly.js). GDPR-compliant analytics (no cookies, daily-salted IP hashing).

Blog Series

An 8-post analysis series in data/reports/blog-series/:

1. The Gold Rush — Growth from 9 drafts to 9.3% of all IETF submissions
2. Who Writes the Rules — Huawei's 16%, geopolitical dynamics, team blocs
3. The OAuth Wars — 14 competing OAuth proposals, fragmentation costs
4. What Nobody Builds — The safety deficit (4:1 ratio), 12 identified gaps
5. Where Drafts Converge — 132 cross-org convergent ideas, implicit consensus
6. The Big Picture — Architectural vision, EU AI Act implications
7. How We Built This — Methodology, cost ($9-15), limitations
8. Agents Building the Agent Analysis — Meta post on using Claude agent teams

Key Findings

• Safety deficit: ~4:1 ratio of capability-building to safety proposals (varies 1.5:1 to 21:1 monthly)
• Extreme fragmentation: 155 competing A2A protocols, 42 overlap clusters
• Organizational concentration: Huawei ~16% of all drafts, Chinese orgs ~40%
• Cross-org convergence: 132 ideas (33%) independently proposed by multiple organizations
• 12 gaps identified: 2 critical (behavior verification, human override), 5 high, 5 medium
• Top-rated drafts: Safety-focused proposals score highest (VOLT 4.75, DAAP 4.75)

CLI Commands

Core Pipeline

Command	Description
ietf fetch	Fetch AI/agent drafts from IETF Datatracker
ietf analyze --all [--cheap] [--dry-run]	Rate drafts using Claude
ietf embed [--dry-run]	Generate semantic embeddings via Ollama
ietf ideas --all [--cheap] [--batch N] [--dry-run]	Extract technical ideas
ietf ideas convergence [--threshold 0.75]	Cross-org convergence analysis
ietf ideas dedup	Deduplicate similar ideas
ietf gaps [--dry-run]	Identify landscape gaps
ietf authors --fetch	Fetch author/affiliation data

Exploration

Command	Description
ietf list	List tracked drafts
ietf show <name>	Show detailed info for a draft
ietf search <query>	Full-text search (FTS5)
ietf similar <name>	Find similar drafts by embedding similarity
ietf clusters	Find clusters of near-duplicate drafts
ietf compare <name1> <name2>	Compare drafts for overlap
ietf authors	Top authors and draft counts
ietf network	Organizational collaboration network

Reports (ietf report)

Command	Description
ietf report overview	Sortable table of all rated drafts
ietf report landscape	Category-grouped view with rankings
ietf report authors	Top authors, organizations, collaboration
ietf report ideas	Ideas by type, most common, unique
ietf report gaps	Gap analysis with severity ratings
ietf report timeline	Monthly submission trends
ietf report overlap-matrix	Similar pairs and cross-category matrix

Rating System

Each draft is scored 1-5 on five dimensions by Claude (LLM-as-judge, see methodology for caveats):

Dimension	What it measures
Novelty	Originality relative to existing standards
Maturity	Completeness of specification
Overlap	Redundancy with other drafts (5 = heavily overlapping)
Momentum	Community engagement, revisions, adoption
Relevance	Importance to the AI/agent ecosystem

Important: Ratings are generated from abstracts and partial full text without human calibration. They should be treated as relative rankings, not absolute quality measures.

Tech Stack

• Python 3.11+ with Click CLI
• SQLite with FTS5 full-text search and WAL mode
• Anthropic Claude (Sonnet/Haiku) for analysis, rating, idea extraction, gap analysis
• Ollama (nomic-embed-text) for local embeddings and similarity
• Flask with Jinja2 for the web dashboard
• Plotly for interactive visualizations
• NumPy/SciPy/scikit-learn for similarity computation and clustering

Project Structure

src/ietf_analyzer/
    cli.py          # Click CLI entry point (~30 commands)
    fetcher.py      # IETF Datatracker API client
    analyzer.py     # Claude-based analysis (rating, ideas, gaps)
    embeddings.py   # Ollama embeddings + similarity + clustering
    db.py           # SQLite with FTS5 (8 tables)
    models.py       # Author, Draft, Rating dataclasses
    reports.py      # Markdown report generation
    authors.py      # Author network analysis
    search.py       # Hybrid FTS5 + embedding search
    classifier.py   # Two-stage Ollama classifier
    readiness.py    # Draft readiness scoring
    config.py       # Configuration

src/webui/
    app.py          # Flask application (20 API endpoints)
    data.py         # Data access layer with TypedDicts
    auth.py         # Admin authentication
    analytics.py    # GDPR-compliant pageview tracking
    templates/      # Jinja2 templates (23 pages)

data/
    drafts.db       # SQLite database
    reports/        # Generated reports + blog series
    .cache/         # Similarity matrix cache

paper/
    main.tex        # arXiv paper

Database Schema

Table	Purpose	Records
drafts	Draft metadata + full text	475
ratings	5-dimension ratings + summary + false_positive flag	475
embeddings	Semantic vectors (nomic-embed-text, 768-dim)	475
llm_cache	Claude API response cache (SHA-256 dedup)	~1,500
authors	Person records from Datatracker	713
draft_authors	Author-draft relationships	~1,400
ideas	Extracted + deduplicated technical ideas	501
gaps	Gap analysis results	12

Cost

Full pipeline for 475 drafts: ~$9-15 USD total

• Sonnet for rating + gap analysis (~$3)
• Haiku for bulk idea extraction (~$1)
• Ollama embeddings: free (local)

License

MIT — see LICENSE