IETF Draft Analyzer

Track, categorize, rate, and visualize AI/agent-related IETF Internet-Drafts.

260 drafts analyzed across 19 categories with 403 authors, 1,262 extracted ideas, and 12 identified gaps — spanning June 2025 to February 2026.

What This Does

The IETF is experiencing an unprecedented wave of standardization activity around AI agents. This tool provides a quantitative lens on that activity:

• Fetches draft metadata and full text from the IETF Datatracker API
• Rates each draft on 5 dimensions (novelty, maturity, overlap, momentum, relevance) using Claude
• Embeds drafts with Ollama for pairwise similarity and clustering
• Extracts discrete technical ideas and identifies landscape gaps
• Maps the author collaboration network and organizational affiliations
• Generates interactive visualizations, markdown reports, and a filterable browser
• Produces publication-ready figures for an arXiv paper

Quick Start

# Install
pip install -e .

# Set your API key (or add to .env file)
export ANTHROPIC_API_KEY=sk-ant-...

# Fetch drafts from IETF Datatracker
ietf fetch

# Rate all unrated drafts with Claude
ietf analyze --all

# Generate embeddings (requires Ollama running locally)
ietf embed

# Extract technical ideas
ietf ideas --all

# Identify gaps in the landscape
ietf gaps

# Generate all visualizations
ietf viz all

# Open the interactive browser
xdg-open data/figures/browser.html

CLI Commands

Core Pipeline

Command	Description
ietf fetch	Fetch AI/agent drafts from IETF Datatracker
ietf analyze --all	Rate all unrated drafts using Claude (5 dimensions + summary)
ietf embed	Generate semantic embeddings via Ollama
ietf ideas --all	Extract technical ideas from drafts using Claude
ietf gaps	Identify under-addressed areas in the landscape
ietf authors --fetch	Fetch author/affiliation data from Datatracker

Exploration

Command	Description
ietf list	List tracked drafts
ietf show <name>	Show detailed info for a specific draft
ietf search <query>	Full-text search across all stored drafts
ietf similar <name>	Find the most similar drafts by embedding similarity
ietf clusters	Find clusters of near-duplicate drafts
ietf compare <name1> <name2> ...	Compare drafts for overlap and unique contributions
ietf authors	Show top authors and their draft counts
ietf network	Show organizational collaboration network

Visualizations (ietf viz)

All outputs go to data/figures/. Interactive charts are standalone HTML files (no server needed).

Command	Output	Format
ietf viz all	Generate everything below	mixed
ietf viz browser	Filterable draft browser with search, category chips, score sliders	HTML
ietf viz landscape	t-SNE/UMAP 2D scatter of all drafts colored by category	HTML
ietf viz heatmap	260x260 clustered pairwise similarity matrix	PNG
ietf viz distributions	Violin plots for all 5 rating dimensions by category	PNG
ietf viz timeline	Stacked area chart of monthly submissions by category	HTML
ietf viz bubble	Novelty vs Maturity explorer (size=relevance, color=category)	HTML
ietf viz radar	Average rating profile per category	HTML
ietf viz network	Author co-authorship force-directed graph	HTML
ietf viz treemap	Category composition treemap (color=avg score)	HTML
ietf viz quality	Score vs uniqueness with quadrant annotations	HTML
ietf viz orgs	Organization contribution horizontal bar chart	HTML
ietf viz ideas	Ideas frequency by type	HTML

Reports (ietf report)

Markdown reports saved to data/reports/.

Command	Description
ietf report overview	Sortable table of all rated drafts with bar-chart scores
ietf report landscape	Category-grouped view with per-category rankings
ietf report timeline	Monthly submission volume and category trends
ietf report overlap-matrix	Top similar pairs, per-category overlap, cross-category matrix
ietf report authors	Top authors, organizations, collaboration pairs
ietf report digest	Weekly digest of recently fetched drafts
ietf report ideas	Most common ideas, unique ideas, ideas by type

Other

Command	Description
ietf draft-gen <topic>	Generate an Internet-Draft addressing a landscape gap
ietf config	Show or modify configuration

Rating System

Each draft is scored 1-5 on five dimensions:

Dimension	What it measures
Novelty	Originality relative to existing standards
Maturity	Completeness of specification
Overlap	Redundancy with other drafts (5 = heavily overlapping)
Momentum	Community engagement, revisions, adoption
Relevance	Importance to the AI/agent ecosystem

Composite score:

score = 0.30 * novelty + 0.25 * relevance + 0.20 * maturity + 0.15 * momentum + 0.10 * (6 - overlap)

Key Findings

• 36x growth in 9 months (2 drafts/month to 72)
• 7.9% of draft pairs exceed 0.80 cosine similarity — significant redundancy
• Safety deficit: AI safety proposals (36) are vastly outnumbered by protocol proposals (290+)
• Organizational concentration: Top 5 orgs contribute ~35% of all drafts
• 1,262 technical ideas extracted across 6 types (mechanism, architecture, protocol, pattern, extension, requirement)
• 12 identified gaps in the current landscape

Tech Stack

• Python 3.11+ with Click CLI
• SQLite with FTS5 full-text search and WAL mode
• Anthropic Claude (Sonnet 4) for analysis, rating, idea extraction, gap analysis
• Ollama (nomic-embed-text) for local embeddings and similarity
• Plotly for interactive HTML visualizations
• Matplotlib/Seaborn for publication-ready static figures
• NetworkX for author collaboration graph analysis
• NumPy/SciPy/scikit-learn for similarity computation and dimensionality reduction

Project Structure

src/ietf_analyzer/
    cli.py          # Click CLI entry point (all commands)
    fetcher.py      # IETF Datatracker API client
    analyzer.py     # Claude-based analysis, rating, idea extraction, gap analysis
    embeddings.py   # Ollama embeddings + cosine similarity + clustering
    db.py           # SQLite with FTS5 (7 tables: drafts, ratings, embeddings, llm_cache, authors, draft_authors, ideas, gaps)
    models.py       # Author, Draft, Rating dataclasses
    reports.py      # Markdown report generation
    visualize.py    # Interactive HTML + static PNG visualizations
    authors.py      # AuthorNetwork: Datatracker author fetching, collaboration graph
    draftgen.py     # Internet-Draft generation from gap analysis
    config.py       # Configuration with defaults

data/
    drafts.db       # SQLite database (all analysis data)
    reports/        # Generated markdown reports
    figures/        # Generated visualizations (HTML + PNG)

paper/
    main.tex        # arXiv paper: "The AI Agent Standardization Wave"
    export_figures.py  # Export interactive charts to static images
    Makefile        # Build: make pdf

Database Schema

Table	Purpose	Records
drafts	Draft metadata + full text	260
ratings	5-dimension AI ratings + summary	260
embeddings	Semantic vectors (nomic-embed-text)	260
llm_cache	Claude API response cache	~500
authors	Person records from Datatracker	403
draft_authors	Author-draft relationships	742
ideas	Extracted technical ideas	1,262
gaps	Gap analysis results	12
drafts_fts	FTS5 full-text search index	—

arXiv Paper

A 13-page paper is included in paper/main.tex:

The AI Agent Standardization Wave: A Quantitative Analysis of 260 IETF Internet-Drafts on Autonomous Agents and Artificial Intelligence

Build with:

cd paper
python3 export_figures.py   # copy/export figures
pdflatex main.tex           # compile (run twice for references)

Configuration

# Show current config
ietf config

# Change Claude model
ietf config --set claude_model claude-sonnet-4-20250514

# API key via .env file (auto-loaded)
echo "ANTHROPIC_API_KEY=sk-ant-..." > .env

Cost

Full analysis of 260 drafts consumed ~475K API tokens (rating + idea extraction + gap analysis). At current Sonnet pricing, this is approximately $2-3 USD.

License

MIT