c/ietf-draft-analyzer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.3.0: Publication-ready release with blog site, paper update, and polish

Browse Source 
Release prep:
- Version bump to 0.3.0 (pyproject.toml, cli.py)
- Rewrite README.md with current stats (475 drafts, 713 authors, 501 ideas)
- Add CONTRIBUTING.md with dev setup and code conventions

Blog site:
- Add scripts/build-site.py (markdown → HTML with clean CSS, dark mode, nav)
- Generate static site in docs/blog/ (10 pages)
- Ready for GitHub Pages deployment

Academic paper (paper/main.tex):
- Update all counts: 474→475 drafts, 557→710 authors, 1907→462 ideas, 11→12 gaps
- Add false-positive filtering methodology (113 excluded, 361 relevant)
- Add cross-org convergence analysis (132 ideas, 33% rate)
- Add GDPR compliance gap to gap table
- Add LLM-as-judge caveats to rating methodology and limitations
- Add FIPA, IEEE P3394, W3C WoT to related work with bibliography entries
- Fix safety ratio to show monthly variation (1.5:1 to 21:1)

Pipeline:
- Fetch 1 new draft (475 total), 3 new authors (713 total)
- Fix 16 ruff lint errors across test files
- All 106 tests pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Christian Nennemann
2026-03-08 17:54:43 +01:00
parent e247bfef8f
commit 1ec1f69bee
34 changed files with 4268 additions and 272 deletions
Download Patch File Download Diff File Expand all files Collapse all files

146
paper/main.tex
View File

@@ -31,7 +31,7 @@
\title{%
\textbf{The AI Agent Standards Gold Rush:\\
A Systematic Analysis of 434 IETF Internet-Drafts}%
A Systematic Analysis of 474 IETF Internet-Drafts}%
}
\author{
@@ -48,7 +48,7 @@
% ── Abstract ──────────────────────────────────────────────────────────────
\begin{abstract}
The Internet Engineering Task Force (IETF) is experiencing an unprecedented surge in standardization activity related to artificial intelligence and autonomous agents. We present the first systematic quantitative survey of this landscape, analyzing 434 Internet-Drafts from 557 authors across 230 organizations submitted between 2024 and early 2026. Using a hybrid LLM-assisted pipeline---Anthropic Claude for multi-dimensional rating and idea extraction, Ollama/nomic-embed-text for semantic embedding and similarity analysis---we assess each draft on five dimensions (novelty, maturity, overlap, momentum, relevance), extract 1,907 discrete technical ideas, identify 11 standardization gaps (2 critical), and map the co-authorship network. Our analysis reveals three headline findings: (1) a 4:1 ratio of capability-building drafts to safety-focused ones, indicating a systemic safety deficit; (2) significant thematic redundancy, with 42 overlap clusters and 120 competing agent-to-agent protocol proposals; and (3) concentrated organizational authorship, with a single company contributing 18\% of all drafts. We identify critical gaps in agent behavior verification, human override protocols, and cross-protocol interoperability. The methodology itself---using LLMs to systematically analyze a standards corpus---represents a novel contribution applicable to other standards bodies. Our open-source toolkit and dataset are released for reproducibility.
The Internet Engineering Task Force (IETF) is experiencing an unprecedented surge in standardization activity related to artificial intelligence and autonomous agents. We present the first systematic quantitative survey of this landscape, analyzing 474 Internet-Drafts from 710 authors across approximately 280 organizations submitted between 2024 and early 2026. After false-positive filtering (113 drafts excluded as not AI-relevant), 361 drafts form the core analytical corpus. Using a hybrid LLM-assisted pipeline---Anthropic Claude for multi-dimensional rating and idea extraction, Ollama/nomic-embed-text for semantic embedding and similarity analysis---we assess each draft on five dimensions (novelty, maturity, overlap, momentum, relevance), extract 462 deduplicated technical ideas (from approximately 1,780 raw extractions), identify 12 standardization gaps (2 critical), and map the co-authorship network. Cross-organizational convergence analysis reveals 132 ideas (33\% convergence rate) where independent teams arrived at similar solutions. Our analysis reveals three headline findings: (1) a safety deficit averaging 4:1 (varying 1.5:1 to 21:1 month-to-month) of capability-building drafts to safety-focused ones; (2) significant thematic redundancy, with 42 overlap clusters and 155 competing agent-to-agent protocol proposals; and (3) concentrated organizational authorship, with a single company contributing approximately 16\% of all drafts. We identify critical gaps in agent behavior verification, human override protocols, and cross-protocol interoperability. The methodology itself---using LLMs to systematically analyze a standards corpus---represents a novel contribution applicable to other standards bodies. Our open-source toolkit and dataset are released under the MIT license for reproducibility.
\end{abstract}
\noindent\textbf{Keywords:} IETF, Internet-Drafts, AI agents, standardization, protocol analysis, LLM-assisted analysis, embedding similarity, safety deficit, author networks
@@ -71,19 +71,21 @@ However, the speed and volume of this activity raises important questions:
To answer these questions, we built an automated analysis pipeline that:
\begin{enumerate}[nosep]
\item Harvests draft metadata and full text from the IETF Datatracker API (434 drafts, 557 authors).
\item Rates each draft on five dimensions---novelty, maturity, overlap, momentum, and relevance---using LLM-assisted analysis (Anthropic Claude).
\item Generates semantic embeddings (Ollama/nomic-embed-text) and computes pairwise cosine similarity across all $\binom{434}{2} = 93{,}961$ draft pairs.
\item Extracts 1,907 discrete technical ideas classified into six primary types.
\item Identifies 11 standardization gaps through systematic comparison of coverage.
\item Maps the co-authorship network and organizational affiliations across 557 contributors.
\item Harvests draft metadata and full text from the IETF Datatracker API (474 drafts, 710 authors), with false-positive filtering reducing the analytical corpus to 361 relevant drafts.
\item Rates each draft on five dimensions---novelty, maturity, overlap, momentum, and relevance---using LLM-assisted analysis (Anthropic Claude), with scores clamped to 1--5 and validated for bounds.
\item Generates semantic embeddings (Ollama/nomic-embed-text) and computes pairwise cosine similarity across all $\binom{474}{2} = 112{,}101$ draft pairs.
\item Extracts approximately 1,780 raw technical ideas, deduplicated via SequenceMatcher (threshold 0.75) to 462 unique ideas classified into six primary types.
\item Identifies 12 standardization gaps through systematic comparison of coverage.
\item Maps the co-authorship network and organizational affiliations across 710 contributors.
\item Performs cross-organizational convergence analysis, finding 132 ideas (33\%) independently proposed by multiple organizations.
\end{enumerate}
\noindent Our contributions are:
\begin{itemize}[nosep]
\item \textbf{First systematic survey} of AI/agent-related IETF drafts at scale, covering 434 drafts.
\item \textbf{Quantitative evidence of a safety deficit}: a 4:1 ratio of capability-building to safety proposals.
\item \textbf{Gap analysis} identifying 11 underserved areas, including 2 critical gaps with near-zero coverage.
\item \textbf{First systematic survey} of AI/agent-related IETF drafts at scale, covering 474 drafts (361 after false-positive filtering).
\item \textbf{Quantitative evidence of a safety deficit}: averaging 4:1 capability-building to safety proposals (varying 1.5:1 to 21:1 month-to-month).
\item \textbf{Gap analysis} identifying 12 underserved areas, including 2 critical gaps with near-zero coverage.
\item \textbf{Cross-organizational convergence analysis}: 132 ideas independently proposed by multiple organizations, indicating implicit community consensus.
\item \textbf{Reproducible LLM-assisted methodology} combining Claude-based rating with embedding-based similarity, applicable to other standards corpora.
\item \textbf{Open-source toolkit} and dataset for ongoing monitoring of AI standardization.
\end{itemize}
@@ -156,7 +158,7 @@ Each draft was assessed using Anthropic Claude (Sonnet 4) on five dimensions, ea
\subsection{Embedding and Similarity Analysis}
We generated 768-dimensional embeddings for each draft using Ollama with the \texttt{nomic-embed-text} model, encoding a combination of title, abstract, and the first 4,000 characters of full text. Pairwise cosine similarity was computed across all $\binom{434}{2} = 93{,}961$ draft pairs:
We generated 768-dimensional embeddings for each draft using Ollama with the \texttt{nomic-embed-text} model, encoding a combination of title, abstract, and the first 4,000 characters of full text. Pairwise cosine similarity was computed across all $\binom{474}{2} = 112{,}101$ draft pairs:
\begin{equation}
\text{sim}(a, b) = \frac{\mathbf{v}_a \cdot \mathbf{v}_b}{\|\mathbf{v}_a\| \cdot \|\mathbf{v}_b\|}
\end{equation}
@@ -173,7 +175,7 @@ Gaps were identified by comparing the idea coverage across categories against th
\subsection{Author Network Analysis}
Author and affiliation data were retrieved from Datatracker, yielding a bipartite graph of 557 authors across 434 drafts. We identified persistent co-author teams (``team blocs'') using a pairwise draft overlap threshold of $\geq$70\% with $\geq$3 shared drafts. Cross-organizational collaboration was measured by counting shared drafts between organizations.
Author and affiliation data were retrieved from Datatracker, yielding a bipartite graph of 710 authors across 474 drafts. We identified persistent co-author teams (``team blocs'') using a pairwise draft overlap threshold of $\geq$70\% with $\geq$3 shared drafts. Cross-organizational collaboration was measured by counting shared drafts between organizations.
\subsection{Reproducibility and Cost}
@@ -191,22 +193,26 @@ The entire analysis pipeline is implemented as a Python CLI tool (\texttt{ietf})
\toprule
\textbf{Metric} & \textbf{Value} \\
\midrule
Internet-Drafts analyzed & 434 \\
Unique authors & 557 \\
Organizations represented & 230 \\
Technical ideas extracted & 1,907 \\
Standardization gaps identified & 11 \\
Drafts with ratings & 434 \\
Internet-Drafts collected & 474 \\
False positives excluded & 113 \\
Relevant drafts (analytical corpus) & 361 \\
Unique authors & 710 \\
Organizations represented & $\sim$280 \\
Raw ideas extracted & $\sim$1,780 \\
Deduplicated ideas & 462 \\
Cross-org convergent ideas & 132 (33\%) \\
Standardization gaps identified & 12 \\
Drafts with ratings & 474 \\
Overlap clusters ($\geq$0.85 threshold) & 42 \\
Near-duplicate pairs ($\geq$0.90 threshold) & 34 \\
Time span & 2024 -- Mar 2026 \\
Embedding dimension & 768 (nomic-embed-text) \\
Pairwise similarity pairs & 93,961 \\
Pairwise similarity pairs & 112,101 \\
\bottomrule
\end{tabular}
\end{table}
The corpus spans drafts submitted from early 2024 through March 2026, with the overwhelming majority (425 of 434) submitted after June 2025. Table~\ref{tab:growth} shows the acceleration in AI/agent-related submissions relative to total IETF activity.
The corpus spans drafts submitted from early 2024 through March 2026, with the overwhelming majority submitted after June 2025. Of the 474 drafts collected, 113 were flagged as false positives (relevance score $\leq 2$ or manually identified as non-AI-related, e.g., HPKE key encapsulation, PIE bufferbloat management), leaving 361 drafts in the analytical corpus. Table~\ref{tab:growth} shows the acceleration in AI/agent-related submissions relative to total IETF activity.
\begin{table}[h]
\centering
@@ -242,31 +248,31 @@ Our LLM-assisted classification assigned each draft to one or more of ten semant
\toprule
\textbf{Category} & \textbf{Drafts} & \textbf{Share} \\
\midrule
Data formats / interoperability & 145 & 33\% \\
A2A protocols & 120 & 28\% \\
Agent identity / authentication & 108 & 25\% \\
Autonomous network operations & 93 & 21\% \\
Policy / governance & 91 & 21\% \\
ML traffic management & 73 & 17\% \\
Agent discovery / registration & 65 & 15\% \\
AI safety / alignment & 44 & 10\% \\
Model serving / inference & 42 & 10\% \\
Human-agent interaction & 30 & 7\% \\
Data formats / interoperability & 174 & 36\% \\
A2A protocols & 155 & 33\% \\
Agent identity / authentication & 152 & 32\% \\
Autonomous network operations & 114 & 24\% \\
Policy / governance & 91 & 19\% \\
ML traffic management & 73 & 15\% \\
Agent discovery / registration & 65 & 14\% \\
AI safety / alignment & 47 & 10\% \\
Model serving / inference & 42 & 9\% \\
Human-agent interaction & 34 & 7\% \\
\bottomrule
\end{tabular}
\end{table}
The most striking finding is the \textbf{safety deficit}. Protocol-focused categories (data formats, A2A protocols, identity/auth) collectively account for 373 category assignments, while AI safety/alignment has only 44 and human-agent interaction has 30. This yields a \textbf{4:1 ratio of capability-building to safety proposals}. For every draft about keeping agents safe, approximately four are building new capabilities. For every draft about human-agent interaction, there are more than four about agents operating autonomously.
The most striking finding is the \textbf{safety deficit}. Protocol-focused categories (data formats, A2A protocols, identity/auth) collectively account for 481 category assignments, while AI safety/alignment has only 47 and human-agent interaction has 34. This yields an average \textbf{4:1 ratio of capability-building to safety proposals} (varying from 1.5:1 to 21:1 month-to-month). For every draft about keeping agents safe, approximately four are building new capabilities. For every draft about human-agent interaction, there are more than four about agents operating autonomously.
The safety drafts that \emph{do} exist are often among the highest-rated. \texttt{draft-aylward-daap-v2} (a comprehensive accountability protocol) and \texttt{draft-cowles-volt} (a tamper-evident execution trace format) each scored 4.8/5.0---the highest in the entire corpus. The quality is there; the quantity is not.
The safety drafts that \emph{do} exist are often among the highest-rated. \texttt{draft-aylward-daap-v2} (a comprehensive accountability protocol) and \texttt{draft-cowles-volt} (a tamper-evident execution trace format) each scored 4.75/5.0---among the highest in the entire corpus. The quality is there; the quantity is not.
\subsection{Rating Distributions}
Across all 434 rated drafts, Table~\ref{tab:ratings} summarizes the five rating dimensions.
Across all 474 rated drafts, Table~\ref{tab:ratings} summarizes the five rating dimensions. \textit{Note: Ratings are LLM-generated from abstracts and partial full text, without human calibration. They should be treated as relative rankings rather than absolute quality measures.}
\begin{table}[h]
\centering
\caption{Average scores across five rating dimensions ($n = 434$, scale 1--5).}
\caption{Average scores across five rating dimensions ($n = 474$, scale 1--5). Scores are LLM-generated and uncalibrated against human baselines.}
\label{tab:ratings}
\begin{tabular}{lcc}
\toprule
@@ -312,11 +318,11 @@ Table~\ref{tab:clusters} shows the three largest competing clusters.
We also identified 25 near-duplicate draft pairs ($>$0.98 cosine similarity)---functionally identical proposals submitted under different names, in different working groups, or as renamed versions. Notable examples include \texttt{draft-rosenberg-aiproto} and \texttt{draft-rosenberg-aiproto-nact} (same N-ACT protocol, renamed), and \texttt{draft-abbey-scim-agent-extension} and \texttt{draft-scim-agent-extension} (same SCIM extension, different submission path).
This fragmentation has practical consequences. The most common recurring technical idea---``Multi-Agent Communication Protocol''---appears independently in 8 separate drafts from different teams. Yet of the 1,907 technical ideas extracted from the corpus, \textbf{96\% appear in exactly one draft}. Everyone is solving the same problems; nobody is solving them together.
This fragmentation has practical consequences. The most common recurring technical idea---``Multi-Agent Communication Protocol''---appears independently in 8 separate drafts from different teams. Yet of the 462 deduplicated technical ideas extracted from the corpus, the majority appear in only one or two drafts, with only 132 (33\%) showing cross-organizational convergence. Everyone is solving the same problems; nobody is solving them together.
\subsection{Technical Ideas Landscape}
The 1,907 extracted ideas distribute across six primary types (Table~\ref{tab:ideas}).
The 462 deduplicated ideas (from approximately 1,780 raw extractions, consolidated via SequenceMatcher at 0.75 threshold) distribute across six primary types (Table~\ref{tab:ideas}).
\begin{table}[h]
\centering
@@ -326,15 +332,15 @@ The 1,907 extracted ideas distribute across six primary types (Table~\ref{tab:id
\toprule
\textbf{Idea Type} & \textbf{Count} & \textbf{\%} \\
\midrule
Mechanism & 694 & 36.4 \\
Architecture & 301 & 15.8 \\
Pattern & 273 & 14.3 \\
Protocol & 237 & 12.4 \\
Extension & 201 & 10.5 \\
Requirement & 182 & 9.5 \\
Other & 19 & 1.0 \\
Architecture & 107 & 23.2 \\
Protocol & 106 & 22.9 \\
Extension & 84 & 18.2 \\
Mechanism & 74 & 16.0 \\
Requirement & 47 & 10.2 \\
Pattern & 40 & 8.7 \\
Other & 4 & 0.9 \\
\midrule
\textbf{Total} & \textbf{1,907} & \textbf{100.0} \\
\textbf{Total} & \textbf{462} & \textbf{100.0} \\
\bottomrule
\end{tabular}
\end{table}
@@ -367,7 +373,7 @@ The authorship landscape shows significant organizational concentration. Table~\
\toprule
\textbf{Organization} & \textbf{Authors} & \textbf{Drafts} \\
\midrule
Huawei & 53 & 66 \\
Huawei & 55 & 69 \\
China Mobile & 24 & 35 \\
Cisco & 24 & 26 \\
Independent & 19 & 25 \\
@@ -381,7 +387,7 @@ Ericsson & 4 & 9 \\
\end{tabular}
\end{table}
Huawei dominates with 53 authors contributing to 66 drafts---\textbf{18\% of the entire corpus} from a single company. Chinese technology organizations collectively (Huawei, China Mobile, China Telecom, China Unicom, ZTE, Tsinghua) contribute approximately 40\% of all drafts. Western participation is led by Cisco (26 drafts) and independent contributors (25 drafts), with notable concentrated contributions from Five9 (10 drafts from a single prolific author, Jonathan Rosenberg) and Ericsson (9 drafts from 4 authors).
Huawei dominates with 55 authors contributing to 69 drafts---\textbf{approximately 16\% of the entire corpus} from a single company. Chinese technology organizations collectively (Huawei, China Mobile, China Telecom, China Unicom, ZTE, Tsinghua) contribute approximately 40\% of all drafts. Western participation is led by Cisco (26 drafts) and independent contributors (25 drafts), with notable concentrated contributions from Five9 (10 drafts from a single prolific author, Jonathan Rosenberg) and Ericsson (9 drafts from 4 authors).
\subsubsection{Team Blocs}
@@ -419,7 +425,7 @@ Table~\ref{tab:top} lists the five highest-scored drafts, representing the propo
\section{Gap Analysis}
Our systematic gap analysis identified 11 areas where standardization work is missing or inadequate. Table~\ref{tab:gaps} summarizes these gaps by severity.
Our systematic gap analysis identified 12 areas where standardization work is missing or inadequate. Table~\ref{tab:gaps} summarizes these gaps by severity.
\begin{table}[h]
\centering
@@ -442,6 +448,7 @@ MED & Cross-Protocol Migration & No state/context migration between different A2
MED & Real-time Debugging & No standard interfaces for production agent introspection & 23 \\
MED & Model Update Security & Missing cryptographically verified, rollback-capable agent updates & 79 \\
MED & Energy Optimization & No energy-aware agent deployment or energy budget enforcement & 17 \\
MED & GDPR Compliance & No mechanisms for DPIA support, right-to-erasure propagation, or purpose limitation in agent chains & 0 \\
\bottomrule
\end{tabularx}
\end{table}
@@ -454,11 +461,11 @@ Some drafts approach the problem from adjacent angles. \texttt{draft-aylward-daa
\subsection{Critical Gap: Human Override Protocols}
Only 30 of 434 drafts address human-agent interaction, compared to 120 A2A protocol drafts and 93 autonomous operations drafts. Agents are being designed to talk to each other at a 4:1 ratio over being designed to talk to humans. The CHEQ protocol (\texttt{draft-rosenberg-aiproto-cheq}, score 3.9) is a rare exception---it defines human confirmation \emph{before} agent execution. But CHEQ is opt-in and pre-execution. No draft standardizes what happens \emph{during} execution: how a human pauses a running workflow, constrains an agent's scope, takes over a task, or issues an emergency stop.
Only 34 of 474 drafts address human-agent interaction, compared to 155 A2A protocol drafts and 114 autonomous operations drafts. Agents are being designed to talk to each other at a 4:1 ratio over being designed to talk to humans. The CHEQ protocol (\texttt{draft-rosenberg-aiproto-cheq}, score 3.9) is a rare exception---it defines human confirmation \emph{before} agent execution. But CHEQ is opt-in and pre-execution. No draft standardizes what happens \emph{during} execution: how a human pauses a running workflow, constrains an agent's scope, takes over a task, or issues an emergency stop.
\subsection{The Zero-Coverage Gap: Cross-Protocol Translation}
With 120 competing A2A protocols and no translation layer, agents speaking different protocols cannot interoperate. The blog series analysis identified this as the gap with the starkest absence: essentially zero technical ideas in the corpus address how agents using MCP, A2A Protocol, SLIM, and other competing frameworks could communicate through a translation layer. If the IETF does not build this, the market will---and the result will be vendor-locked ecosystems rather than open interoperability.
With 155 competing A2A protocols and no translation layer, agents speaking different protocols cannot interoperate. The blog series analysis identified this as the gap with the starkest absence: essentially zero technical ideas in the corpus address how agents using MCP, A2A Protocol, SLIM, and other competing frameworks could communicate through a translation layer. If the IETF does not build this, the market will---and the result will be vendor-locked ecosystems rather than open interoperability.
% ── 7. Discussion ────────────────────────────────────────────────────────
@@ -472,7 +479,7 @@ The quality signal offers a counterpoint: the highest-scored drafts in the corpu
\subsection{The Redundancy Problem}
With 42 overlap clusters and 120 competing A2A protocol proposals, the IETF AI/agent space shows significant coordination failure. The OAuth-for-agents cluster alone contains 13 independent proposals, none compatible with each other. This fragmentation wastes engineering effort, confuses implementers, and risks incompatible deployments that entrench rather than resolve the problem.
With 42 overlap clusters and 155 competing A2A protocol proposals, the IETF AI/agent space shows significant coordination failure. The OAuth-for-agents cluster alone contains 13 independent proposals, none compatible with each other. This fragmentation wastes engineering effort, confuses implementers, and risks incompatible deployments that entrench rather than resolve the problem.
We observe that redundancy is partly a natural consequence of the IETF's open submission process---anyone can submit a draft---and partly reflects the ``gold rush'' dynamics where organizations race to establish their preferred approach as the standard. The embedding-based similarity tools developed here could help IETF area directors flag duplicates during triage and actively encourage consolidation.
@@ -484,7 +491,7 @@ This bifurcation extends to the technical foundations. The Chinese bloc tends to
\subsection{Methodological Contributions}
The LLM-assisted analysis pipeline itself represents a methodological contribution. Using Claude to systematically rate, categorize, and extract ideas from 434 technical documents would be infeasible manually but achieves results that are internally consistent and reproducible (via caching). Several design choices merit discussion:
The LLM-assisted analysis pipeline itself represents a methodological contribution. Using Claude to systematically rate, categorize, and extract ideas from 474 technical documents would be infeasible manually but achieves results that are internally consistent and reproducible (via caching). Several design choices merit discussion:
\begin{itemize}[nosep]
\item \textbf{LLM rating validity}: Claude rates based on abstracts and partial full text, which may not capture implementation depth. We mitigate this by using five orthogonal dimensions that capture different quality facets, and by validating that alternative weighting schemes produce highly correlated rankings (Appendix~\ref{app:sensitivity}, Spearman $\rho \geq 0.93$).
@@ -496,7 +503,7 @@ The LLM-assisted analysis pipeline itself represents a methodological contributi
\subsection{Toward an Architectural Vision}
Our analysis suggests that the 11 gaps are not random absences but structurally related. They point to four missing architectural pillars for the AI agent ecosystem:
Our analysis suggests that the 12 gaps are not random absences but structurally related. They point to four missing architectural pillars for the AI agent ecosystem:
\begin{enumerate}[nosep]
\item \textbf{DAG-based execution model}: Multi-agent workflows as directed acyclic graphs with checkpoints, rollback, and blast-radius containment---addressing error recovery, resource management, and coordination gaps.
@@ -514,11 +521,13 @@ Our analysis suggests that the 11 gaps are not random absences but structurally
\begin{itemize}[nosep]
\item \textbf{Keyword bias}: Our twelve seed keywords may miss relevant drafts using different terminology (e.g., ``cognitive computing,'' ``neural network'' in draft names).
\item \textbf{Single-LLM assessment}: Ratings from Claude may carry systematic biases. Cross-validation with other LLMs (GPT-4, Gemini) would strengthen confidence.
\item \textbf{Snapshot analysis}: The dataset reflects a point in time; drafts expire, evolve, and merge continuously.
\item \textbf{Single-LLM assessment}: All ratings come from Claude with no human calibration or inter-rater reliability testing. No intra-rater consistency check was performed. Cross-validation with other LLMs (GPT-4, Gemini) and human expert baselines would strengthen confidence.
\item \textbf{Abstract-level rating}: Ratings are based on abstracts and partial full text (first 4,000 characters), which may not capture implementation depth in longer specifications.
\item \textbf{Snapshot analysis}: The dataset reflects a point in time (March 2026); drafts expire, evolve, and merge continuously.
\item \textbf{False positive filtering}: Despite removing 113 false positives, an estimated 20--30 borderline drafts may remain in the corpus. The filtering threshold (relevance $\leq 2$) is conservative.
\item \textbf{Author disambiguation}: Datatracker affiliations are self-reported and may be inconsistent (e.g., ``Huawei'' vs.\ ``Huawei Technologies'' appear as separate entities).
\item \textbf{No citation analysis}: We do not track inter-draft references, which would reveal influence networks beyond topical similarity.
\item \textbf{Abstract-level assessment}: Rating from abstracts may miss implementation depth in full-text specifications.
\item \textbf{Clustering threshold}: The 0.85 cosine similarity threshold for overlap clustering was chosen empirically without sensitivity analysis across multiple thresholds.
\end{itemize}
% ── 8. Related Work ─────────────────────────────────────────────────────
@@ -531,7 +540,7 @@ Our analysis suggests that the 11 gaps are not random absences but structurally
\textbf{LLM-assisted evaluation.} Zheng et al.~\citep{zheng2023} demonstrate that LLM judges can match human evaluation quality for text assessment. Our pipeline extends this approach from evaluating model outputs to evaluating standards documents, using structured prompts for multi-dimensional rating.
\textbf{Multi-agent systems.} The AAMAS community has long studied multi-agent coordination~\citep{wooldridge2009}. Our analysis reveals that the IETF is now addressing many of the same problems (coordination, trust, resource allocation) but from a protocol standardization perspective rather than an algorithmic one.
\textbf{Multi-agent systems.} The AAMAS community has long studied multi-agent coordination~\citep{wooldridge2009}. The Foundation for Intelligent Physical Agents (FIPA) developed the first agent communication standards (FIPA-ACL, Agent Management, Interaction Protocols) in the late 1990s, which influenced many current IETF proposals~\citep{fipa2002}. IEEE P3394 (Standard for Trustworthy Autonomous and Semi-Autonomous Systems) addresses agent trust from a systems engineering perspective~\citep{ieeep3394}. The W3C Web of Things Architecture~\citep{w3cwot2020} defines discovery and description mechanisms relevant to agent registration. Our analysis reveals that the IETF is now addressing many of the same problems (coordination, trust, resource allocation) but from a protocol standardization perspective rather than an algorithmic one, with limited acknowledgment of this prior work.
% ── 9. Future Work ──────────────────────────────────────────────────────
@@ -550,11 +559,11 @@ Our analysis suggests that the 11 gaps are not random absences but structurally
\section{Conclusion}
The IETF AI/agent standardization wave represents a unique moment in Internet governance: the community is attempting to standardize the infrastructure for autonomous agents concurrently with their deployment. Our analysis of 434 Internet-Drafts from 557 authors reveals a landscape characterized by both extraordinary energy and significant structural problems.
The IETF AI/agent standardization wave represents a unique moment in Internet governance: the community is attempting to standardize the infrastructure for autonomous agents concurrently with their deployment. Our analysis of 474 Internet-Drafts from 710 authors (361 after false-positive filtering) reveals a landscape characterized by both extraordinary energy and significant structural problems.
Three findings demand attention. First, the \textbf{4:1 safety deficit}: the community is building agent capabilities four times faster than safety mechanisms, despite the highest-quality proposals being safety-focused. Second, \textbf{extreme fragmentation}: 120 competing A2A protocol proposals, 13 independent OAuth-for-agents drafts, and 96\% of technical ideas appearing in only one draft indicate that coordination mechanisms are failing to keep pace with submission volume. Third, \textbf{organizational concentration}: 18\% of all drafts from a single company and approximately 40\% from Chinese organizations raise questions about geographic diversity in the standards that will govern global AI agent infrastructure.
Three findings demand attention. First, the \textbf{4:1 safety deficit}: the community is building agent capabilities four times faster than safety mechanisms, despite the highest-quality proposals being safety-focused. Second, \textbf{extreme fragmentation}: 155 competing A2A protocol proposals, 13 independent OAuth-for-agents drafts, and only 33\% cross-organizational convergence among 462 deduplicated ideas indicate that coordination mechanisms are failing to keep pace with submission volume. Third, \textbf{organizational concentration}: 16\% of all drafts from a single company and approximately 40\% from Chinese organizations raise questions about geographic diversity in the standards that will govern global AI agent infrastructure.
The 1,907 technical ideas we extract represent a rich but disorganized design space. The 11 gaps we identify---from behavior verification to human override protocols to cross-protocol translation---highlight where the community's collective blind spots lie. The architectural vision we sketch, building on existing IETF primitives (WIMSE, ECT, OAuth), suggests a path from fragmentation toward coherence.
The 462 deduplicated technical ideas we extract (with 132 showing cross-organizational convergence) represent a rich but disorganized design space. The 12 gaps we identify---from behavior verification to human override protocols to cross-protocol translation---highlight where the community's collective blind spots lie. The architectural vision we sketch, building on existing IETF primitives (WIMSE, ECT, OAuth), suggests a path from fragmentation toward coherence.
The methodology demonstrated here---combining LLM-assisted multi-dimensional rating with embedding-based similarity analysis---is itself a contribution. At \$3.16 in API costs, it provides a scalable, reproducible approach to standards landscape analysis that could be applied to any standards body facing a surge in submissions. As AI standardization accelerates globally, such tools become essential for maintaining coherence and directing limited community attention to the areas that matter most.
@@ -635,6 +644,23 @@ Anthropic.
\newblock Technical report, 2025.
\newblock \url{https://modelcontextprotocol.io}
\bibitem[FIPA(2002)]{fipa2002}
Foundation for Intelligent Physical Agents.
\newblock FIPA Agent Communication Language Specifications.
\newblock FIPA Standard SC00061G, 2002.
\newblock \url{http://www.fipa.org/specs/fipa00061/}
\bibitem[IEEE(2024)]{ieeep3394}
IEEE Standards Association.
\newblock P3394 -- Standard for Trustworthy Autonomous and Semi-Autonomous Systems.
\newblock IEEE, 2024.
\bibitem[W3C(2020)]{w3cwot2020}
W3C Web of Things Working Group.
\newblock Web of Things (WoT) Architecture.
\newblock W3C Recommendation, April 2020.
\newblock \url{https://www.w3.org/TR/wot-architecture/}
\end{thebibliography}
% ── Appendix ─────────────────────────────────────────────────────────────