feat(survey): add IETF landscape survey (kappa, phase0, rerate), gaps update; bump wimse-ect; gitignore run logs
This commit is contained in:
24
workspace/drafts/landscape-survey/sections/00-abstract.tex
Normal file
24
workspace/drafts/landscape-survey/sections/00-abstract.tex
Normal file
@@ -0,0 +1,24 @@
|
||||
\begin{abstract}
|
||||
Between 2024 and 2026 the Internet Engineering Task Force (IETF) saw a sharp
|
||||
rise in Internet-Drafts addressing AI agents and autonomous systems: monthly
|
||||
submissions in our corpus grew from an average of 3.7 (June 2024--May 2025) to
|
||||
38.8 (since June 2025), peaking at 106 in March 2026---roughly $35\times$ a
|
||||
typical 2024 month. We present a quantitative survey of this emerging area
|
||||
based on a curated corpus of 524 AI/agent-related IETF Internet-Drafts spanning
|
||||
January 2024 to May 2026. We characterise the corpus along four axes: temporal
|
||||
submission dynamics, thematic category distribution, authorship and
|
||||
working-group structure, and semantic redundancy measured through text
|
||||
embeddings. Two findings stand out: the area is overwhelmingly
|
||||
\emph{pre-standardization}---87\% of drafts are individual submissions not
|
||||
adopted by any working group---and it is semantically dense, with 32\% of
|
||||
drafts having a near-duplicate (cosine $>0.9$) elsewhere in the corpus.
|
||||
Because the thematic categories are produced by an LLM-assisted pipeline, we
|
||||
explicitly quantify their reliability through a two-model re-rating experiment:
|
||||
categorical assignment is substantially reproducible (Cohen's $\kappa = 0.65$),
|
||||
whereas LLM ordinal \emph{quality} scores such as novelty and overlap are not
|
||||
($\kappa_w = 0.13$--$0.21$). We therefore report the category landscape but
|
||||
deliberately exclude quality scores from our findings, and we argue this
|
||||
distinction is a general caution for the growing practice of LLM-assisted
|
||||
corpus analysis. All data, queries, and rating artifacts are released for
|
||||
reproduction.
|
||||
\end{abstract}
|
||||
Reference in New Issue
Block a user