feat(survey): add IETF landscape survey (kappa, phase0, rerate), gaps update; bump wimse-ect; gitignore run logs

workspace/drafts/landscape-survey/sections/01-intro.tex

@@ -0,0 +1,61 @@
+\section{Introduction}
+\label{sec:intro}
+
+Standardization activity around AI agents and autonomous systems has surged at
+the Internet Engineering Task Force (IETF). In the corpus studied here, monthly
+submissions of AI/agent-related Internet-Drafts grew from an average of $3.7$
+per month over June 2024--May 2025 to $38.8$ per month since June 2025, peaking
+at $106$ drafts in March 2026---roughly $35\times$ a typical 2024 month. A surge
+of this size, concentrated in barely a year, makes the area difficult to
+navigate: there is no settled taxonomy, the boundaries between proposals are
+fluid, and it is unclear how much of the activity represents distinct technical
+work versus overlapping attempts at the same problems.
+
+A natural response is to characterise the space quantitatively, and
+LLM-assisted corpus analyses are an increasingly common instrument for doing
+so---using a large language model to classify documents, score them, and
+summarise trends at a scale that manual reading cannot reach
+\cite{tan2024survey,gilardi2023chatgpt}. Such studies, however, rarely report whether the labels they rely on
+are reproducible. An LLM applied to the same documents under a different model,
+or even a different run, may produce different categories and scores; without a
+reliability check, it is impossible to tell which conclusions rest on stable
+signal and which on rater noise.
+
+To our knowledge no verified quantitative survey of the AI-agent
+standardization space at the IETF exists. We address this gap with a
+descriptive survey that is explicit about which of its measurements can be
+trusted. This paper is deliberately neutral: it characterises the landscape
+rather than advocating for any particular proposal or design.
+
+\paragraph{Contributions.}
+\begin{itemize}[leftmargin=1.4em,topsep=2pt,itemsep=1pt]
+  \item A curated corpus of \textbf{524} AI/agent-related IETF Internet-Drafts
+        spanning January 2024 to May 2026, constructed by keyword candidate
+        selection followed by explicit false-positive filtering
+        (Section~\ref{sec:method}).
+  \item A characterization of the corpus along four axes---temporal submission
+        dynamics, thematic category distribution, authorship and working-group
+        structure, and semantic redundancy measured through text embeddings
+        (Section~\ref{sec:landscape}).
+  \item An explicit two-model inter-rater reliability check that separates the
+        labels we can trust from those we cannot: categorical assignment is
+        substantially reproducible (Cohen's $\kappa \approx 0.65$), whereas the
+        LLM ordinal \emph{quality} scores are not ($\kappa_w = 0.13$--$0.21$ for
+        the least stable dimensions). We report the former and exclude the
+        latter (Section~\ref{sec:reliability}).
+  \item A full release of the data, queries, and rating artifacts used in the
+        analysis, so that every reported number can be recomputed.
+\end{itemize}
+
+\paragraph{Roadmap.}
+Section~\ref{sec:method} describes corpus construction, the LLM-assisted
+classification pipeline, and the embedding setup.
+Section~\ref{sec:landscape} presents the landscape across the four axes:
+temporal dynamics (Section~\ref{sec:temporal}), category distribution
+(Section~\ref{sec:categories}), authorship and working-group structure
+(Section~\ref{sec:authors}), and semantic redundancy
+(Section~\ref{sec:redundancy}).
+Section~\ref{sec:reliability} reports the two-model reliability experiment that
+underwrites the category labels and disqualifies the quality scores.
+Section~\ref{sec:discussion} discusses implications and limitations, and
+Section~\ref{sec:conclusion} concludes with directions for future work.