c/ietf-draft-analyzer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(survey): add IETF landscape survey (kappa, phase0, rerate), gaps update; bump wimse-ect; gitignore run logs

Browse Source
This commit is contained in:
Christian Nennemann
2026-05-25 12:35:31 +02:00
parent 89df70a6c0
commit 6e6e0489b8
43 changed files with 11956 additions and 1384 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
workspace/drafts/landscape-survey/Makefile Normal file
View File

@@ -0,0 +1,12 @@
.PHONY: pdf clean
pdf: main.pdf
main.pdf: main.tex references.bib
pdflatex -interaction=nonstopmode main.tex
bibtex main
pdflatex -interaction=nonstopmode main.tex
pdflatex -interaction=nonstopmode main.tex
clean:
rm -f main.aux main.bbl main.blg main.log main.out main.pdf main.toc

840
workspace/drafts/landscape-survey/build.log Normal file
View File

@@ -0,0 +1,840 @@
pdflatex -interaction=nonstopmode main.tex
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode
(./main.tex
LaTeX2e <2022-11-01> patch level 1
L3 programming layer <2023-02-22>
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2022/07/02 v1.4n Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size11.clo))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/lm/lmodern.sty)
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/generic/iftex/ifvtex.sty
(/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def)))
(/usr/share/texlive/texmf-dist/tex/latex/booktabs/booktabs.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty
(/usr/share/texlive/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdftexcmds/pdftexcmds.sty
(/usr/share/texlive/texmf-dist/tex/generic/infwarerr/infwarerr.sty))
(/usr/share/texlive/texmf-dist/tex/latex/kvsetkeys/kvsetkeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/kvdefinekeys/kvdefinekeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdfescape/pdfescape.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hycolor/hycolor.sty)
(/usr/share/texlive/texmf-dist/tex/latex/letltxmacro/letltxmacro.sty)
(/usr/share/texlive/texmf-dist/tex/latex/auxhook/auxhook.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty
(/usr/share/texlive/texmf-dist/tex/latex/refcount/refcount.sty)
(/usr/share/texlive/texmf-dist/tex/generic/gettitlestring/gettitlestring.sty
(/usr/share/texlive/texmf-dist/tex/latex/kvoptions/kvoptions.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def)
(/usr/share/texlive/texmf-dist/tex/generic/intcalc/intcalc.sty)
(/usr/share/texlive/texmf-dist/tex/generic/etexcmds/etexcmds.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def)
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)
(/usr/share/texlive/texmf-dist/tex/generic/bitset/bitset.sty
(/usr/share/texlive/texmf-dist/tex/generic/bigintcalc/bigintcalc.sty))
(/usr/share/texlive/texmf-dist/tex/latex/base/atbegshi-ltx.sty))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def
(/usr/share/texlive/texmf-dist/tex/latex/base/atveryend-ltx.sty)
(/usr/share/texlive/texmf-dist/tex/latex/rerunfilecheck/rerunfilecheck.sty
(/usr/share/texlive/texmf-dist/tex/generic/uniquecounter/uniquecounter.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/mathcolor.ltx))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/share/texlive/texmf-dist/tex/latex/float/float.sty)
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption.sty
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption3.sty))
(/usr/share/texlive/texmf-dist/tex/latex/tools/tabularx.sty
(/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty))
(/usr/share/texlive/texmf-dist/tex/latex/enumitem/enumitem.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/pgf.revision.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeyslibraryfiltered
.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.def
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-pdf.de
f)))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol.code.
tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigonomet
ric.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.random.cod
e.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.comparison
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integerari
thmetics.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfint.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconstruct.
code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicstate.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransformation
s.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.code.t
ex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathprocessing
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransparency.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorerdf.code.tex))
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.code.tex
) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code.tex
)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-0-65
.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-1-18
.sty)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgffor.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/math/pgfmath.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik
zlibrarytopaths.code.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmr.fd)
(/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-pdftex.def)
(./main.aux)
*geometry* driver: auto-detecting
*geometry* detected driver: pdftex
(/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
) (/usr/share/texlive/texmf-dist/tex/latex/epstopdf-pkg/epstopdf-base.sty
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg))
(./main.out) (./main.out) (/usr/share/texlive/texmf-dist/tex/latex/lm/ot1lmr.fd
) (/usr/share/texlive/texmf-dist/tex/latex/lm/omllmm.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omslmsy.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omxlmex.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmtt.fd)
(./sections/00-abstract.tex) (./sections/01-intro.tex
LaTeX Warning: Citation `tan2024survey' on page 1 undefined on input line 18.
LaTeX Warning: Citation `gilardi2023chatgpt' on page 1 undefined on input line
18.
[1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}{/usr/share/texlive/texmf-
dist/fonts/enc/dvips/lm/lm-ec.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvip
s/lm/lm-rm.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathsy.enc}
{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathit.enc}]
(/usr/share/texlive/texmf-dist/tex/latex/lm/ts1lmr.fd))
(./sections/02-related.tex
LaTeX Warning: Citation `mcquistin2021characterising' on page 2 undefined on in
put line 9.
LaTeX Warning: Citation `zhang2025affiliations' on page 2 undefined on input li
ne 13.
LaTeX Warning: Citation `tenoever2022aid' on page 2 undefined on input line 16.
LaTeX Warning: Citation `jimenez2024automating' on page 2 undefined on input li
ne 18.
LaTeX Warning: Citation `gilardi2023chatgpt' on page 2 undefined on input line
31.
LaTeX Warning: Citation `tan2024survey' on page 2 undefined on input line 34.
LaTeX Warning: Citation `yang2025agentprotocols' on page 2 undefined on input l
ine 37.
[2{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-ts1.enc}]
LaTeX Warning: Citation `reiss2023reliability' on page 3 undefined on input lin
e 48.
LaTeX Warning: Citation `wang2023nlgeval' on page 3 undefined on input line 52.
LaTeX Warning: Citation `cohen1960kappa' on page 3 undefined on input line 54.
LaTeX Warning: Citation `landis1977kappa' on page 3 undefined on input line 55.
LaTeX Warning: Citation `jimenez2024automating' on page 3 undefined on input li
ne 60.
LaTeX Warning: Citation `reimers2019sbert' on page 3 undefined on input line 69
.
LaTeX Warning: Citation `nussbaum2024nomic' on page 3 undefined on input line 7
1.
LaTeX Warning: Citation `mcquistin2021characterising' on page 3 undefined on in
put line 74.
) (./sections/03-method.tex [3]) (./sections/04-landscape.tex
Overfull \hbox (34.79286pt too wide) in paragraph at lines 53--1
\T1/lmr/m/n/10.95 scripts (\T1/lmtt/m/n/10.95 scripts/survey-phase0.py\T1/lmr/m
/n/10.95 , \T1/lmtt/m/n/10.95 scripts/rerate-intercoder.py\T1/lmr/m/n/10.95 , \
T1/lmtt/m/n/10.95 scripts/survey-kappa.py\T1/lmr/m/n/10.95 ).
LaTeX Warning: `h' float specifier changed to `ht'.
LaTeX Warning: `h' float specifier changed to `ht'.
[4{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathex.enc}])
(./sections/05-reliability.tex [5]) (./sections/06-discussion.tex [6]
Overfull \hbox (1.63pt too wide) in paragraph at lines 6--19
\T1/lmr/m/n/10.95 Two struc-tural fea-tures of the cor-pus re-in-force one an-o
ther. First, the area is \T1/lmr/m/it/10.95 pre-standardization\T1/lmr/m/n/10.9
5 :
[7]) (./sections/07-conclusion.tex) (./main.bbl
LaTeX Warning: Empty `thebibliography' environment on input line 3.
) [8] (./main.aux)
LaTeX Warning: There were undefined references.
LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.
Package rerunfilecheck Warning: File `main.out' has changed.
(rerunfilecheck) Rerun to get outlines right
(rerunfilecheck) or use package `bookmark'.
)
(see the transcript file for additional information)
pdfTeX warning (dest): name{table.caption.6} has been referenced but does not e
xist, replaced by a fixed one
pdfTeX warning (dest): name{figure.caption.3} has been referenced but does not
exist, replaced by a fixed one
pdfTeX warning (dest): name{table.caption.4} has been referenced but does not e
xist, replaced by a fixed one
pdfTeX warning (dest): name{subsection.5.2} has been referenced but does not ex
ist, replaced by a fixed one
pdfTeX warning (dest): name{subsection.5.1} has been referenced but does not ex
ist, replaced by a fixed one
pdfTeX warning (dest): name{subsection.2.4} has been referenced but does not ex
ist, replaced by a fixed one
pdfTeX warning (dest): name{subsection.2.3} has been referenced but does not ex
ist, replaced by a fixed one
pdfTeX warning (dest): name{subsection.2.2} has been referenced but does not ex
ist, replaced by a fixed one
pdfTeX warning (dest): name{subsection.2.1} has been referenced but does not ex
ist, replaced by a fixed one
</usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmbx10.pfb></usr/share/tex
live/texmf-dist/fonts/type1/public/lm/lmbx12.pfb></usr/share/texlive/texmf-dist
/fonts/type1/public/lm/lmex10.pfb></usr/share/texlive/texmf-dist/fonts/type1/pu
blic/lm/lmmi10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmmi7.p
fb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmmi8.pfb></usr/share/t
exlive/texmf-dist/fonts/type1/public/lm/lmr10.pfb></usr/share/texlive/texmf-dis
t/fonts/type1/public/lm/lmr12.pfb></usr/share/texlive/texmf-dist/fonts/type1/pu
blic/lm/lmr8.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmri10.pf
b></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmsy10.pfb></usr/share/t
exlive/texmf-dist/fonts/type1/public/lm/lmtt10.pfb></usr/share/texlive/texmf-di
st/fonts/type1/public/lm/lmtt12.pfb>
Output written on main.pdf (8 pages, 268959 bytes).
Transcript written on main.log.
bibtex main
This is BibTeX, Version 0.99d (TeX Live 2023)
The top-level auxiliary file: main.aux
The style file: plain.bst
Database file #1: references.bib
pdflatex -interaction=nonstopmode main.tex
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode
(./main.tex
LaTeX2e <2022-11-01> patch level 1
L3 programming layer <2023-02-22>
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2022/07/02 v1.4n Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size11.clo))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/lm/lmodern.sty)
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/generic/iftex/ifvtex.sty
(/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def)))
(/usr/share/texlive/texmf-dist/tex/latex/booktabs/booktabs.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty
(/usr/share/texlive/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdftexcmds/pdftexcmds.sty
(/usr/share/texlive/texmf-dist/tex/generic/infwarerr/infwarerr.sty))
(/usr/share/texlive/texmf-dist/tex/latex/kvsetkeys/kvsetkeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/kvdefinekeys/kvdefinekeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdfescape/pdfescape.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hycolor/hycolor.sty)
(/usr/share/texlive/texmf-dist/tex/latex/letltxmacro/letltxmacro.sty)
(/usr/share/texlive/texmf-dist/tex/latex/auxhook/auxhook.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty
(/usr/share/texlive/texmf-dist/tex/latex/refcount/refcount.sty)
(/usr/share/texlive/texmf-dist/tex/generic/gettitlestring/gettitlestring.sty
(/usr/share/texlive/texmf-dist/tex/latex/kvoptions/kvoptions.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def)
(/usr/share/texlive/texmf-dist/tex/generic/intcalc/intcalc.sty)
(/usr/share/texlive/texmf-dist/tex/generic/etexcmds/etexcmds.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def)
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)
(/usr/share/texlive/texmf-dist/tex/generic/bitset/bitset.sty
(/usr/share/texlive/texmf-dist/tex/generic/bigintcalc/bigintcalc.sty))
(/usr/share/texlive/texmf-dist/tex/latex/base/atbegshi-ltx.sty))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def
(/usr/share/texlive/texmf-dist/tex/latex/base/atveryend-ltx.sty)
(/usr/share/texlive/texmf-dist/tex/latex/rerunfilecheck/rerunfilecheck.sty
(/usr/share/texlive/texmf-dist/tex/generic/uniquecounter/uniquecounter.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/mathcolor.ltx))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/share/texlive/texmf-dist/tex/latex/float/float.sty)
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption.sty
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption3.sty))
(/usr/share/texlive/texmf-dist/tex/latex/tools/tabularx.sty
(/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty))
(/usr/share/texlive/texmf-dist/tex/latex/enumitem/enumitem.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/pgf.revision.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeyslibraryfiltered
.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.def
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-pdf.de
f)))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol.code.
tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigonomet
ric.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.random.cod
e.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.comparison
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integerari
thmetics.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfint.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconstruct.
code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicstate.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransformation
s.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.code.t
ex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathprocessing
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransparency.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorerdf.code.tex))
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.code.tex
) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code.tex
)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-0-65
.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-1-18
.sty)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgffor.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/math/pgfmath.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik
zlibrarytopaths.code.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmr.fd)
(/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-pdftex.def)
(./main.aux)
*geometry* driver: auto-detecting
*geometry* detected driver: pdftex
(/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
) (/usr/share/texlive/texmf-dist/tex/latex/epstopdf-pkg/epstopdf-base.sty
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg))
(./main.out) (./main.out) (/usr/share/texlive/texmf-dist/tex/latex/lm/ot1lmr.fd
) (/usr/share/texlive/texmf-dist/tex/latex/lm/omllmm.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omslmsy.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omxlmex.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmtt.fd)
(./sections/00-abstract.tex) (./sections/01-intro.tex
LaTeX Warning: Citation `tan2024survey' on page 1 undefined on input line 18.
LaTeX Warning: Citation `gilardi2023chatgpt' on page 1 undefined on input line
18.
[1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}{/usr/share/texlive/texmf-
dist/fonts/enc/dvips/lm/lm-ec.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvip
s/lm/lm-rm.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathsy.enc}
{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathit.enc}]
(/usr/share/texlive/texmf-dist/tex/latex/lm/ts1lmr.fd))
(./sections/02-related.tex
LaTeX Warning: Citation `mcquistin2021characterising' on page 2 undefined on in
put line 9.
LaTeX Warning: Citation `zhang2025affiliations' on page 2 undefined on input li
ne 13.
LaTeX Warning: Citation `tenoever2022aid' on page 2 undefined on input line 16.
LaTeX Warning: Citation `jimenez2024automating' on page 2 undefined on input li
ne 18.
LaTeX Warning: Citation `gilardi2023chatgpt' on page 2 undefined on input line
31.
LaTeX Warning: Citation `tan2024survey' on page 2 undefined on input line 34.
LaTeX Warning: Citation `yang2025agentprotocols' on page 2 undefined on input l
ine 37.
[2{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-ts1.enc}]
LaTeX Warning: Citation `reiss2023reliability' on page 3 undefined on input lin
e 48.
LaTeX Warning: Citation `wang2023nlgeval' on page 3 undefined on input line 52.
LaTeX Warning: Citation `cohen1960kappa' on page 3 undefined on input line 54.
LaTeX Warning: Citation `landis1977kappa' on page 3 undefined on input line 55.
LaTeX Warning: Citation `jimenez2024automating' on page 3 undefined on input li
ne 60.
LaTeX Warning: Citation `reimers2019sbert' on page 3 undefined on input line 69
.
LaTeX Warning: Citation `nussbaum2024nomic' on page 3 undefined on input line 7
1.
LaTeX Warning: Citation `mcquistin2021characterising' on page 3 undefined on in
put line 74.
) (./sections/03-method.tex [3]) (./sections/04-landscape.tex
Overfull \hbox (34.79286pt too wide) in paragraph at lines 53--1
\T1/lmr/m/n/10.95 scripts (\T1/lmtt/m/n/10.95 scripts/survey-phase0.py\T1/lmr/m
/n/10.95 , \T1/lmtt/m/n/10.95 scripts/rerate-intercoder.py\T1/lmr/m/n/10.95 , \
T1/lmtt/m/n/10.95 scripts/survey-kappa.py\T1/lmr/m/n/10.95 ).
LaTeX Warning: `h' float specifier changed to `ht'.
LaTeX Warning: `h' float specifier changed to `ht'.
[4{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathex.enc}])
(./sections/05-reliability.tex [5]) (./sections/06-discussion.tex [6]
Overfull \hbox (1.63pt too wide) in paragraph at lines 6--19
\T1/lmr/m/n/10.95 Two struc-tural fea-tures of the cor-pus re-in-force one an-o
ther. First, the area is \T1/lmr/m/it/10.95 pre-standardization\T1/lmr/m/n/10.9
5 :
[7]) (./sections/07-conclusion.tex) (./main.bbl [8]) [9] (./main.aux)
LaTeX Warning: There were undefined references.
LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.
)
(see the transcript file for additional information)</usr/share/texlive/texmf-d
ist/fonts/type1/public/lm/lmbx10.pfb></usr/share/texlive/texmf-dist/fonts/type1
/public/lm/lmbx12.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmex
10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmmi10.pfb></usr/sh
are/texlive/texmf-dist/fonts/type1/public/lm/lmmi7.pfb></usr/share/texlive/texm
f-dist/fonts/type1/public/lm/lmmi8.pfb></usr/share/texlive/texmf-dist/fonts/typ
e1/public/lm/lmr10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmr
12.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmr8.pfb></usr/shar
e/texlive/texmf-dist/fonts/type1/public/lm/lmri10.pfb></usr/share/texlive/texmf
-dist/fonts/type1/public/lm/lmsy10.pfb></usr/share/texlive/texmf-dist/fonts/typ
e1/public/lm/lmtt10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lm
tt12.pfb>
Output written on main.pdf (9 pages, 277634 bytes).
Transcript written on main.log.
pdflatex -interaction=nonstopmode main.tex
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode
(./main.tex
LaTeX2e <2022-11-01> patch level 1
L3 programming layer <2023-02-22>
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2022/07/02 v1.4n Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size11.clo))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/lm/lmodern.sty)
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/generic/iftex/ifvtex.sty
(/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def)))
(/usr/share/texlive/texmf-dist/tex/latex/booktabs/booktabs.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty
(/usr/share/texlive/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdftexcmds/pdftexcmds.sty
(/usr/share/texlive/texmf-dist/tex/generic/infwarerr/infwarerr.sty))
(/usr/share/texlive/texmf-dist/tex/latex/kvsetkeys/kvsetkeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/kvdefinekeys/kvdefinekeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdfescape/pdfescape.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hycolor/hycolor.sty)
(/usr/share/texlive/texmf-dist/tex/latex/letltxmacro/letltxmacro.sty)
(/usr/share/texlive/texmf-dist/tex/latex/auxhook/auxhook.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty
(/usr/share/texlive/texmf-dist/tex/latex/refcount/refcount.sty)
(/usr/share/texlive/texmf-dist/tex/generic/gettitlestring/gettitlestring.sty
(/usr/share/texlive/texmf-dist/tex/latex/kvoptions/kvoptions.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def)
(/usr/share/texlive/texmf-dist/tex/generic/intcalc/intcalc.sty)
(/usr/share/texlive/texmf-dist/tex/generic/etexcmds/etexcmds.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def)
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)
(/usr/share/texlive/texmf-dist/tex/generic/bitset/bitset.sty
(/usr/share/texlive/texmf-dist/tex/generic/bigintcalc/bigintcalc.sty))
(/usr/share/texlive/texmf-dist/tex/latex/base/atbegshi-ltx.sty))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def
(/usr/share/texlive/texmf-dist/tex/latex/base/atveryend-ltx.sty)
(/usr/share/texlive/texmf-dist/tex/latex/rerunfilecheck/rerunfilecheck.sty
(/usr/share/texlive/texmf-dist/tex/generic/uniquecounter/uniquecounter.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/mathcolor.ltx))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/share/texlive/texmf-dist/tex/latex/float/float.sty)
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption.sty
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption3.sty))
(/usr/share/texlive/texmf-dist/tex/latex/tools/tabularx.sty
(/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty))
(/usr/share/texlive/texmf-dist/tex/latex/enumitem/enumitem.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/pgf.revision.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeyslibraryfiltered
.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.def
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-pdf.de
f)))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol.code.
tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigonomet
ric.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.random.cod
e.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.comparison
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integerari
thmetics.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfint.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconstruct.
code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicstate.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransformation
s.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.code.t
ex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathprocessing
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransparency.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorerdf.code.tex))
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.code.tex
) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code.tex
)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-0-65
.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-1-18
.sty)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgffor.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/math/pgfmath.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik
zlibrarytopaths.code.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmr.fd)
(/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-pdftex.def)
(./main.aux)
*geometry* driver: auto-detecting
*geometry* detected driver: pdftex
(/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
) (/usr/share/texlive/texmf-dist/tex/latex/epstopdf-pkg/epstopdf-base.sty
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg))
(./main.out) (./main.out) (/usr/share/texlive/texmf-dist/tex/latex/lm/ot1lmr.fd
) (/usr/share/texlive/texmf-dist/tex/latex/lm/omllmm.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omslmsy.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omxlmex.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmtt.fd)
(./sections/00-abstract.tex) (./sections/01-intro.tex [1{/var/lib/texmf/fonts/m
ap/pdftex/updmap/pdftex.map}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/l
m-ec.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-rm.enc}{/usr/shar
e/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathsy.enc}{/usr/share/texlive/texmf
-dist/fonts/enc/dvips/lm/lm-mathit.enc}]
(/usr/share/texlive/texmf-dist/tex/latex/lm/ts1lmr.fd))
(./sections/02-related.tex [2{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/
lm-ts1.enc}]) (./sections/03-method.tex [3]) (./sections/04-landscape.tex
Overfull \hbox (34.79286pt too wide) in paragraph at lines 53--1
\T1/lmr/m/n/10.95 scripts (\T1/lmtt/m/n/10.95 scripts/survey-phase0.py\T1/lmr/m
/n/10.95 , \T1/lmtt/m/n/10.95 scripts/rerate-intercoder.py\T1/lmr/m/n/10.95 , \
T1/lmtt/m/n/10.95 scripts/survey-kappa.py\T1/lmr/m/n/10.95 ).
LaTeX Warning: `h' float specifier changed to `ht'.
LaTeX Warning: `h' float specifier changed to `ht'.
[4{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathex.enc}])
(./sections/05-reliability.tex [5]) (./sections/06-discussion.tex [6]
Overfull \hbox (1.63pt too wide) in paragraph at lines 6--19
\T1/lmr/m/n/10.95 Two struc-tural fea-tures of the cor-pus re-in-force one an-o
ther. First, the area is \T1/lmr/m/it/10.95 pre-standardization\T1/lmr/m/n/10.9
5 :
[7]) (./sections/07-conclusion.tex) (./main.bbl [8]) [9] (./main.aux) )
(see the transcript file for additional information)</usr/share/texlive/texmf-d
ist/fonts/type1/public/lm/lmbx10.pfb></usr/share/texlive/texmf-dist/fonts/type1
/public/lm/lmbx12.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmex
10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmmi10.pfb></usr/sh
are/texlive/texmf-dist/fonts/type1/public/lm/lmmi7.pfb></usr/share/texlive/texm
f-dist/fonts/type1/public/lm/lmmi8.pfb></usr/share/texlive/texmf-dist/fonts/typ
e1/public/lm/lmr10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmr
12.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmr8.pfb></usr/shar
e/texlive/texmf-dist/fonts/type1/public/lm/lmri10.pfb></usr/share/texlive/texmf
-dist/fonts/type1/public/lm/lmsy10.pfb></usr/share/texlive/texmf-dist/fonts/typ
e1/public/lm/lmtt10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lm
tt12.pfb>
Output written on main.pdf (9 pages, 278200 bytes).
Transcript written on main.log.

214
workspace/drafts/landscape-survey/final.log Normal file
View File

@@ -0,0 +1,214 @@
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode
(./main.tex
LaTeX2e <2022-11-01> patch level 1
L3 programming layer <2023-02-22>
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2022/07/02 v1.4n Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size11.clo))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/lm/lmodern.sty)
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/generic/iftex/ifvtex.sty
(/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def)))
(/usr/share/texlive/texmf-dist/tex/latex/booktabs/booktabs.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty
(/usr/share/texlive/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdftexcmds/pdftexcmds.sty
(/usr/share/texlive/texmf-dist/tex/generic/infwarerr/infwarerr.sty))
(/usr/share/texlive/texmf-dist/tex/latex/kvsetkeys/kvsetkeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/kvdefinekeys/kvdefinekeys.sty)
(/usr/share/texlive/texmf-dist/tex/generic/pdfescape/pdfescape.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hycolor/hycolor.sty)
(/usr/share/texlive/texmf-dist/tex/latex/letltxmacro/letltxmacro.sty)
(/usr/share/texlive/texmf-dist/tex/latex/auxhook/auxhook.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty
(/usr/share/texlive/texmf-dist/tex/latex/refcount/refcount.sty)
(/usr/share/texlive/texmf-dist/tex/generic/gettitlestring/gettitlestring.sty
(/usr/share/texlive/texmf-dist/tex/latex/kvoptions/kvoptions.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def)
(/usr/share/texlive/texmf-dist/tex/generic/intcalc/intcalc.sty)
(/usr/share/texlive/texmf-dist/tex/generic/etexcmds/etexcmds.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def)
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)
(/usr/share/texlive/texmf-dist/tex/generic/bitset/bitset.sty
(/usr/share/texlive/texmf-dist/tex/generic/bigintcalc/bigintcalc.sty))
(/usr/share/texlive/texmf-dist/tex/latex/base/atbegshi-ltx.sty))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def
(/usr/share/texlive/texmf-dist/tex/latex/base/atveryend-ltx.sty)
(/usr/share/texlive/texmf-dist/tex/latex/rerunfilecheck/rerunfilecheck.sty
(/usr/share/texlive/texmf-dist/tex/generic/uniquecounter/uniquecounter.sty)))
(/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/mathcolor.ltx))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/share/texlive/texmf-dist/tex/latex/float/float.sty)
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption.sty
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption3.sty))
(/usr/share/texlive/texmf-dist/tex/latex/tools/tabularx.sty
(/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty))
(/usr/share/texlive/texmf-dist/tex/latex/enumitem/enumitem.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/pgf.revision.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeyslibraryfiltered
.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.def
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-pdf.de
f)))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol.code.
tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigonomet
ric.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.random.cod
e.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.comparison
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integerari
thmetics.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfint.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconstruct.
code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicstate.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransformation
s.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.code.t
ex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathprocessing
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.code.te
x)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransparency.c
ode.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorerdf.code.tex))
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.code.tex
) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code.tex
)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-0-65
.sty)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-1-18
.sty)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgffor.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/math/pgfmath.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.code.tex
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik
zlibrarytopaths.code.tex)))
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmr.fd)
(/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-pdftex.def)
(./main.aux)
*geometry* driver: auto-detecting
*geometry* detected driver: pdftex
(/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
) (/usr/share/texlive/texmf-dist/tex/latex/epstopdf-pkg/epstopdf-base.sty
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg))
(./main.out) (./main.out) (/usr/share/texlive/texmf-dist/tex/latex/lm/ot1lmr.fd
) (/usr/share/texlive/texmf-dist/tex/latex/lm/omllmm.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omslmsy.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/omxlmex.fd)
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmtt.fd)
(./sections/00-abstract.tex) (./sections/01-intro.tex [1{/var/lib/texmf/fonts/m
ap/pdftex/updmap/pdftex.map}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/l
m-ec.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-rm.enc}{/usr/shar
e/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathsy.enc}{/usr/share/texlive/texmf
-dist/fonts/enc/dvips/lm/lm-mathit.enc}]
(/usr/share/texlive/texmf-dist/tex/latex/lm/ts1lmr.fd))
(./sections/02-related.tex [2{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/
lm-ts1.enc}]) (./sections/03-method.tex [3]) (./sections/04-landscape.tex
Overfull \hbox (34.79286pt too wide) in paragraph at lines 53--1
\T1/lmr/m/n/10.95 scripts (\T1/lmtt/m/n/10.95 scripts/survey-phase0.py\T1/lmr/m
/n/10.95 , \T1/lmtt/m/n/10.95 scripts/rerate-intercoder.py\T1/lmr/m/n/10.95 , \
T1/lmtt/m/n/10.95 scripts/survey-kappa.py\T1/lmr/m/n/10.95 ).
LaTeX Warning: `h' float specifier changed to `ht'.
LaTeX Warning: `h' float specifier changed to `ht'.
[4{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathex.enc}])
(./sections/05-reliability.tex [5]) (./sections/06-discussion.tex [6]
Overfull \hbox (1.63pt too wide) in paragraph at lines 6--19
\T1/lmr/m/n/10.95 Two struc-tural fea-tures of the cor-pus re-in-force one an-o
ther. First, the area is \T1/lmr/m/it/10.95 pre-standardization\T1/lmr/m/n/10.9
5 :
[7]) (./sections/07-conclusion.tex) (./main.bbl [8]) [9] (./main.aux) )
(see the transcript file for additional information)</usr/share/texlive/texmf-d
ist/fonts/type1/public/lm/lmbx10.pfb></usr/share/texlive/texmf-dist/fonts/type1
/public/lm/lmbx12.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmex
10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmmi10.pfb></usr/sh
are/texlive/texmf-dist/fonts/type1/public/lm/lmmi7.pfb></usr/share/texlive/texm
f-dist/fonts/type1/public/lm/lmmi8.pfb></usr/share/texlive/texmf-dist/fonts/typ
e1/public/lm/lmr10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmr
12.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmr8.pfb></usr/shar
e/texlive/texmf-dist/fonts/type1/public/lm/lmri10.pfb></usr/share/texlive/texmf
-dist/fonts/type1/public/lm/lmsy10.pfb></usr/share/texlive/texmf-dist/fonts/typ
e1/public/lm/lmtt10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lm
tt12.pfb>
Output written on main.pdf (9 pages, 278200 bytes).
Transcript written on main.log.

94
workspace/drafts/landscape-survey/main.aux Normal file
View File

@@ -0,0 +1,94 @@
\relax
\providecommand\hyper@newdestlabel[2]{}
\providecommand\HyperFirstAtBeginDocument{\AtBeginDocument}
\HyperFirstAtBeginDocument{\ifx\hyper@anchor\@undefined
\global\let\oldnewlabel\newlabel
\gdef\newlabel#1#2{\newlabelxx{#1}#2}
\gdef\newlabelxx#1#2#3#4#5#6{\oldnewlabel{#1}{{#2}{#3}}}
\AtEndDocument{\ifx\hyper@anchor\@undefined
\let\newlabel\oldnewlabel
\fi}
\fi}
\global\let\hyper@last\relax
\gdef\HyperFirstAtBeginDocument#1{#1}
\providecommand\HyField@AuxAddToFields[1]{}
\providecommand\HyField@AuxAddToCoFields[2]{}
\citation{tan2024survey}
\citation{gilardi2023chatgpt}
\@writefile{toc}{\contentsline {section}{\numberline {1}Introduction}{1}{section.1}\protected@file@percent }
\newlabel{sec:intro}{{1}{1}{Introduction}{section.1}{}}
\citation{mcquistin2021characterising}
\citation{zhang2025affiliations}
\citation{tenoever2022aid}
\citation{jimenez2024automating}
\citation{gilardi2023chatgpt}
\citation{tan2024survey}
\citation{yang2025agentprotocols}
\@writefile{toc}{\contentsline {paragraph}{Contributions.}{2}{section*.1}\protected@file@percent }
\@writefile{toc}{\contentsline {paragraph}{Roadmap.}{2}{section*.2}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {2}Related Work}{2}{section.2}\protected@file@percent }
\newlabel{sec:related}{{2}{2}{Related Work}{section.2}{}}
\@writefile{toc}{\contentsline {paragraph}{Quantitative analysis of IETF activity.}{2}{section*.3}\protected@file@percent }
\@writefile{toc}{\contentsline {paragraph}{LLM-assisted text classification and annotation.}{2}{section*.4}\protected@file@percent }
\citation{reiss2023reliability}
\citation{wang2023nlgeval}
\citation{cohen1960kappa}
\citation{landis1977kappa}
\citation{jimenez2024automating}
\citation{reimers2019sbert}
\citation{nussbaum2024nomic}
\citation{mcquistin2021characterising}
\@writefile{toc}{\contentsline {paragraph}{Inter-rater reliability of LLM labels.}{3}{section*.5}\protected@file@percent }
\@writefile{toc}{\contentsline {paragraph}{Embedding-based semantic analysis of document corpora.}{3}{section*.6}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {3}Corpus and Method}{3}{section.3}\protected@file@percent }
\newlabel{sec:method}{{3}{3}{Corpus and Method}{section.3}{}}
\@writefile{toc}{\contentsline {subsection}{\numberline {3.1}Corpus construction}{3}{subsection.3.1}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.2}Thematic classification}{4}{subsection.3.2}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.3}Semantic embeddings}{4}{subsection.3.3}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.4}Reproducibility}{4}{subsection.3.4}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {4}The Landscape}{4}{section.4}\protected@file@percent }
\newlabel{sec:landscape}{{4}{4}{The Landscape}{section.4}{}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.1}Temporal dynamics}{4}{subsection.4.1}\protected@file@percent }
\newlabel{sec:temporal}{{4.1}{4}{Temporal dynamics}{subsection.4.1}{}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.2}Category distribution}{4}{subsection.4.2}\protected@file@percent }
\newlabel{sec:categories}{{4.2}{4}{Category distribution}{subsection.4.2}{}}
\@writefile{lof}{\contentsline {figure}{\numberline {1}{\ignorespaces Monthly counts of AI/agent IETF Internet-Drafts, January 2024--May 2026 ($n=524$). The April--May 2026 tail is provisional (indexing lag).\relax }}{5}{figure.caption.7}\protected@file@percent }
\providecommand*\caption@xref[2]{\@setref\relax\@undefined{#1}}
\newlabel{fig:temporal}{{1}{5}{Monthly counts of AI/agent IETF Internet-Drafts, January 2024--May 2026 ($n=524$). The April--May 2026 tail is provisional (indexing lag).\relax }{figure.caption.7}{}}
\@writefile{lot}{\contentsline {table}{\numberline {1}{\ignorespaces Distribution of the clean IETF corpus by primary category. $92.9\%$ of drafts also carry one or more secondary categories.\relax }}{5}{table.caption.8}\protected@file@percent }
\newlabel{tab:categories}{{1}{5}{Distribution of the clean IETF corpus by primary category. $92.9\%$ of drafts also carry one or more secondary categories.\relax }{table.caption.8}{}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.3}Authorship and working-group structure}{5}{subsection.4.3}\protected@file@percent }
\newlabel{sec:authors}{{4.3}{5}{Authorship and working-group structure}{subsection.4.3}{}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.4}Semantic redundancy}{5}{subsection.4.4}\protected@file@percent }
\newlabel{sec:redundancy}{{4.4}{5}{Semantic redundancy}{subsection.4.4}{}}
\@writefile{toc}{\contentsline {section}{\numberline {5}How Reliable Are the Labels?}{6}{section.5}\protected@file@percent }
\newlabel{sec:reliability}{{5}{6}{How Reliable Are the Labels?}{section.5}{}}
\@writefile{toc}{\contentsline {paragraph}{Category assignment is substantially reliable.}{6}{section*.9}\protected@file@percent }
\@writefile{lot}{\contentsline {table}{\numberline {2}{\ignorespaces Most-confused primary-category pairs between the two re-rating models. Disagreements concentrate on semantic neighbours.\relax }}{6}{table.caption.10}\protected@file@percent }
\newlabel{tab:confusion}{{2}{6}{Most-confused primary-category pairs between the two re-rating models. Disagreements concentrate on semantic neighbours.\relax }{table.caption.10}{}}
\@writefile{toc}{\contentsline {paragraph}{Ordinal quality scores are not reliable.}{6}{section*.11}\protected@file@percent }
\@writefile{lot}{\contentsline {table}{\numberline {3}{\ignorespaces Quadratic-weighted $\kappa $ for the ordinal quality dimensions. Low and inconsistent values---especially for \emph {overlap} and \emph {novelty}---indicate these scores are not reproducible across raters.\relax }}{6}{table.caption.12}\protected@file@percent }
\newlabel{tab:kappa}{{3}{6}{Quadratic-weighted $\kappa $ for the ordinal quality dimensions. Low and inconsistent values---especially for \emph {overlap} and \emph {novelty}---indicate these scores are not reproducible across raters.\relax }{table.caption.12}{}}
\@writefile{toc}{\contentsline {paragraph}{Consequence.}{7}{section*.13}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {6}Discussion and Limitations}{7}{section.6}\protected@file@percent }
\newlabel{sec:discussion}{{6}{7}{Discussion and Limitations}{section.6}{}}
\@writefile{toc}{\contentsline {subsection}{\numberline {6.1}Implications}{7}{subsection.6.1}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {6.2}Limitations}{7}{subsection.6.2}\protected@file@percent }
\bibstyle{plain}
\bibdata{references}
\bibcite{cohen1960kappa}{1}
\bibcite{gilardi2023chatgpt}{2}
\bibcite{jimenez2024automating}{3}
\bibcite{landis1977kappa}{4}
\bibcite{mcquistin2021characterising}{5}
\bibcite{nussbaum2024nomic}{6}
\bibcite{reimers2019sbert}{7}
\@writefile{toc}{\contentsline {section}{\numberline {7}Conclusion and Future Work}{8}{section.7}\protected@file@percent }
\newlabel{sec:conclusion}{{7}{8}{Conclusion and Future Work}{section.7}{}}
\bibcite{reiss2023reliability}{8}
\bibcite{tan2024survey}{9}
\bibcite{tenoever2022aid}{10}
\bibcite{wang2023nlgeval}{11}
\bibcite{yang2025agentprotocols}{12}
\bibcite{zhang2025affiliations}{13}
\gdef \@abspage@last{9}

83
workspace/drafts/landscape-survey/main.bbl Normal file
View File

@@ -0,0 +1,83 @@
\begin{thebibliography}{10}
\bibitem{cohen1960kappa}
Jacob Cohen.
\newblock A coefficient of agreement for nominal scales.
\newblock {\em Educational and Psychological Measurement}, 20(1):37--46, 1960.
\bibitem{gilardi2023chatgpt}
Fabrizio Gilardi, Meysam Alizadeh, and Ma{\"e}l Kubli.
\newblock {ChatGPT} outperforms crowd-workers for text-annotation tasks.
\newblock {\em Proceedings of the National Academy of Sciences},
120(30):e2305016120, 2023.
\bibitem{jimenez2024automating}
Jaime Jim{\'e}nez.
\newblock Automating {IETF} insights generation with {AI}, 2024.
\bibitem{landis1977kappa}
J.~Richard Landis and Gary~G. Koch.
\newblock The measurement of observer agreement for categorical data.
\newblock {\em Biometrics}, 33(1):159--174, 1977.
\bibitem{mcquistin2021characterising}
Stephen McQuistin, Mladen Karan, Prashant Khare, Colin Perkins, Gareth Tyson,
Matthew Purver, Patrick Healey, Waleed Iqbal, Junaid Qadir, and Ignacio
Castro.
\newblock Characterising the {IETF} through the lens of {RFC} deployment.
\newblock In {\em Proceedings of the 21st ACM Internet Measurement Conference
({IMC} '21)}, pages 137--149, 2021.
\bibitem{nussbaum2024nomic}
Zach Nussbaum, John~X. Morris, Brandon Duderstadt, and Andriy Mulyar.
\newblock Nomic embed: Training a reproducible long context text embedder,
2024.
\bibitem{reimers2019sbert}
Nils Reimers and Iryna Gurevych.
\newblock Sentence-{BERT}: Sentence embeddings using siamese {BERT}-networks.
\newblock In {\em Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint Conference on
Natural Language Processing ({EMNLP-IJCNLP} 2019)}, pages 3982--3992, 2019.
\bibitem{reiss2023reliability}
Michael~V. Reiss.
\newblock Testing the reliability of {ChatGPT} for text annotation and
classification: {A} cautionary remark, 2023.
\bibitem{tan2024survey}
Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita
Bhattacharjee, Mansooreh Karami, Jundong Li, Lu~Cheng, and Huan Liu.
\newblock Large language models for data annotation and synthesis: {A} survey.
\newblock In {\em Proceedings of the 2024 Conference on Empirical Methods in
Natural Language Processing ({EMNLP} 2024)}, pages 930--957, Miami, Florida,
USA, 2024.
\bibitem{tenoever2022aid}
Niels ten Oever, Corinne Cath, Mirja K{\"u}hlewind, and Colin~S. Perkins.
\newblock Report from the {IAB} workshop on analyzing {IETF} data ({AID}) 2021.
\newblock RFC 9307, Internet Architecture Board, September 2022.
\newblock Informational.
\bibitem{wang2023nlgeval}
Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li,
Jinan Xu, Jianfeng Qu, and Jie Zhou.
\newblock Is {ChatGPT} a good {NLG} evaluator? {A} preliminary study.
\newblock In {\em Proceedings of the 4th Workshop on New Frontiers in
Summarization ({NewSumm@EMNLP} 2023)}, 2023.
\newblock arXiv:2303.04048.
\bibitem{yang2025agentprotocols}
Yingxuan Yang, Huacan Chai, Yuanyi Song, Siyuan Qi, Muning Wen, Ning Li, Junwei
Liao, Haoyi Hu, Jianghao Lin, Gaowei Chang, Weiwen Liu, Ying Wen, Yong Yu,
and Weinan Zhang.
\newblock A survey of {AI} agent protocols, 2025.
\bibitem{zhang2025affiliations}
Yangjun Zhang, Stephen McQuistin, Mladen Karan, Hugo~Enrique Ramirez-Centeno,
Colin Perkins, Gareth Tyson, and Ignacio Castro.
\newblock Two decades of {IETF} affiliations: Evolution and impact.
\newblock In {\em Proceedings of the 2025 Applied Networking Research Workshop
({ANRW} '25)}, 2025.
\end{thebibliography}

46
workspace/drafts/landscape-survey/main.blg Normal file
View File

@@ -0,0 +1,46 @@
This is BibTeX, Version 0.99d (TeX Live 2023)
Capacity: max_strings=200000, hash_size=200000, hash_prime=170003
The top-level auxiliary file: main.aux
The style file: plain.bst
Database file #1: references.bib
You've used 13 entries,
2118 wiz_defined-function locations,
568 strings with 6992 characters,
and the built_in function-call counts, 5447 in all, are:
= -- 487
> -- 376
< -- 4
+ -- 152
- -- 136
* -- 413
:= -- 909
add.period$ -- 37
call.type$ -- 13
change.case$ -- 107
chr.to.int$ -- 0
cite$ -- 13
duplicate$ -- 169
empty$ -- 367
format.name$ -- 136
if$ -- 1109
int.to.chr$ -- 0
int.to.str$ -- 13
missing$ -- 8
newline$ -- 66
num.names$ -- 26
pop$ -- 135
preamble$ -- 1
purify$ -- 94
quote$ -- 0
skip$ -- 134
stack$ -- 0
substring$ -- 249
swap$ -- 46
text.length$ -- 4
text.prefix$ -- 0
top$ -- 0
type$ -- 52
warning$ -- 0
while$ -- 40
width$ -- 15
write$ -- 136

888
workspace/drafts/landscape-survey/main.log Normal file
View File

@@ -0,0 +1,888 @@
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex 2026.4.8) 23 MAY 2026 23:04
entering extended mode
restricted \write18 enabled.
%&-line parsing enabled.
**main.tex
(./main.tex
LaTeX2e <2022-11-01> patch level 1
L3 programming layer <2023-02-22>
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2022/07/02 v1.4n Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size11.clo
File: size11.clo 2022/07/02 v1.4n Standard LaTeX file (size option)
)
\c@part=\count185
\c@section=\count186
\c@subsection=\count187
\c@subsubsection=\count188
\c@paragraph=\count189
\c@subparagraph=\count190
\c@figure=\count191
\c@table=\count192
\abovecaptionskip=\skip48
\belowcaptionskip=\skip49
\bibindent=\dimen140
)
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty
Package: inputenc 2021/02/14 v1.3d Input encoding file
\inpenc@prehook=\toks16
\inpenc@posthook=\toks17
)
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty
Package: fontenc 2021/04/29 v2.0v Standard LaTeX package
)
(/usr/share/texlive/texmf-dist/tex/latex/lm/lmodern.sty
Package: lmodern 2015/05/01 v1.6.1 Latin Modern Fonts
LaTeX Font Info: Overwriting symbol font `operators' in version `normal'
(Font) OT1/cmr/m/n --> OT1/lmr/m/n on input line 22.
LaTeX Font Info: Overwriting symbol font `letters' in version `normal'
(Font) OML/cmm/m/it --> OML/lmm/m/it on input line 23.
LaTeX Font Info: Overwriting symbol font `symbols' in version `normal'
(Font) OMS/cmsy/m/n --> OMS/lmsy/m/n on input line 24.
LaTeX Font Info: Overwriting symbol font `largesymbols' in version `normal'
(Font) OMX/cmex/m/n --> OMX/lmex/m/n on input line 25.
LaTeX Font Info: Overwriting symbol font `operators' in version `bold'
(Font) OT1/cmr/bx/n --> OT1/lmr/bx/n on input line 26.
LaTeX Font Info: Overwriting symbol font `letters' in version `bold'
(Font) OML/cmm/b/it --> OML/lmm/b/it on input line 27.
LaTeX Font Info: Overwriting symbol font `symbols' in version `bold'
(Font) OMS/cmsy/b/n --> OMS/lmsy/b/n on input line 28.
LaTeX Font Info: Overwriting symbol font `largesymbols' in version `bold'
(Font) OMX/cmex/m/n --> OMX/lmex/m/n on input line 29.
LaTeX Font Info: Overwriting math alphabet `\mathbf' in version `normal'
(Font) OT1/cmr/bx/n --> OT1/lmr/bx/n on input line 31.
LaTeX Font Info: Overwriting math alphabet `\mathsf' in version `normal'
(Font) OT1/cmss/m/n --> OT1/lmss/m/n on input line 32.
LaTeX Font Info: Overwriting math alphabet `\mathit' in version `normal'
(Font) OT1/cmr/m/it --> OT1/lmr/m/it on input line 33.
LaTeX Font Info: Overwriting math alphabet `\mathtt' in version `normal'
(Font) OT1/cmtt/m/n --> OT1/lmtt/m/n on input line 34.
LaTeX Font Info: Overwriting math alphabet `\mathbf' in version `bold'
(Font) OT1/cmr/bx/n --> OT1/lmr/bx/n on input line 35.
LaTeX Font Info: Overwriting math alphabet `\mathsf' in version `bold'
(Font) OT1/cmss/bx/n --> OT1/lmss/bx/n on input line 36.
LaTeX Font Info: Overwriting math alphabet `\mathit' in version `bold'
(Font) OT1/cmr/bx/it --> OT1/lmr/bx/it on input line 37.
LaTeX Font Info: Overwriting math alphabet `\mathtt' in version `bold'
(Font) OT1/cmtt/m/n --> OT1/lmtt/m/n on input line 38.
)
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
Package: geometry 2020/01/02 v5.9 Page Geometry
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty
Package: keyval 2022/05/29 v1.15 key=value parser (DPC)
\KV@toks@=\toks18
)
(/usr/share/texlive/texmf-dist/tex/generic/iftex/ifvtex.sty
Package: ifvtex 2019/10/25 v1.7 ifvtex legacy package. Use iftex instead.
(/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty
Package: iftex 2022/02/03 v1.0f TeX engine tests
))
\Gm@cnth=\count193
\Gm@cntv=\count194
\c@Gm@tempcnt=\count195
\Gm@bindingoffset=\dimen141
\Gm@wd@mp=\dimen142
\Gm@odd@mp=\dimen143
\Gm@even@mp=\dimen144
\Gm@layoutwidth=\dimen145
\Gm@layoutheight=\dimen146
\Gm@layouthoffset=\dimen147
\Gm@layoutvoffset=\dimen148
\Gm@dimlist=\toks19
)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
Package: graphicx 2021/09/16 v1.2d Enhanced LaTeX Graphics (DPC,SPQR)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
Package: graphics 2022/03/10 v1.4e Standard LaTeX Graphics (DPC,SPQR)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty
Package: trig 2021/08/11 v1.11 sin cos tan (DPC)
)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg
File: graphics.cfg 2016/06/04 v1.11 sample graphics configuration
)
Package graphics Info: Driver file: pdftex.def on input line 107.
(/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def
File: pdftex.def 2022/09/22 v1.2b Graphics/color driver for pdftex
))
\Gin@req@height=\dimen149
\Gin@req@width=\dimen150
)
(/usr/share/texlive/texmf-dist/tex/latex/booktabs/booktabs.sty
Package: booktabs 2020/01/12 v1.61803398 Publication quality tables
\heavyrulewidth=\dimen151
\lightrulewidth=\dimen152
\cmidrulewidth=\dimen153
\belowrulesep=\dimen154
\belowbottomsep=\dimen155
\aboverulesep=\dimen156
\abovetopsep=\dimen157
\cmidrulesep=\dimen158
\cmidrulekern=\dimen159
\defaultaddspace=\dimen160
\@cmidla=\count196
\@cmidlb=\count197
\@aboverulesep=\dimen161
\@belowrulesep=\dimen162
\@thisruleclass=\count198
\@lastruleclass=\count199
\@thisrulewidth=\dimen163
)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty
Package: hyperref 2023-02-07 v7.00v Hypertext links for LaTeX
(/usr/share/texlive/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty
Package: ltxcmds 2020-05-10 v1.25 LaTeX kernel commands for general use (HO)
)
(/usr/share/texlive/texmf-dist/tex/generic/pdftexcmds/pdftexcmds.sty
Package: pdftexcmds 2020-06-27 v0.33 Utility functions of pdfTeX for LuaTeX (HO
)
(/usr/share/texlive/texmf-dist/tex/generic/infwarerr/infwarerr.sty
Package: infwarerr 2019/12/03 v1.5 Providing info/warning/error messages (HO)
)
Package pdftexcmds Info: \pdf@primitive is available.
Package pdftexcmds Info: \pdf@ifprimitive is available.
Package pdftexcmds Info: \pdfdraftmode found.
)
(/usr/share/texlive/texmf-dist/tex/latex/kvsetkeys/kvsetkeys.sty
Package: kvsetkeys 2022-10-05 v1.19 Key value parser (HO)
)
(/usr/share/texlive/texmf-dist/tex/generic/kvdefinekeys/kvdefinekeys.sty
Package: kvdefinekeys 2019-12-19 v1.6 Define keys (HO)
)
(/usr/share/texlive/texmf-dist/tex/generic/pdfescape/pdfescape.sty
Package: pdfescape 2019/12/09 v1.15 Implements pdfTeX's escape features (HO)
)
(/usr/share/texlive/texmf-dist/tex/latex/hycolor/hycolor.sty
Package: hycolor 2020-01-27 v1.10 Color options for hyperref/bookmark (HO)
)
(/usr/share/texlive/texmf-dist/tex/latex/letltxmacro/letltxmacro.sty
Package: letltxmacro 2019/12/03 v1.6 Let assignment for LaTeX macros (HO)
)
(/usr/share/texlive/texmf-dist/tex/latex/auxhook/auxhook.sty
Package: auxhook 2019-12-17 v1.6 Hooks for auxiliary files (HO)
)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty
Package: nameref 2022-05-17 v2.50 Cross-referencing by name of section
(/usr/share/texlive/texmf-dist/tex/latex/refcount/refcount.sty
Package: refcount 2019/12/15 v3.6 Data extraction from label references (HO)
)
(/usr/share/texlive/texmf-dist/tex/generic/gettitlestring/gettitlestring.sty
Package: gettitlestring 2019/12/15 v1.6 Cleanup title references (HO)
(/usr/share/texlive/texmf-dist/tex/latex/kvoptions/kvoptions.sty
Package: kvoptions 2022-06-15 v3.15 Key value format for package options (HO)
))
\c@section@level=\count266
)
\@linkdim=\dimen164
\Hy@linkcounter=\count267
\Hy@pagecounter=\count268
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def
File: pd1enc.def 2023-02-07 v7.00v Hyperref: PDFDocEncoding definition (HO)
Now handling font encoding PD1 ...
... no UTF-8 mapping file for font encoding PD1
)
(/usr/share/texlive/texmf-dist/tex/generic/intcalc/intcalc.sty
Package: intcalc 2019/12/15 v1.3 Expandable calculations with integers (HO)
)
(/usr/share/texlive/texmf-dist/tex/generic/etexcmds/etexcmds.sty
Package: etexcmds 2019/12/15 v1.7 Avoid name clashes with e-TeX commands (HO)
)
\Hy@SavedSpaceFactor=\count269
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def
File: puenc.def 2023-02-07 v7.00v Hyperref: PDF Unicode definition (HO)
Now handling font encoding PU ...
... no UTF-8 mapping file for font encoding PU
)
Package hyperref Info: Hyper figures OFF on input line 4177.
Package hyperref Info: Link nesting OFF on input line 4182.
Package hyperref Info: Hyper index ON on input line 4185.
Package hyperref Info: Plain pages OFF on input line 4192.
Package hyperref Info: Backreferencing OFF on input line 4197.
Package hyperref Info: Implicit mode ON; LaTeX internals redefined.
Package hyperref Info: Bookmarks ON on input line 4425.
\c@Hy@tempcnt=\count270
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty
\Urlmuskip=\muskip16
Package: url 2013/09/16 ver 3.4 Verb mode for urls, etc.
)
LaTeX Info: Redefining \url on input line 4763.
\XeTeXLinkMargin=\dimen165
(/usr/share/texlive/texmf-dist/tex/generic/bitset/bitset.sty
Package: bitset 2019/12/09 v1.3 Handle bit-vector datatype (HO)
(/usr/share/texlive/texmf-dist/tex/generic/bigintcalc/bigintcalc.sty
Package: bigintcalc 2019/12/15 v1.5 Expandable calculations on big integers (HO
)
))
\Fld@menulength=\count271
\Field@Width=\dimen166
\Fld@charsize=\dimen167
Package hyperref Info: Hyper figures OFF on input line 6042.
Package hyperref Info: Link nesting OFF on input line 6047.
Package hyperref Info: Hyper index ON on input line 6050.
Package hyperref Info: backreferencing OFF on input line 6057.
Package hyperref Info: Link coloring OFF on input line 6062.
Package hyperref Info: Link coloring with OCG OFF on input line 6067.
Package hyperref Info: PDF/A mode OFF on input line 6072.
(/usr/share/texlive/texmf-dist/tex/latex/base/atbegshi-ltx.sty
Package: atbegshi-ltx 2021/01/10 v1.0c Emulation of the original atbegshi
package with kernel methods
)
\Hy@abspage=\count272
\c@Item=\count273
\c@Hfootnote=\count274
)
Package hyperref Info: Driver (autodetected): hpdftex.
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def
File: hpdftex.def 2023-02-07 v7.00v Hyperref driver for pdfTeX
(/usr/share/texlive/texmf-dist/tex/latex/base/atveryend-ltx.sty
Package: atveryend-ltx 2020/08/19 v1.0a Emulation of the original atveryend pac
kage
with kernel methods
)
\Fld@listcount=\count275
\c@bookmark@seq@number=\count276
(/usr/share/texlive/texmf-dist/tex/latex/rerunfilecheck/rerunfilecheck.sty
Package: rerunfilecheck 2022-07-10 v1.10 Rerun checks for auxiliary files (HO)
(/usr/share/texlive/texmf-dist/tex/generic/uniquecounter/uniquecounter.sty
Package: uniquecounter 2019/12/15 v1.4 Provide unlimited unique counter (HO)
)
Package uniquecounter Info: New unique counter `rerunfilecheck' on input line 2
85.
)
\Hy@SectionHShift=\skip50
)
(/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty
Package: xcolor 2022/06/12 v2.14 LaTeX color extensions (UK)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg
File: color.cfg 2016/01/02 v1.6 sample color configuration
)
Package xcolor Info: Driver file: pdftex.def on input line 227.
(/usr/share/texlive/texmf-dist/tex/latex/graphics/mathcolor.ltx)
Package xcolor Info: Model `cmy' substituted by `cmy0' on input line 1353.
Package xcolor Info: Model `hsb' substituted by `rgb' on input line 1357.
Package xcolor Info: Model `RGB' extended on input line 1369.
Package xcolor Info: Model `HTML' substituted by `rgb' on input line 1371.
Package xcolor Info: Model `Hsb' substituted by `hsb' on input line 1372.
Package xcolor Info: Model `tHsb' substituted by `hsb' on input line 1373.
Package xcolor Info: Model `HSB' substituted by `hsb' on input line 1374.
Package xcolor Info: Model `Gray' substituted by `gray' on input line 1375.
Package xcolor Info: Model `wave' substituted by `hsb' on input line 1376.
)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
Package: amsmath 2022/04/08 v2.17n AMS math features
\@mathmargin=\skip51
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
Package: amstext 2021/08/26 v2.01 AMS text
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty
File: amsgen.sty 1999/11/30 v2.0 generic functions
\@emptytoks=\toks20
\ex@=\dimen168
))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty
Package: amsbsy 1999/11/29 v1.2d Bold Symbols
\pmbraise@=\dimen169
)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty
Package: amsopn 2022/04/08 v2.04 operator names
)
\inf@bad=\count277
LaTeX Info: Redefining \frac on input line 234.
\uproot@=\count278
\leftroot@=\count279
LaTeX Info: Redefining \overline on input line 399.
LaTeX Info: Redefining \colon on input line 410.
\classnum@=\count280
\DOTSCASE@=\count281
LaTeX Info: Redefining \ldots on input line 496.
LaTeX Info: Redefining \dots on input line 499.
LaTeX Info: Redefining \cdots on input line 620.
\Mathstrutbox@=\box51
\strutbox@=\box52
LaTeX Info: Redefining \big on input line 722.
LaTeX Info: Redefining \Big on input line 723.
LaTeX Info: Redefining \bigg on input line 724.
LaTeX Info: Redefining \Bigg on input line 725.
\big@size=\dimen170
LaTeX Font Info: Redeclaring font encoding OML on input line 743.
LaTeX Font Info: Redeclaring font encoding OMS on input line 744.
\macc@depth=\count282
LaTeX Info: Redefining \bmod on input line 905.
LaTeX Info: Redefining \pmod on input line 910.
LaTeX Info: Redefining \smash on input line 940.
LaTeX Info: Redefining \relbar on input line 970.
LaTeX Info: Redefining \Relbar on input line 971.
\c@MaxMatrixCols=\count283
\dotsspace@=\muskip17
\c@parentequation=\count284
\dspbrk@lvl=\count285
\tag@help=\toks21
\row@=\count286
\column@=\count287
\maxfields@=\count288
\andhelp@=\toks22
\eqnshift@=\dimen171
\alignsep@=\dimen172
\tagshift@=\dimen173
\tagwidth@=\dimen174
\totwidth@=\dimen175
\lineht@=\dimen176
\@envbody=\toks23
\multlinegap=\skip52
\multlinetaggap=\skip53
\mathdisplay@stack=\toks24
LaTeX Info: Redefining \[ on input line 2953.
LaTeX Info: Redefining \] on input line 2954.
)
(/usr/share/texlive/texmf-dist/tex/latex/float/float.sty
Package: float 2001/11/08 v1.3d Float enhancements (AL)
\c@float@type=\count289
\float@exts=\toks25
\float@box=\box53
\@float@everytoks=\toks26
\@floatcapt=\box54
)
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption.sty
Package: caption 2023/03/12 v3.6j Customizing captions (AR)
(/usr/share/texlive/texmf-dist/tex/latex/caption/caption3.sty
Package: caption3 2023/03/12 v2.4 caption3 kernel (AR)
\caption@tempdima=\dimen177
\captionmargin=\dimen178
\caption@leftmargin=\dimen179
\caption@rightmargin=\dimen180
\caption@width=\dimen181
\caption@indent=\dimen182
\caption@parindent=\dimen183
\caption@hangindent=\dimen184
Package caption Info: Standard document class detected.
)
\c@caption@flags=\count290
\c@continuedfloat=\count291
Package caption Info: float package is loaded.
Package caption Info: hyperref package is loaded.
)
(/usr/share/texlive/texmf-dist/tex/latex/tools/tabularx.sty
Package: tabularx 2020/01/15 v2.11c `tabularx' package (DPC)
(/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty
Package: array 2022/09/04 v2.5g Tabular extension package (FMi)
\col@sep=\dimen185
\ar@mcellbox=\box55
\extrarowheight=\dimen186
\NC@list=\toks27
\extratabsurround=\skip54
\backup@length=\skip55
\ar@cellbox=\box56
)
\TX@col@width=\dimen187
\TX@old@table=\dimen188
\TX@old@col=\dimen189
\TX@target=\dimen190
\TX@delta=\dimen191
\TX@cols=\count292
\TX@ftn=\toks28
)
(/usr/share/texlive/texmf-dist/tex/latex/enumitem/enumitem.sty
Package: enumitem 2019/06/20 v3.9 Customized lists
\labelindent=\skip56
\enit@outerparindent=\dimen192
\enit@toks=\toks29
\enit@inbox=\box57
\enit@count@id=\count293
\enitdp@description=\count294
)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.tex
\pgfutil@everybye=\toks30
\pgfutil@tempdima=\dimen193
\pgfutil@tempdimb=\dimen194
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def
\pgfutil@abb=\box58
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/pgf.revision.tex)
Package: pgfrcs 2023-01-15 v3.1.10 (3.1.10)
))
Package: pgf 2023-01-15 v3.1.10 (3.1.10)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex
Package: pgfsys 2023-01-15 v3.1.10 (3.1.10)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex
\pgfkeys@pathtoks=\toks31
\pgfkeys@temptoks=\toks32
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeyslibraryfiltered
.code.tex
\pgfkeys@tmptoks=\toks33
))
\pgf@x=\dimen195
\pgf@y=\dimen196
\pgf@xa=\dimen197
\pgf@ya=\dimen198
\pgf@xb=\dimen199
\pgf@yb=\dimen256
\pgf@xc=\dimen257
\pgf@yc=\dimen258
\pgf@xd=\dimen259
\pgf@yd=\dimen260
\w@pgf@writea=\write3
\r@pgf@reada=\read2
\c@pgf@counta=\count295
\c@pgf@countb=\count296
\c@pgf@countc=\count297
\c@pgf@countd=\count298
\t@pgf@toka=\toks34
\t@pgf@tokb=\toks35
\t@pgf@tokc=\toks36
\pgf@sys@id@count=\count299
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg
File: pgf.cfg 2023-01-15 v3.1.10 (3.1.10)
)
Driver file for pgf: pgfsys-pdftex.def
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.def
File: pgfsys-pdftex.def 2023-01-15 v3.1.10 (3.1.10)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-pdf.de
f
File: pgfsys-common-pdf.def 2023-01-15 v3.1.10 (3.1.10)
)))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath.code.
tex
File: pgfsyssoftpath.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgfsyssoftpath@smallbuffer@items=\count300
\pgfsyssoftpath@bigbuffer@items=\count301
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol.code.
tex
File: pgfsysprotocol.code.tex 2023-01-15 v3.1.10 (3.1.10)
))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex
Package: pgfcore 2023-01-15 v3.1.10 (3.1.10)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex
\pgfmath@dimen=\dimen261
\pgfmath@count=\count302
\pgfmath@box=\box59
\pgfmath@toks=\toks37
\pgfmath@stack@operand=\toks38
\pgfmath@stack@operation=\toks39
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigonomet
ric.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.random.cod
e.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.comparison
.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round.code
.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc.code.
tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integerari
thmetics.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex
\c@pgfmathroundto@lastzeros=\count303
))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfint.code.tex)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.code.te
x
File: pgfcorepoints.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgf@picminx=\dimen262
\pgf@picmaxx=\dimen263
\pgf@picminy=\dimen264
\pgf@picmaxy=\dimen265
\pgf@pathminx=\dimen266
\pgf@pathmaxx=\dimen267
\pgf@pathminy=\dimen268
\pgf@pathmaxy=\dimen269
\pgf@xx=\dimen270
\pgf@xy=\dimen271
\pgf@yx=\dimen272
\pgf@yy=\dimen273
\pgf@zx=\dimen274
\pgf@zy=\dimen275
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconstruct.
code.tex
File: pgfcorepathconstruct.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgf@path@lastx=\dimen276
\pgf@path@lasty=\dimen277
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage.code
.tex
File: pgfcorepathusage.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgf@shorten@end@additional=\dimen278
\pgf@shorten@start@additional=\dimen279
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.code.te
x
File: pgfcorescopes.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgfpic=\box60
\pgf@hbox=\box61
\pgf@layerbox@main=\box62
\pgf@picture@serial@count=\count304
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicstate.c
ode.tex
File: pgfcoregraphicstate.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgflinewidth=\dimen280
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransformation
s.code.tex
File: pgfcoretransformations.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgf@pt@x=\dimen281
\pgf@pt@y=\dimen282
\pgf@pt@temp=\dimen283
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.code.tex
File: pgfcorequick.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.code.t
ex
File: pgfcoreobjects.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathprocessing
.code.tex
File: pgfcorepathprocessing.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.code.te
x
File: pgfcorearrows.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgfarrowsep=\dimen284
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.code.tex
File: pgfcoreshade.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgf@max=\dimen285
\pgf@sys@shading@range@num=\count305
\pgf@shadingcount=\count306
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.code.tex
File: pgfcoreimage.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal.code.
tex
File: pgfcoreexternal.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgfexternal@startupbox=\box63
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.code.te
x
File: pgfcorelayers.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransparency.c
ode.tex
File: pgfcoretransparency.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns.code.
tex
File: pgfcorepatterns.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorerdf.code.tex
File: pgfcorerdf.code.tex 2023-01-15 v3.1.10 (3.1.10)
)))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.code.tex
File: pgfmoduleshapes.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgfnodeparttextbox=\box64
) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code.tex
File: pgfmoduleplot.code.tex 2023-01-15 v3.1.10 (3.1.10)
)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-0-65
.sty
Package: pgfcomp-version-0-65 2023-01-15 v3.1.10 (3.1.10)
\pgf@nodesepstart=\dimen286
\pgf@nodesepend=\dimen287
)
(/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-1-18
.sty
Package: pgfcomp-version-1-18 2023-01-15 v3.1.10 (3.1.10)
)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgffor.sty
(/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex))
(/usr/share/texlive/texmf-dist/tex/latex/pgf/math/pgfmath.sty
(/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex
Package: pgffor 2023-01-15 v3.1.10 (3.1.10)
\pgffor@iter=\dimen288
\pgffor@skip=\dimen289
\pgffor@stack=\toks40
\pgffor@toks=\toks41
))
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex
Package: tikz 2023-01-15 v3.1.10 (3.1.10)
(/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers
.code.tex
File: pgflibraryplothandlers.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgf@plot@mark@count=\count307
\pgfplotmarksize=\dimen290
)
\tikz@lastx=\dimen291
\tikz@lasty=\dimen292
\tikz@lastxsaved=\dimen293
\tikz@lastysaved=\dimen294
\tikz@lastmovetox=\dimen295
\tikz@lastmovetoy=\dimen296
\tikzleveldistance=\dimen297
\tikzsiblingdistance=\dimen298
\tikz@figbox=\box65
\tikz@figbox@bg=\box66
\tikz@tempbox=\box67
\tikz@tempbox@bg=\box68
\tikztreelevel=\count308
\tikznumberofchildren=\count309
\tikznumberofcurrentchild=\count310
\tikz@fig@count=\count311
(/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.code.tex
File: pgfmodulematrix.code.tex 2023-01-15 v3.1.10 (3.1.10)
\pgfmatrixcurrentrow=\count312
\pgfmatrixcurrentcolumn=\count313
\pgf@matrix@numberofcolumns=\count314
)
\tikz@expandcount=\count315
(/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik
zlibrarytopaths.code.tex
File: tikzlibrarytopaths.code.tex 2023-01-15 v3.1.10 (3.1.10)
)))
Package hyperref Info: Option `colorlinks' set `true' on input line 24.
LaTeX Font Info: Trying to load font information for T1+lmr on input line 38
.
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmr.fd
File: t1lmr.fd 2015/05/01 v1.6.1 Font defs for Latin Modern
)
(/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-pdftex.def
File: l3backend-pdftex.def 2023-01-16 L3 backend support: PDF output (pdfTeX)
\l__color_backend_stack_int=\count316
\l__pdf_internal_box=\box69
)
(./main.aux)
\openout1 = `main.aux'.
LaTeX Font Info: Checking defaults for OML/cmm/m/it on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for OMS/cmsy/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for OT1/cmr/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for T1/cmr/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for TS1/cmr/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for OMX/cmex/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for U/cmr/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for PD1/pdf/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
LaTeX Font Info: Checking defaults for PU/pdf/m/n on input line 38.
LaTeX Font Info: ... okay on input line 38.
*geometry* driver: auto-detecting
*geometry* detected driver: pdftex
*geometry* verbose mode - [ preamble ] result:
* driver: pdftex
* paper: a4paper
* layout: <same size as paper>
* layoutoffset:(h,v)=(0.0pt,0.0pt)
* modes:
* h-part:(L,W,R)=(72.26999pt, 452.9679pt, 72.26999pt)
* v-part:(T,H,B)=(72.26999pt, 700.50687pt, 72.26999pt)
* \paperwidth=597.50787pt
* \paperheight=845.04684pt
* \textwidth=452.9679pt
* \textheight=700.50687pt
* \oddsidemargin=0.0pt
* \evensidemargin=0.0pt
* \topmargin=-37.0pt
* \headheight=12.0pt
* \headsep=25.0pt
* \topskip=11.0pt
* \footskip=30.0pt
* \marginparwidth=50.0pt
* \marginparsep=10.0pt
* \columnsep=10.0pt
* \skip\footins=10.0pt plus 4.0pt minus 2.0pt
* \hoffset=0.0pt
* \voffset=0.0pt
* \mag=1000
* \@twocolumnfalse
* \@twosidefalse
* \@mparswitchfalse
* \@reversemarginfalse
* (1in=72.27pt=25.4mm, 1cm=28.453pt)
(/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
\scratchcounter=\count317
\scratchdimen=\dimen299
\scratchbox=\box70
\nofMPsegments=\count318
\nofMParguments=\count319
\everyMPshowfont=\toks42
\MPscratchCnt=\count320
\MPscratchDim=\dimen300
\MPnumerator=\count321
\makeMPintoPDFobject=\count322
\everyMPtoPDFconversion=\toks43
) (/usr/share/texlive/texmf-dist/tex/latex/epstopdf-pkg/epstopdf-base.sty
Package: epstopdf-base 2020-01-24 v2.11 Base part for package epstopdf
Package epstopdf-base Info: Redefining graphics rule for `.eps' on input line 4
85.
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg
File: epstopdf-sys.cfg 2010/07/13 v1.3 Configuration of (r)epstopdf for TeX Liv
e
))
Package hyperref Info: Link coloring ON on input line 38.
(./main.out) (./main.out)
\@outlinefile=\write4
\openout4 = `main.out'.
Package caption Info: Begin \AtBeginDocument code.
Package caption Info: End \AtBeginDocument code.
LaTeX Font Info: Trying to load font information for OT1+lmr on input line 4
0.
(/usr/share/texlive/texmf-dist/tex/latex/lm/ot1lmr.fd
File: ot1lmr.fd 2015/05/01 v1.6.1 Font defs for Latin Modern
)
LaTeX Font Info: Trying to load font information for OML+lmm on input line 4
0.
(/usr/share/texlive/texmf-dist/tex/latex/lm/omllmm.fd
File: omllmm.fd 2015/05/01 v1.6.1 Font defs for Latin Modern
)
LaTeX Font Info: Trying to load font information for OMS+lmsy on input line
40.
(/usr/share/texlive/texmf-dist/tex/latex/lm/omslmsy.fd
File: omslmsy.fd 2015/05/01 v1.6.1 Font defs for Latin Modern
)
LaTeX Font Info: Trying to load font information for OMX+lmex on input line
40.
(/usr/share/texlive/texmf-dist/tex/latex/lm/omxlmex.fd
File: omxlmex.fd 2015/05/01 v1.6.1 Font defs for Latin Modern
)
LaTeX Font Info: External font `lmex10' loaded for size
(Font) <12> on input line 40.
LaTeX Font Info: External font `lmex10' loaded for size
(Font) <8> on input line 40.
LaTeX Font Info: External font `lmex10' loaded for size
(Font) <6> on input line 40.
LaTeX Font Info: Trying to load font information for T1+lmtt on input line 4
0.
(/usr/share/texlive/texmf-dist/tex/latex/lm/t1lmtt.fd
File: t1lmtt.fd 2015/05/01 v1.6.1 Font defs for Latin Modern
)
(./sections/00-abstract.tex
LaTeX Font Info: External font `lmex10' loaded for size
(Font) <10> on input line 5.
LaTeX Font Info: External font `lmex10' loaded for size
(Font) <7> on input line 5.
LaTeX Font Info: External font `lmex10' loaded for size
(Font) <5> on input line 5.
) (./sections/01-intro.tex
LaTeX Font Info: External font `lmex10' loaded for size
(Font) <10.95> on input line 6.
[1
{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}{/usr/share/texlive/texmf-di
st/fonts/enc/dvips/lm/lm-ec.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/
lm/lm-rm.enc}{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathsy.enc}{/
usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathit.enc}]
LaTeX Font Info: Trying to load font information for TS1+lmr on input line 3
2.
(/usr/share/texlive/texmf-dist/tex/latex/lm/ts1lmr.fd
File: ts1lmr.fd 2015/05/01 v1.6.1 Font defs for Latin Modern
))
(./sections/02-related.tex [2{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/
lm-ts1.enc}]) (./sections/03-method.tex [3]) (./sections/04-landscape.tex
Overfull \hbox (34.79286pt too wide) in paragraph at lines 53--1
\T1/lmr/m/n/10.95 scripts (\T1/lmtt/m/n/10.95 scripts/survey-phase0.py\T1/lmr/m
/n/10.95 , \T1/lmtt/m/n/10.95 scripts/rerate-intercoder.py\T1/lmr/m/n/10.95 , \
T1/lmtt/m/n/10.95 scripts/survey-kappa.py\T1/lmr/m/n/10.95 ).
[]
LaTeX Warning: `h' float specifier changed to `ht'.
LaTeX Warning: `h' float specifier changed to `ht'.
[4{/usr/share/texlive/texmf-dist/fonts/enc/dvips/lm/lm-mathex.enc}])
(./sections/05-reliability.tex [5]) (./sections/06-discussion.tex [6]
Overfull \hbox (1.63pt too wide) in paragraph at lines 6--19
\T1/lmr/m/n/10.95 Two struc-tural fea-tures of the cor-pus re-in-force one an-o
ther. First, the area is \T1/lmr/m/it/10.95 pre-standardization\T1/lmr/m/n/10.9
5 :
[]
[7]) (./sections/07-conclusion.tex) (./main.bbl [8]) [9] (./main.aux)
Package rerunfilecheck Info: File `main.out' has not changed.
(rerunfilecheck) Checksum: C94D38AE5E8EEA58F269BB007EA276D5;2585.
)
Here is how much of TeX's memory you used:
22331 strings out of 477985
418461 string characters out of 5839381
1867388 words of memory out of 6000000
42065 multiletter control sequences out of 15000+600000
581269 words of font info for 72 fonts, out of 8000000 for 9000
14 hyphenation exceptions out of 8191
84i,9n,80p,385b,639s stack positions out of 10000i,1000n,20000p,200000b,200000s
</usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmbx10.pfb></usr/share/t
exlive/texmf-dist/fonts/type1/public/lm/lmbx12.pfb></usr/share/texlive/texmf-di
st/fonts/type1/public/lm/lmex10.pfb></usr/share/texlive/texmf-dist/fonts/type1/
public/lm/lmmi10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmmi7
.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmmi8.pfb></usr/share
/texlive/texmf-dist/fonts/type1/public/lm/lmr10.pfb></usr/share/texlive/texmf-d
ist/fonts/type1/public/lm/lmr12.pfb></usr/share/texlive/texmf-dist/fonts/type1/
public/lm/lmr8.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmri10.
pfb></usr/share/texlive/texmf-dist/fonts/type1/public/lm/lmsy10.pfb></usr/share
/texlive/texmf-dist/fonts/type1/public/lm/lmtt10.pfb></usr/share/texlive/texmf-
dist/fonts/type1/public/lm/lmtt12.pfb>
Output written on main.pdf (9 pages, 278200 bytes).
PDF statistics:
283 PDF objects out of 1000 (max. 8388607)
240 compressed objects within 3 object streams
54 named destinations out of 1000 (max. 500000)
149 words of extra memory for PDF output out of 10000 (max. 10000000)

17
workspace/drafts/landscape-survey/main.out Normal file
View File

@@ -0,0 +1,17 @@
\BOOKMARK [1][-]{section.1}{\376\377\000I\000n\000t\000r\000o\000d\000u\000c\000t\000i\000o\000n}{}% 1
\BOOKMARK [1][-]{section.2}{\376\377\000R\000e\000l\000a\000t\000e\000d\000\040\000W\000o\000r\000k}{}% 2
\BOOKMARK [1][-]{section.3}{\376\377\000C\000o\000r\000p\000u\000s\000\040\000a\000n\000d\000\040\000M\000e\000t\000h\000o\000d}{}% 3
\BOOKMARK [2][-]{subsection.3.1}{\376\377\000C\000o\000r\000p\000u\000s\000\040\000c\000o\000n\000s\000t\000r\000u\000c\000t\000i\000o\000n}{section.3}% 4
\BOOKMARK [2][-]{subsection.3.2}{\376\377\000T\000h\000e\000m\000a\000t\000i\000c\000\040\000c\000l\000a\000s\000s\000i\000f\000i\000c\000a\000t\000i\000o\000n}{section.3}% 5
\BOOKMARK [2][-]{subsection.3.3}{\376\377\000S\000e\000m\000a\000n\000t\000i\000c\000\040\000e\000m\000b\000e\000d\000d\000i\000n\000g\000s}{section.3}% 6
\BOOKMARK [2][-]{subsection.3.4}{\376\377\000R\000e\000p\000r\000o\000d\000u\000c\000i\000b\000i\000l\000i\000t\000y}{section.3}% 7
\BOOKMARK [1][-]{section.4}{\376\377\000T\000h\000e\000\040\000L\000a\000n\000d\000s\000c\000a\000p\000e}{}% 8
\BOOKMARK [2][-]{subsection.4.1}{\376\377\000T\000e\000m\000p\000o\000r\000a\000l\000\040\000d\000y\000n\000a\000m\000i\000c\000s}{section.4}% 9
\BOOKMARK [2][-]{subsection.4.2}{\376\377\000C\000a\000t\000e\000g\000o\000r\000y\000\040\000d\000i\000s\000t\000r\000i\000b\000u\000t\000i\000o\000n}{section.4}% 10
\BOOKMARK [2][-]{subsection.4.3}{\376\377\000A\000u\000t\000h\000o\000r\000s\000h\000i\000p\000\040\000a\000n\000d\000\040\000w\000o\000r\000k\000i\000n\000g\000-\000g\000r\000o\000u\000p\000\040\000s\000t\000r\000u\000c\000t\000u\000r\000e}{section.4}% 11
\BOOKMARK [2][-]{subsection.4.4}{\376\377\000S\000e\000m\000a\000n\000t\000i\000c\000\040\000r\000e\000d\000u\000n\000d\000a\000n\000c\000y}{section.4}% 12
\BOOKMARK [1][-]{section.5}{\376\377\000H\000o\000w\000\040\000R\000e\000l\000i\000a\000b\000l\000e\000\040\000A\000r\000e\000\040\000t\000h\000e\000\040\000L\000a\000b\000e\000l\000s\000?}{}% 13
\BOOKMARK [1][-]{section.6}{\376\377\000D\000i\000s\000c\000u\000s\000s\000i\000o\000n\000\040\000a\000n\000d\000\040\000L\000i\000m\000i\000t\000a\000t\000i\000o\000n\000s}{}% 14
\BOOKMARK [2][-]{subsection.6.1}{\376\377\000I\000m\000p\000l\000i\000c\000a\000t\000i\000o\000n\000s}{section.6}% 15
\BOOKMARK [2][-]{subsection.6.2}{\376\377\000L\000i\000m\000i\000t\000a\000t\000i\000o\000n\000s}{section.6}% 16
\BOOKMARK [1][-]{section.7}{\376\377\000C\000o\000n\000c\000l\000u\000s\000i\000o\000n\000\040\000a\000n\000d\000\040\000F\000u\000t\000u\000r\000e\000\040\000W\000o\000r\000k}{}% 17

BIN
workspace/drafts/landscape-survey/main.pdf Normal file
View File

Binary file not shown.

58
workspace/drafts/landscape-survey/main.tex Normal file
View File

@@ -0,0 +1,58 @@
\documentclass[11pt,a4paper]{article}
% ── Packages ──────────────────────────────────────────────────────────────
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage[margin=1in]{geometry}
\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{xcolor}
\usepackage{amsmath}
\usepackage{float}
\usepackage{caption}
\usepackage{tabularx}
\usepackage{enumitem}
\usepackage{tikz}
\hypersetup{
colorlinks=true,
linkcolor=blue!60!black,
citecolor=blue!60!black,
urlcolor=blue!60!black,
}
% ── Title ─────────────────────────────────────────────────────────────────
\title{%
\textbf{The AI-Agent Standardization Surge at the IETF:\\
A Verified Quantitative Survey of 524 Internet-Drafts (2024--2026)}%
}
\author{
Christian Nennemann\\
Independent Researcher\\
\texttt{ietf@nennemann.de}
}
\date{May 2026}
\begin{document}
\maketitle
\input{sections/00-abstract}
\noindent\textbf{Keywords:} IETF, Internet-Drafts, AI agents, standardization,
landscape analysis, LLM-assisted classification, inter-rater reliability,
text embeddings
\input{sections/01-intro}
\input{sections/02-related}
\input{sections/03-method}
\input{sections/04-landscape}
\input{sections/05-reliability}
\input{sections/06-discussion}
\input{sections/07-conclusion}
\bibliographystyle{plain}
\bibliography{references}
\end{document}

164
workspace/drafts/landscape-survey/references.bib Normal file
View File

@@ -0,0 +1,164 @@
% References for the IETF AI/agent landscape survey.
% Real, verifiable entries only. Agent-added entries must be checked.
% ── IETF / Standards quantitative analysis ────────────────────────────────
@inproceedings{mcquistin2021characterising,
author = {McQuistin, Stephen and Karan, Mladen and Khare, Prashant and
Perkins, Colin and Tyson, Gareth and Purver, Matthew and
Healey, Patrick and Iqbal, Waleed and Qadir, Junaid and
Castro, Ignacio},
title = {Characterising the {IETF} Through the Lens of {RFC} Deployment},
booktitle = {Proceedings of the 21st ACM Internet Measurement Conference
({IMC} '21)},
year = {2021},
pages = {137--149},
doi = {10.1145/3487552.3487821},
isbn = {9781450391290},
}
@inproceedings{zhang2025affiliations,
author = {Zhang, Yangjun and McQuistin, Stephen and Karan, Mladen and
Ramirez-Centeno, Hugo Enrique and Perkins, Colin and
Tyson, Gareth and Castro, Ignacio},
title = {Two Decades of {IETF} Affiliations: Evolution and Impact},
booktitle = {Proceedings of the 2025 Applied Networking Research Workshop
({ANRW} '25)},
year = {2025},
doi = {10.1145/3744200.3744757},
}
@techreport{tenoever2022aid,
author = {ten Oever, Niels and Cath, Corinne and K{\"u}hlewind, Mirja
and Perkins, Colin S.},
title = {Report from the {IAB} Workshop on Analyzing {IETF} Data
({AID}) 2021},
institution = {Internet Architecture Board},
type = {RFC},
number = {9307},
year = {2022},
month = sep,
issn = {2070-1721},
note = {Informational},
doi = {10.17487/RFC9307},
}
@misc{jimenez2024automating,
author = {Jim{\'e}nez, Jaime},
title = {Automating {IETF} Insights Generation with {AI}},
year = {2024},
eprint = {2410.13301},
archivePrefix = {arXiv},
primaryClass = {cs.NI},
}
% ── LLM-assisted annotation / classification ──────────────────────────────
@article{gilardi2023chatgpt,
author = {Gilardi, Fabrizio and Alizadeh, Meysam and Kubli, Ma{\"e}l},
title = {{ChatGPT} Outperforms Crowd-Workers for Text-Annotation Tasks},
journal = {Proceedings of the National Academy of Sciences},
year = {2023},
volume = {120},
number = {30},
pages = {e2305016120},
doi = {10.1073/pnas.2305016120},
}
@inproceedings{tan2024survey,
author = {Tan, Zhen and Li, Dawei and Wang, Song and Beigi, Alimohammad
and Jiang, Bohan and Bhattacharjee, Amrita and Karami,
Mansooreh and Li, Jundong and Cheng, Lu and Liu, Huan},
title = {Large Language Models for Data Annotation and Synthesis:
{A} Survey},
booktitle = {Proceedings of the 2024 Conference on Empirical Methods in
Natural Language Processing ({EMNLP} 2024)},
year = {2024},
pages = {930--957},
address = {Miami, Florida, USA},
doi = {10.18653/v1/2024.emnlp-main.54},
}
@misc{yang2025agentprotocols,
author = {Yang, Yingxuan and Chai, Huacan and Song, Yuanyi and
Qi, Siyuan and Wen, Muning and Li, Ning and Liao, Junwei
and Hu, Haoyi and Lin, Jianghao and Chang, Gaowei and
Liu, Weiwen and Wen, Ying and Yu, Yong and Zhang, Weinan},
title = {A Survey of {AI} Agent Protocols},
year = {2025},
eprint = {2504.16736},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
}
% ── Inter-rater reliability of LLM labels ─────────────────────────────────
@misc{reiss2023reliability,
author = {Reiss, Michael V.},
title = {Testing the Reliability of {ChatGPT} for Text Annotation
and Classification: {A} Cautionary Remark},
year = {2023},
eprint = {2304.11085},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
}
@inproceedings{wang2023nlgeval,
author = {Wang, Jiaan and Liang, Yunlong and Meng, Fandong and
Sun, Zengkui and Shi, Haoxiang and Li, Zhixu and Xu, Jinan
and Qu, Jianfeng and Zhou, Jie},
title = {Is {ChatGPT} a Good {NLG} Evaluator? {A} Preliminary Study},
booktitle = {Proceedings of the 4th Workshop on New Frontiers in
Summarization ({NewSumm@EMNLP} 2023)},
year = {2023},
note = {arXiv:2303.04048},
}
% ── Cohen's kappa / Landis-Koch (methodological) ──────────────────────────
@article{cohen1960kappa,
author = {Cohen, Jacob},
title = {A Coefficient of Agreement for Nominal Scales},
journal = {Educational and Psychological Measurement},
year = {1960},
volume = {20},
number = {1},
pages = {37--46},
doi = {10.1177/001316446002000104},
}
@article{landis1977kappa,
author = {Landis, J. Richard and Koch, Gary G.},
title = {The Measurement of Observer Agreement for Categorical Data},
journal = {Biometrics},
year = {1977},
volume = {33},
number = {1},
pages = {159--174},
doi = {10.2307/2529310},
}
% ── Text embeddings / semantic similarity ─────────────────────────────────
@inproceedings{reimers2019sbert,
author = {Reimers, Nils and Gurevych, Iryna},
title = {Sentence-{BERT}: Sentence Embeddings using Siamese {BERT}-Networks},
booktitle = {Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint
Conference on Natural Language Processing
({EMNLP-IJCNLP} 2019)},
year = {2019},
pages = {3982--3992},
doi = {10.18653/v1/D19-1410},
}
@misc{nussbaum2024nomic,
author = {Nussbaum, Zach and Morris, John X. and Duderstadt, Brandon
and Mulyar, Andriy},
title = {Nomic Embed: Training a Reproducible Long Context Text
Embedder},
year = {2024},
eprint = {2402.01613},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
}

24
workspace/drafts/landscape-survey/sections/00-abstract.tex Normal file
View File

@@ -0,0 +1,24 @@
\begin{abstract}
Between 2024 and 2026 the Internet Engineering Task Force (IETF) saw a sharp
rise in Internet-Drafts addressing AI agents and autonomous systems: monthly
submissions in our corpus grew from an average of 3.7 (June 2024--May 2025) to
38.8 (since June 2025), peaking at 106 in March 2026---roughly $35\times$ a
typical 2024 month. We present a quantitative survey of this emerging area
based on a curated corpus of 524 AI/agent-related IETF Internet-Drafts spanning
January 2024 to May 2026. We characterise the corpus along four axes: temporal
submission dynamics, thematic category distribution, authorship and
working-group structure, and semantic redundancy measured through text
embeddings. Two findings stand out: the area is overwhelmingly
\emph{pre-standardization}---87\% of drafts are individual submissions not
adopted by any working group---and it is semantically dense, with 32\% of
drafts having a near-duplicate (cosine $>0.9$) elsewhere in the corpus.
Because the thematic categories are produced by an LLM-assisted pipeline, we
explicitly quantify their reliability through a two-model re-rating experiment:
categorical assignment is substantially reproducible (Cohen's $\kappa = 0.65$),
whereas LLM ordinal \emph{quality} scores such as novelty and overlap are not
($\kappa_w = 0.13$--$0.21$). We therefore report the category landscape but
deliberately exclude quality scores from our findings, and we argue this
distinction is a general caution for the growing practice of LLM-assisted
corpus analysis. All data, queries, and rating artifacts are released for
reproduction.
\end{abstract}

61
workspace/drafts/landscape-survey/sections/01-intro.tex Normal file
View File

@@ -0,0 +1,61 @@
\section{Introduction}
\label{sec:intro}
Standardization activity around AI agents and autonomous systems has surged at
the Internet Engineering Task Force (IETF). In the corpus studied here, monthly
submissions of AI/agent-related Internet-Drafts grew from an average of $3.7$
per month over June 2024--May 2025 to $38.8$ per month since June 2025, peaking
at $106$ drafts in March 2026---roughly $35\times$ a typical 2024 month. A surge
of this size, concentrated in barely a year, makes the area difficult to
navigate: there is no settled taxonomy, the boundaries between proposals are
fluid, and it is unclear how much of the activity represents distinct technical
work versus overlapping attempts at the same problems.
A natural response is to characterise the space quantitatively, and
LLM-assisted corpus analyses are an increasingly common instrument for doing
so---using a large language model to classify documents, score them, and
summarise trends at a scale that manual reading cannot reach
\cite{tan2024survey,gilardi2023chatgpt}. Such studies, however, rarely report whether the labels they rely on
are reproducible. An LLM applied to the same documents under a different model,
or even a different run, may produce different categories and scores; without a
reliability check, it is impossible to tell which conclusions rest on stable
signal and which on rater noise.
To our knowledge no verified quantitative survey of the AI-agent
standardization space at the IETF exists. We address this gap with a
descriptive survey that is explicit about which of its measurements can be
trusted. This paper is deliberately neutral: it characterises the landscape
rather than advocating for any particular proposal or design.
\paragraph{Contributions.}
\begin{itemize}[leftmargin=1.4em,topsep=2pt,itemsep=1pt]
\item A curated corpus of \textbf{524} AI/agent-related IETF Internet-Drafts
spanning January 2024 to May 2026, constructed by keyword candidate
selection followed by explicit false-positive filtering
(Section~\ref{sec:method}).
\item A characterization of the corpus along four axes---temporal submission
dynamics, thematic category distribution, authorship and working-group
structure, and semantic redundancy measured through text embeddings
(Section~\ref{sec:landscape}).
\item An explicit two-model inter-rater reliability check that separates the
labels we can trust from those we cannot: categorical assignment is
substantially reproducible (Cohen's $\kappa \approx 0.65$), whereas the
LLM ordinal \emph{quality} scores are not ($\kappa_w = 0.13$--$0.21$ for
the least stable dimensions). We report the former and exclude the
latter (Section~\ref{sec:reliability}).
\item A full release of the data, queries, and rating artifacts used in the
analysis, so that every reported number can be recomputed.
\end{itemize}
\paragraph{Roadmap.}
Section~\ref{sec:method} describes corpus construction, the LLM-assisted
classification pipeline, and the embedding setup.
Section~\ref{sec:landscape} presents the landscape across the four axes:
temporal dynamics (Section~\ref{sec:temporal}), category distribution
(Section~\ref{sec:categories}), authorship and working-group structure
(Section~\ref{sec:authors}), and semantic redundancy
(Section~\ref{sec:redundancy}).
Section~\ref{sec:reliability} reports the two-model reliability experiment that
underwrites the category labels and disqualifies the quality scores.
Section~\ref{sec:discussion} discusses implications and limitations, and
Section~\ref{sec:conclusion} concludes with directions for future work.

76
workspace/drafts/landscape-survey/sections/02-related.tex Normal file
View File

@@ -0,0 +1,76 @@
\section{Related Work}
\label{sec:related}
\paragraph{Quantitative analysis of IETF activity.}
Several studies have examined the IETF corpus empirically. McQuistin
et al.\ characterised the IETF through the lens of RFC deployment,
using a dataset of 8,711 RFCs, 4,512 authors, and 2.4 million emails
to measure shifts in publication rate, author concentration, and
cross-document dependency over time~\cite{mcquistin2021characterising}.
Zhang et al.\ extended this line of work with a longitudinal analysis
of author affiliations across 2001--2023, covering 73,764 individuals
and finding that organisational diversity peaked and then
stagnated~\cite{zhang2025affiliations}. The IAB's own workshop on
Analyzing IETF Data (AID, 2021) surveyed open questions about
what drives drafts to become RFCs and how community diversity
evolves~\cite{tenoever2022aid}. More recently, Jim\'enez applied LLMs
directly to IETF working-group records to automate the generation of
summary reports~\cite{jimenez2024automating}. Our work complements this
body of evidence by providing the first quantitative survey focused
\emph{exclusively on the AI/agent topic area}: we document the growth
trajectory, authorship structure, and thematic distribution of a
cohort that did not exist in large numbers before 2025, and we do
so at the Internet-Draft stage rather than the post-publication RFC
stage.
\paragraph{LLM-assisted text classification and annotation.}
The use of LLMs as automated annotators is well established.
Gilardi et al.\ showed that ChatGPT outperforms crowd-workers on
political-science annotation tasks (stance, relevance, frames)
across 6,183 tweets, with zero-shot accuracy exceeding MTurk workers
by roughly 25 percentage points on average~\cite{gilardi2023chatgpt}.
Tan et al.\ survey the broader landscape of LLMs for data annotation
and synthesis, covering annotation generation, quality assessment, and
downstream utilisation~\cite{tan2024survey}. Yang et al.\ survey
emerging AI-agent communication protocols---the class of specification
documents our corpus comprises---providing context for the
standardisation subjects under analysis~\cite{yang2025agentprotocols}.
Our contribution in this dimension is not classification per se but
the application of the LLM-annotator pipeline to a \emph{technical
standards corpus} rather than short social-media texts, and the
explicit reliability gate described next.
\paragraph{Inter-rater reliability of LLM labels.}
The reproducibility of LLM judgments is contested.
Reiss identified prompt-sensitivity as a reliability hazard, showing
that minor wording changes or repeated identical inputs can shift
ChatGPT's classification output below the thresholds conventionally
required for scientific use~\cite{reiss2023reliability}. Wang et al.\
found that ChatGPT correlates well with human judgments on several
natural-language-generation evaluation dimensions but that the
strength of correlation varies substantially across
tasks~\cite{wang2023nlgeval}. These results motivate---but do not
themselves perform---inter-rater reliability analysis using
established statistics. We use Cohen's $\kappa$~\cite{cohen1960kappa}
with the Landis--Koch interpretation scale~\cite{landis1977kappa} to
report agreement both between two independent LLM re-rating runs and
between those runs and the production labels. To our knowledge, no
prior quantitative survey of an IETF (or comparable standards-body)
corpus has reported this reliability decomposition; the closest
precedent is Jim\'enez~\cite{jimenez2024automating}, who generates
LLM summaries of IETF records but does not measure label
reproducibility.
\paragraph{Embedding-based semantic analysis of document corpora.}
We use cosine similarity over dense text embeddings for the redundancy
analysis. Reimers and Gurevych established that sentence-level
embeddings computed with siamese BERT networks provide efficient and
accurate representations for semantic similarity
search~\cite{reimers2019sbert}; more recent work by Nussbaum et al.\
produces the \texttt{nomic-embed-text} model used in our
pipeline~\cite{nussbaum2024nomic}. The application of embedding-based
near-duplicate detection to a \emph{standards corpus} is, to our
knowledge, novel: prior IETF studies (e.g.,
\cite{mcquistin2021characterising}) rely on structural metadata---author
lists, citation graphs, working-group membership---rather than
semantic similarity over document content.

57
workspace/drafts/landscape-survey/sections/03-method.tex Normal file
View File

@@ -0,0 +1,57 @@
\section{Corpus and Method}
\label{sec:method}
\subsection{Corpus construction}
Internet-Drafts were collected from the IETF Datatracker. Candidate documents
were identified by keyword and topic matching for AI-agent and autonomous-system
terminology (e.g.\ ``agent'', ``autonomous'', ``LLM'', ``inference''), then
filtered to remove false positives---documents in which such terms occur in an
unrelated sense (``user agent'', ``autonomous system'' in the BGP sense). Of
597 candidate IETF documents, 73 ($12.2\%$) were flagged as false positives and
excluded, yielding a clean corpus of \textbf{524 IETF Internet-Drafts}.
We restrict this survey to IETF Internet-Drafts. The collection pipeline also
ingests documents from other standards bodies (ISO, ITU, ETSI, NIST, W3C), but
those exhibit heterogeneous and frequently missing publication dates and a
different document model; mixing them would compromise the temporal and
authorship analyses. All 524 IETF drafts carry ISO-8601 submission timestamps,
so the IETF-only scope is also the cleanest with respect to date quality. The
corpus snapshot used throughout is \texttt{data/drafts.db} as of 2026-05-23.
\subsection{Thematic classification}
Each draft is assigned to one or more of eleven thematic categories
(Table~\ref{tab:categories}) by an LLM-assisted rating pipeline. The classifier
is prompted with the draft's title, metadata, and abstract (truncated to the
first 2000 characters), and returns a JSON object containing the category
assignment and five ordinal $1$--$5$ quality dimensions (novelty, maturity,
overlap, momentum, relevance). We emphasise two properties of this pipeline
that bear directly on interpretation:
\begin{itemize}[leftmargin=1.4em,topsep=2pt,itemsep=1pt]
\item \textbf{Abstract-only.} Ratings are derived from the abstract, not the
full document text. This keeps the pipeline cheap and uniform but means
classifications reflect how a draft \emph{presents} itself, not a full
reading of its mechanics.
\item \textbf{Model-derived.} Categories and scores are produced by a large
language model (Claude Sonnet, \texttt{claude-sonnet-4-20250514}).
Section~\ref{sec:reliability} quantifies how reproducible these labels
are; the headline result is that we trust the categories but not the
ordinal scores.
\end{itemize}
\subsection{Semantic embeddings}
For the redundancy analysis (Section~\ref{sec:redundancy}) each draft is
embedded into a 768-dimensional vector using the \texttt{nomic-embed-text}
model run locally via Ollama. Embedding coverage is complete (524/524). We
compute cosine similarity over all $\binom{524}{2}=137{,}026$ document pairs.
\subsection{Reproducibility}
The pipeline, queries, and the two-model re-rating used for the reliability
analysis are released as scripts (\texttt{scripts/survey-phase0.py},
\texttt{scripts/rerate-intercoder.py}, \texttt{scripts/survey-kappa.py}). The
re-rating itself was executed through the Anthropic Batch API at a total cost of
USD~2.41. Raw model outputs are released as line-delimited JSON.

137
workspace/drafts/landscape-survey/sections/04-landscape.tex Normal file
View File

@@ -0,0 +1,137 @@
\section{The Landscape}
\label{sec:landscape}
\subsection{Temporal dynamics}
\label{sec:temporal}
Figure~\ref{fig:temporal} shows monthly submission counts for the clean corpus.
Activity is negligible through most of 2024 (mean 3.7 drafts/month over June
2024--May 2025) and inflects sharply in late 2025: October 2025 jumps to 33
drafts, and monthly counts since June 2025 average 38.8. The peak month is
March 2026 with 106 drafts---roughly $35\times$ a typical 2024 month. We report
this as a peak-to-baseline ratio rather than an ``average growth rate'': the
distribution is dominated by a recent spike, not steady exponential growth. The
final two months (April--May 2026) dip relative to the March peak; because
drafts continue to be submitted and indexed, the tail of the curve should be
read as provisional rather than as a downturn.
\begin{figure}[h]
\centering
\begin{tikzpicture}
\node[left,font=\scriptsize] at (-0.35,2.0) {\rotatebox{90}{Drafts/month}};
\draw[gray!40] (-0.1,0.000) -- (11.60,0.000); \node[left,font=\scriptsize] at (-0.1,0.000) {0};
\draw[gray!40] (-0.1,0.943) -- (11.60,0.943); \node[left,font=\scriptsize] at (-0.1,0.943) {25};
\draw[gray!40] (-0.1,1.887) -- (11.60,1.887); \node[left,font=\scriptsize] at (-0.1,1.887) {50};
\draw[gray!40] (-0.1,2.830) -- (11.60,2.830); \node[left,font=\scriptsize] at (-0.1,2.830) {75};
\draw[gray!40] (-0.1,3.774) -- (11.60,3.774); \node[left,font=\scriptsize] at (-0.1,3.774) {100};
\fill[blue!60!black] (0.00,0) rectangle (0.30,0.151);
\fill[blue!60!black] (0.40,0) rectangle (0.70,0.075);
\fill[blue!60!black] (0.80,0) rectangle (1.10,0.038);
\fill[blue!60!black] (1.20,0) rectangle (1.50,0.189);
\fill[blue!60!black] (1.60,0) rectangle (1.90,0.113);
\fill[blue!60!black] (2.00,0) rectangle (2.30,0.038);
\fill[blue!60!black] (2.40,0) rectangle (2.70,0.075);
\fill[blue!60!black] (2.80,0) rectangle (3.10,0.038);
\fill[blue!60!black] (3.20,0) rectangle (3.50,0.377);
\fill[blue!60!black] (3.60,0) rectangle (3.90,0.038);
\fill[blue!60!black] (4.00,0) rectangle (4.30,0.189);
\fill[blue!60!black] (4.40,0) rectangle (4.70,0.113);
\fill[blue!60!black] (4.80,0) rectangle (5.10,0.189);
\fill[blue!60!black] (5.20,0) rectangle (5.50,0.038);
\fill[blue!60!black] (5.60,0) rectangle (5.90,0.075);
\fill[blue!60!black] (6.00,0) rectangle (6.30,0.264);
\fill[blue!60!black] (6.40,0) rectangle (6.70,0.226);
\fill[blue!60!black] (6.80,0) rectangle (7.10,0.151);
\fill[blue!60!black] (7.20,0) rectangle (7.50,0.226);
\fill[blue!60!black] (7.60,0) rectangle (7.90,0.226);
\fill[blue!60!black] (8.00,0) rectangle (8.30,0.302);
\fill[blue!60!black] (8.40,0) rectangle (8.70,1.245);
\fill[blue!60!black] (8.80,0) rectangle (9.10,0.830);
\fill[blue!60!black] (9.20,0) rectangle (9.50,0.415);
\fill[blue!60!black] (9.60,0) rectangle (9.90,1.925);
\fill[blue!60!black] (10.00,0) rectangle (10.30,2.491);
\fill[blue!60!black] (10.40,0) rectangle (10.70,4.000);
\fill[blue!60!black] (10.80,0) rectangle (11.10,2.943);
\fill[blue!60!black] (11.20,0) rectangle (11.50,2.792);
\node[below,font=\scriptsize] at (0.15,-0.05) {2024-01};
\node[below,font=\scriptsize] at (2.55,-0.05) {2024-07};
\node[below,font=\scriptsize] at (4.95,-0.05) {2025-01};
\node[below,font=\scriptsize] at (7.35,-0.05) {2025-07};
\node[below,font=\scriptsize] at (9.75,-0.05) {2026-01};
\node[below,font=\scriptsize] at (11.35,-0.05) {2026-05};
\end{tikzpicture}
\caption{Monthly counts of AI/agent IETF Internet-Drafts, January 2024--May
2026 ($n=524$). The April--May 2026 tail is provisional (indexing lag).}
\label{fig:temporal}
\end{figure}
\subsection{Category distribution}
\label{sec:categories}
Table~\ref{tab:categories} gives the distribution of drafts by primary
category. The field is concentrated in identity/authentication (141 drafts,
$26.9\%$), agent-to-agent communication protocols (108, $20.6\%$), and
autonomous network operations (64, $12.2\%$). The sparsest areas are
human-agent interaction (5), the residual ``Other'' bucket (9), and model
serving/inference (16); we report these descriptively and draw no normative
conclusion about whether sparse areas \emph{ought} to receive more attention.
Categories are not mutually exclusive: $92.9\%$ of drafts carry more than one
category, reflecting genuine thematic overlap (e.g.\ a draft may concern both
agent identity and authorization). We use the primary (first) category for the
distribution.
\begin{table}[h]
\centering
\begin{tabular}{lr}
\toprule
\textbf{Primary category} & \textbf{Drafts} \\
\midrule
Agent identity / authentication & 141 \\
Agent-to-agent (A2A) protocols & 108 \\
Autonomous network operations & 64 \\
ML traffic management & 48 \\
Data formats / interoperability & 44 \\
Agent discovery / registration & 35 \\
Policy / governance & 30 \\
AI safety / alignment & 24 \\
Model serving / inference & 16 \\
Other AI/agent & 9 \\
Human-agent interaction & 5 \\
\midrule
\textbf{Total} & \textbf{524} \\
\bottomrule
\end{tabular}
\caption{Distribution of the clean IETF corpus by primary category.
$92.9\%$ of drafts also carry one or more secondary categories.}
\label{tab:categories}
\end{table}
\subsection{Authorship and working-group structure}
\label{sec:authors}
The corpus is authored by 619 distinct individuals and is \emph{not}
concentrated: the ten most prolific authors together account for only $10.9\%$
of drafts. The most active contributors---Bing Liu (22 drafts), Nan Geng (21),
and Zhenbin Li (20)---form a cluster around autonomous network operations.
The strongest structural signal is the maturity of the work. Of the 524 drafts,
\textbf{456 ($87\%$) are individual submissions} not adopted by any IETF working
group; only 28 distinct working groups appear across the remainder. The area is
thus overwhelmingly \emph{pre-standardization}: a large volume of individual
proposals competing for attention, with comparatively little that has cleared
the bar of working-group adoption.
\subsection{Semantic redundancy}
\label{sec:redundancy}
Pairwise cosine similarity over the 137{,}026 document pairs has a mean of
$0.711$ (p90 $0.790$, p99 $0.850$, max $1.000$). We define a near-duplicate as a
pair with cosine $>0.9$: there are 125 such pairs, and \textbf{170 drafts
($32.4\%$) have at least one near-duplicate} in the corpus. The four pairs at
cosine $\approx 1.0$ are legitimate---they are individual versus
working-group-adopted versions of the same document (for example
\texttt{draft-fv-rats-ear} and \texttt{draft-ietf-rats-ear})---rather than data
artifacts. Combined with the dominance of individual submissions
(Section~\ref{sec:authors}), the high redundancy is consistent with a young,
crowded design space in which many authors propose overlapping solutions to the
same problems before consolidation occurs.

81
workspace/drafts/landscape-survey/sections/05-reliability.tex Normal file
View File

@@ -0,0 +1,81 @@
\section{How Reliable Are the Labels?}
\label{sec:reliability}
Every category in Section~\ref{sec:categories} is an LLM judgment. Before
treating the distribution as a finding, we ask how reproducible those judgments
are. We re-rated all 524 drafts a second time with the same pinned prompt using
two models---Claude Sonnet (\texttt{claude-sonnet-4-20250514}) and Claude Haiku
(\texttt{claude-haiku-4-5-20251001})---and compared the assignments. We report
Cohen's $\kappa$ for the nominal primary-category assignment and
quadratic-weighted $\kappa$ ($\kappa_w$) for the ordinal $1$--$5$ dimensions,
interpreting magnitudes by the Landis--Koch convention (\,$<0.2$ slight,
$0.2$--$0.4$ fair, $0.4$--$0.6$ moderate, $0.6$--$0.8$ substantial, $>0.8$
almost perfect\,).
\paragraph{Category assignment is substantially reliable.}
For the primary category, Sonnet and Haiku agree at $\kappa = 0.652$ (raw
agreement $70.8\%$). Each model independently re-rating also agrees
substantially with the original production labels (Sonnet $\kappa = 0.645$,
Haiku $\kappa = 0.596$). The residual disagreements are not random: they
concentrate on semantically adjacent categories
(Table~\ref{tab:confusion})---A2A protocols versus autonomous network
operations, A2A versus agent discovery, and identity versus the residual
``Other'' bucket. These are boundary cases, not classifier noise, which is why
we treat the distribution as informative while acknowledging $\pm$ a few points
of category-boundary uncertainty.
\begin{table}[h]
\centering
\begin{tabular}{llr}
\toprule
\textbf{Category A} & \textbf{Category B} & \textbf{Disagreements} \\
\midrule
A2A protocols & Autonomous netops & 17 \\
A2A protocols & Agent discovery/reg & 16 \\
A2A protocols & Agent identity/auth & 15 \\
Agent identity/auth & Other AI/agent & 14 \\
Data formats/interop & Other AI/agent & 10 \\
\bottomrule
\end{tabular}
\caption{Most-confused primary-category pairs between the two re-rating models.
Disagreements concentrate on semantic neighbours.}
\label{tab:confusion}
\end{table}
\paragraph{Ordinal quality scores are not reliable.}
The five $1$--$5$ quality dimensions tell a different and cautionary story
(Table~\ref{tab:kappa}). Between Sonnet and Haiku, \emph{overlap} reaches only
$\kappa_w = 0.127$ (slight---effectively no better than chance) and
\emph{novelty} $\kappa_w = 0.206$. \emph{relevance} appears substantial between
the two re-rating models ($0.728$) but collapses to $0.234$ against the
production labels, indicating it is not stable across runs either. Only
\emph{maturity} is consistently moderate ($0.59$--$0.62$).
\begin{table}[h]
\centering
\begin{tabular}{lcc}
\toprule
\textbf{Dimension} & \textbf{$\kappa_w$ (Sonnet vs.\ Haiku)} & \textbf{$\kappa_w$ (Sonnet vs.\ prod.)} \\
\midrule
relevance & 0.728 & 0.234 \\
maturity & 0.592 & 0.620 \\
momentum & 0.457 & 0.247 \\
novelty & 0.206 & 0.477 \\
overlap & 0.127 & 0.282 \\
\bottomrule
\end{tabular}
\caption{Quadratic-weighted $\kappa$ for the ordinal quality dimensions.
Low and inconsistent values---especially for \emph{overlap} and
\emph{novelty}---indicate these scores are not reproducible across raters.}
\label{tab:kappa}
\end{table}
\paragraph{Consequence.}
We therefore report the categorical landscape (Section~\ref{sec:categories}) but
deliberately exclude the LLM ordinal quality scores from our findings. We regard
this split as a general caution rather than an artifact of our particular
pipeline: LLMs can place standards documents into a thematic taxonomy with
substantial agreement, but asking them to score subjective qualities such as
``novelty'' or ``overlap'' on a numeric scale produces labels that do not
survive a change of model. Studies that use LLM-assigned quality scores as
quantitative evidence should report inter-rater reliability before doing so.

65
workspace/drafts/landscape-survey/sections/06-discussion.tex Normal file
View File

@@ -0,0 +1,65 @@
\section{Discussion and Limitations}
\label{sec:discussion}
\subsection{Implications}
Two structural features of the corpus reinforce one another. First, the area is
\emph{pre-standardization}: $456$ of the $524$ drafts ($87\%$) are individual
submissions not adopted by any working group (Section~\ref{sec:authors}).
Second, it is semantically redundant: $170$ drafts ($32.4\%$) have at least one
near-duplicate (cosine $>0.9$) elsewhere in the corpus
(Section~\ref{sec:redundancy}). Taken together these point to a young, crowded
design space in which many authors independently propose overlapping solutions
to the same problems. Such a configuration is consistent with the early phase of
a standards effort, where consolidation and competition between proposals have
yet to resolve into adopted work; it does not, on its own, indicate
duplication of effort in any pejorative sense, since some redundancy is the
expected by-product of parallel exploration. We note this dynamic descriptively
rather than predicting which proposals will prevail.
Beyond the IETF, our reliability result carries a general caution for
LLM-assisted corpus studies. The same pipeline that places documents into a
thematic taxonomy with substantial agreement (Cohen's $\kappa \approx 0.65$)
produces ordinal quality scores---``novelty,'' ``overlap''---that do not survive
a change of model ($\kappa_w$ as low as $0.13$). A study that reported the
category distribution and the quality scores side by side, without a reliability
check, would present a reproducible measurement and rater noise with equal
confidence. The discipline of separating the two, by an explicit inter-rater
analysis, is what allowed us to keep the former and discard the latter.
\subsection{Limitations}
Several limitations bound the interpretation of our findings.
\begin{itemize}[leftmargin=1.4em,topsep=2pt,itemsep=1pt]
\item \textbf{Abstract-only classification.} Categories and scores are derived
from each draft's title, metadata, and abstract, not its full text
(Section~\ref{sec:method}). Classifications therefore reflect how a draft
presents itself rather than a full reading of its mechanics, and a draft
whose abstract understates its technical content may be miscategorised.
\item \textbf{Single snapshot.} The analysis rests on one corpus snapshot
(\texttt{data/drafts.db} as of 2026-05-23). All counts, trends, and
similarities are as of that date and will drift as drafts are added,
revised, expired, or adopted.
\item \textbf{IETF-only scope.} We deliberately restrict the corpus to IETF
Internet-Drafts; documents from ISO, ITU, ETSI, NIST, and W3C are
excluded because of heterogeneous metadata and a different document
model. The survey therefore says nothing about AI-agent standardization
outside the IETF, and the overall scale of the field is correspondingly
understated.
\item \textbf{LLM-derived categories.} The thematic taxonomy is produced by a
large language model. The two-model $\kappa$ check
(Section~\ref{sec:reliability}) mitigates but does not eliminate this
dependence: substantial agreement still leaves category-boundary
uncertainty of a few points, concentrated on semantically adjacent
categories.
\item \textbf{Provisional recent tail.} Submission counts for the most recent
months (April--May 2026) are provisional because of indexing and fetch
lag; the dip after the March 2026 peak in Figure~\ref{fig:temporal}
should be read as incomplete data rather than as a downturn.
\item \textbf{Keyword-based candidate selection.} Candidates were identified by
keyword and topic matching, which can both miss relevant drafts whose
abstracts avoid the chosen vocabulary and over-include unrelated
documents; we observed a $12.2\%$ false-positive rate ($73$ of $597$
candidates) before filtering, and an unknown false-negative rate remains.
\end{itemize}

27
workspace/drafts/landscape-survey/sections/07-conclusion.tex Normal file
View File

@@ -0,0 +1,27 @@
\section{Conclusion and Future Work}
\label{sec:conclusion}
We have presented a verified quantitative survey of AI/agent-related IETF
Internet-Drafts, based on a curated corpus of $524$ documents spanning January
2024 to May 2026. The corpus exhibits a sharp recent surge---from $3.7$ to
$38.8$ drafts per month, peaking at $106$ in March 2026---and is dominated by
individual submissions ($87\%$ not adopted by any working group) with
substantial semantic redundancy ($32.4\%$ of drafts having a near-duplicate),
the signature of a young, pre-standardization design space. Crucially, we
treated the LLM-assisted labels themselves as objects of measurement: a
two-model re-rating shows that categorical assignment is substantially
reproducible (Cohen's $\kappa \approx 0.65$), while ordinal quality scores are
not ($\kappa_w = 0.13$--$0.21$ for the least stable dimensions), so we report
the category landscape and exclude the quality scores.
Several directions extend this work. \emph{Full-text classification} would
replace the abstract-only pipeline and test whether categories shift when the
classifier reads the complete document. \emph{Longitudinal re-runs} on later
snapshots would turn the single-snapshot picture into a moving record of how the
surge evolves and whether redundancy consolidates over time. \emph{Cross-SDO
extension} to ISO, ITU, ETSI, NIST, and W3C---contingent on reconciling their
heterogeneous metadata---would situate the IETF activity within the broader
standards landscape. Finally, \emph{tracking working-group adoption} of
individual drafts would reveal which of the many competing proposals clear the
bar from individual submission to adopted work, giving an empirical handle on
how the pre-standardization field resolves.