# drafts

name: draft-liu-rtgwg-llmsync-multicast
rev: 00
title: Multicast Use Cases for Large Language Model Synchronization
time: 2026-02-12T08:52:28Z
group_uri: /api/v1/group/group/1027/
expires: 2026-08-16T08:52:28Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-08T17:35:45.785470+00:00
source: ietf
source_id: 153637
source_url: https://datatracker.ietf.org/doc/draft-liu-rtgwg-llmsync-multicast/
doc_status: active
abstract:
  Large Language Models (LLMs) deployments are becoming increasingly
     widespread, with inference services being the most common
     application.  This draft will discuss multicast use cases for
     inference cloud services.

full_text:
  RTGWG                                                             Y. Liu
  Internet-Draft                                              China Mobile
  Intended status: Informational                                  Z. Zhang
  Expires: 16 August 2026                                  ZTE Corporation
                                                                  J. Zhang
                                                              China Mobile
                                                          12 February 2026


        Multicast Use Cases for Large Language Model Synchronization
                    draft-liu-rtgwg-llmsync-multicast-00

  Abstract

     Large Language Models (LLMs) deployments are becoming increasingly
     widespread, with inference services being the most common
     application.  This draft will discuss multicast use cases for
     inference cloud services.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 16 August 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.











  Liu, et al.              Expires 16 August 2026                 [Page 1]


  Internet-Draft              Abbreviated Title              February 2026


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
       1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   2
     2.  LLM Synchronization . . . . . . . . . . . . . . . . . . . . .   2
     3.  Multicast technologies applying . . . . . . . . . . . . . . .   4
     4.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   5
     5.  Security Considerations . . . . . . . . . . . . . . . . . . .   5
     6.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   5
       6.1.  Normative References  . . . . . . . . . . . . . . . . . .   5
       6.2.  Informative References  . . . . . . . . . . . . . . . . .   5
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   6

  1.  Introduction

     With the rapid development of AI and the widespread application of
     large language models (LLMs), inference services are the most
     frequently used services.  Different users may use different LLMs,
     and the same user may use multiple LLMs simultaneously for inference
     to obtain the optimal solution.  AI inference cloud providers can
     provide large-scale real-time inference, fine-tuning, and model
     optimization services on GPU cloud platforms.  However, the GPU
     infrastructure of AI inference cloud providers may cover multiple
     cloud platforms and regions, facing significant challenges in
     deployment and application, including highly concurrent model loading
     and severe cold start latency.

     This draft will discuss multicast use cases for inference cloud
     services.

  1.1.  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
     document are to be interpreted as described in [RFC2119].

  2.  LLM Synchronization





  Liu, et al.              Expires 16 August 2026                 [Page 2]


  Internet-Draft              Abbreviated Title              February 2026


                             +------------------+
                             | Model Repository |
                             +--+-----+-----+---+
                                |     |     |
                                |     |     |
               +----------------+     |     +----------------+
               |                      |                      |
               |                      |                      |
     +-------------------+  +-------------------+  +-------------------+
     | +---------------+ |  | +---------------+ |  | +---------------+ |
     | | Local Storage | |  | | Local Storage | |  | | Local Storage | |
     | +---------------+ |  | +---------------+ |  | +---------------+ |
     |                   |  |                   |  |                   |
     | +---------------+ |  | +---------------+ |  | +---------------+ |
     | |  GPU Cluster  | |  | |  GPU Cluster  | |  | |  GPU Cluster  | |
     | +---------------+ |  | +---------------+ |  | +---------------+ |
     |                   |  |                   |  |                   |
     |    GPU Cloud 1    |  |    GPU Cloud 2    |  |    GPU Cloud n    |
     +-------------------+  +-------------------+  +-------------------+

                                    Figure 1

     Highly concurrent model loading refers to the peak load and
     concurrency challenge of simultaneously downloading the same popular
     large model across dozens of GPU cloud platforms.  A single
     deployment of a model may involve 10 to 100+ copies, each requiring a
     complete copy of the model file.  Hundreds of GPU servers
     simultaneously downloading the same model (each ranging from 70GB to
     1TB in size) within a single cluster generates enormous bandwidth
     demands and creates I/O bottlenecks.

     Severe cold start latency refers to the high delay in initial model
     deployment caused by slow download speeds.  Replica startup time is
     limited by inbound network bandwidth, which varies significantly
     between different cloud providers and is typically much lower than
     the cluster's internal bandwidth.  This significantly impacts the
     download efficiency of large models in practical applications.

     Highly concurrent model downloading are typical multicast
     applications.  Such multicast applications have the following
     characteristics:

     *  Large data volume: Due to the large size of the models, typically
        a single model can reach 70GB to 1TB, placing extremely high
        demands on network bandwidth.






  Liu, et al.              Expires 16 August 2026                 [Page 3]


  Internet-Draft              Abbreviated Title              February 2026


     *  Transmission time: Due to cold start latency requirements, data
        transmission needs to be completed as quickly as possible.
        Transmission times exceeding tens of minutes will significantly
        impact user experience; therefore, the shorter the time, the
        better.

     Therefore, when applying multicast technology to this scenario, it is
     necessary to consider ensuring high bandwidth and low latency.

  3.  Multicast technologies applying

     Considering the need to conserve network bandwidth, ingress interface
     replication technology is not suitable for this scenario.  PIM-SM, SR
     P2MP or BIER technologies should be considered instead.

     Protocol Independent Multicast - Sparse Mode (PIM-SM) [RFC7761] is a
     traditional multicast technology.  It is widely used in scenarios
     where the receivers are relatively fixed, such as IPTV systems.  When
     the network topology of the multicast tree changes, a new multicast
     tree needs to be established for each multicast stream via PIM-SM
     signaling after the BGP/IGP protocol converges.  The convergence time
     of the multicast tree is much longer than that of the IGP protocol.

     SR-P2MP (Segment Routing Replication for Multipoint Service Delivery)
     [I-D.ietf-pim-sr-p2mp-policy] is a relatively new tunneling
     technology that uses SR-MPLS/SRv6 (Segment Routing over IPv6)
     tunneling technology for multicast traffic transmission.  It requires
     the routing module of the controller or ingress node to calculate and
     determine the path of the multicast traffic.  Then, the controller or
     ingress node issues a SID (Segment Identifier) for multicast
     operations to the replication point (i.e., the multicast replication
     point) in the network.  When multicast traffic enters the tunnel, it
     is replicated and forwarded at the replication point according to the
     multicast operation SID.  When the network topology changes, the
     controller or ingress node needs to recalculate and determine the
     replication point and issue the multicast operation SID to the
     changed replication point, so that subsequent multicast traffic will
     be forwarded through the new path.

     BIER (Bit-Indexed Explicit Replication) [RFC8279] is an architecture
     that provides optimal multicast forwarding through a "multicast
     domain", without requiring intermediate routers to maintain any per-
     flow state or to engage in an explicit tree-building protocol.  BIER
     is more flexible than PIM-SM and SR P2MP.  When link failures or
     other issues occur on the multicast forwarding path, BIER can
     converge along with IGP convergence, a speed far exceeding that of
     PIM-SM and SR P2MP.




  Liu, et al.              Expires 16 August 2026                 [Page 4]


  Internet-Draft              Abbreviated Title              February 2026


     When considering applying multicast technology to large model
     synchronization scenarios, if the model is synchronized to the same
     destination GPU clouds each time, a multicast tree can be pre-
     established, or the SR replication path can be calculated using the
     controller, and PIM-SM or SR P2MP technologies can be used for model
     copying.

     If the destination GPU clouds for each model synchronization is
     different, pre-establishing a multicast tree or multicast path each
     time using PIM-SM/SR P2MP technologies may be inefficient because
     multicast tree establishment takes time.  In this case, using BIER
     technology is a better choice.

  4.  IANA Considerations

     There are no IANA consideration introduced by this draft.

  5.  Security Considerations

     There are no security issues introduced by this draft.

  6.  References

  6.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC7761]  Fenner, B., Handley, M., Holbrook, H., Kouvelas, I.,
                Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent
                Multicast - Sparse Mode (PIM-SM): Protocol Specification
                (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, March
                2016, <https://www.rfc-editor.org/info/rfc7761>.

     [RFC8279]  Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
                Przygienda, T., and S. Aldrin, "Multicast Using Bit Index
                Explicit Replication (BIER)", RFC 8279,
                DOI 10.17487/RFC8279, November 2017,
                <https://www.rfc-editor.org/info/rfc8279>.

  6.2.  Informative References

     [I-D.ietf-pim-sr-p2mp-policy]
                Parekh, R., Voyer, D., Filsfils, C., Bidgoli, H., and Z.
                J. Zhang, "Segment Routing Point-to-Multipoint Policy",
                Work in Progress, Internet-Draft, draft-ietf-pim-sr-p2mp-



  Liu, et al.              Expires 16 August 2026                 [Page 5]


  Internet-Draft              Abbreviated Title              February 2026


                policy-22, 4 September 2025,
                <https://datatracker.ietf.org/doc/html/draft-ietf-pim-sr-
                p2mp-policy-22>.

  Authors' Addresses

     Yisong Liu
     China Mobile
     China
     Email: liuyisong@chinamobile.com


     Zheng Zhang
     ZTE Corporation
     China
     Email: zhang.zheng@zte.com.cn


     Junye Zhang
     China Mobile
     China
     Email: zhangjunye@chinamobile.com





























  Liu, et al.              Expires 16 August 2026                 [Page 6]


name: draft-zhang-rtgwg-llmmoe-multicast
rev: 02
title: Multicast use case in LLM MoE
time: 2026-04-14T00:47:19Z
group_uri: /api/v1/group/group/1027/
expires: 2026-10-16T00:47:19Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-05-22T09:24:52.065795+00:00
source: ietf
abstract:
  Large Language Models (LLMs) have been widely used in recent years.
     The Mixture of Experts (MoE) architecture is one of the features of
     LLMs that enables efficient inference and cost-effective training.
     With the MoE architecture, there are potential multicast use cases
     such as tokens dispatching.  This draft attempts to analyze these use
     cases.

full_text:
  RTGWG                                                           Z. Zhang
  Internet-Draft                                                   W. Duan
  Intended status: Informational                           ZTE Corporation
  Expires: 23 April 2026                                             X. Xu
                                                              China Mobile
                                                           20 October 2025


                         Multicast usage in LLM MoE
                   draft-zhang-rtgwg-llmmoe-multicast-01

  Abstract

     Large Language Models (LLMs) have been widely used in recent years.
     The Mixture of Experts (MoE) architecture is one of the features of
     LLMs that enables efficient inference and cost-effective training.
     With the MoE architecture, there are potential multicast use cases
     such as tokens dispatching.  This draft attempts to analyze these use
     cases.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 23 April 2026.

  Copyright Notice

     Copyright (c) 2025 IETF Trust and the persons identified as the
     document authors.  All rights reserved.










  Zhang, et al.             Expires 23 April 2026                 [Page 1]


  Internet-Draft              Abbreviated Title               October 2025


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
       1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     2.  Use case - tokens dispatching . . . . . . . . . . . . . . . .   3
       2.1.  Intra-node multicast  . . . . . . . . . . . . . . . . . .   4
       2.2.  Inter-node multicast  . . . . . . . . . . . . . . . . . .   4
       2.3.  Dynamic Requirements  . . . . . . . . . . . . . . . . . .   5
       2.4.  Reliability requirements  . . . . . . . . . . . . . . . .   5
     3.  Multicast technologies analysis . . . . . . . . . . . . . . .   5
     4.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   6
     5.  Security Considerations . . . . . . . . . . . . . . . . . . .   6
     6.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   6
       6.1.  Normative References  . . . . . . . . . . . . . . . . . .   6
       6.2.  Informative References  . . . . . . . . . . . . . . . . .   7
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   7

  1.  Introduction

     In recent years, large language models (LLMs) have been widely used.
     Mixture of Experts Model (MoE) is one of the functions of LLM to
     achieve efficient inference and economical training.  Many LLMs
     currently adopt the MoE architecture, such as DeepSeek-V2/V3, Google
     Gemini 1.5 Pro, xAI Grok-1, Mistral 8*22B, Qwen3, etc.  During
     inference, MoE only activates a small number of parameters to
     determine each output token, which significantly reduces the amount
     of computation required by the processor, thereby reducing the
     overall computational requirements.  Therefore, the fewer parameters
     are activated, the less computation the processor needs to perform.
     In the MoE architecture, one token needs to be sent to multiple
     experts, which is a typical multicast use case.

     In most LLMs, two experts are activated during the computation: one
     is a routed expert and the other is a shared expert.  In DeepSeekV3,
     one token activates eight routed experts and one shared expert.

     When all activated experts are located on a node with multiple GPUs
     installed, only intra-node communication is required.  When activated
     experts are located on different nodes, inter-node communication is



  Zhang, et al.             Expires 23 April 2026                 [Page 2]


  Internet-Draft              Abbreviated Title               October 2025


     required.  Due to the bandwidth difference between intra-node and
     inter-node scenarios, communication across leaf switches and even
     spine switches is inevitable.

     This draft analyzes the multicast use case of LLM in data centers.

  1.1.  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
     document are to be interpreted as described in [RFC2119].

  2.  Use case - tokens dispatching

                 +-----------+               +-----------+
                 |  Spine 1  |               |  Spine x  |
                 +-+------+--+               +-+------+--+
                   |      |                    |      |
                   |      |                    |      +---------+
          +--------+      +--------------------|---------+      |
          |      +-----------------------------+         |      |
          |      |                                       |      |
        +-+------+-+     +----------+                  +-+------+-+
        |  Leaf 1  |     |  Leaf 2  |    ......        |  Leaf n  |
        +-+--+---+-+     +----------+                  +--+--+--+-+
          |  |   |                                        |  |  |
          |  |   +-------------------------------------+  |  |  |
          |  +----------------+                        |  |  |  |
          |                   |              +---------|--|--+  |
        +-+            +------|--------------|---------|--+     +-----+
        |              |      |              |         |              |
      +-+--+----+---+--+-+  +-+--+----+---+--+-+     +-+--+----+---+--+-+
      |GPU1|GPU2|...|GPU8|  |GPU1|GPU2|...|GPU8| ... |GPU1|GPU2|...|GPU8|
      +----+----+---+----+  +----+----+---+----+     +----+----+---+----+
             node 1                 node 2                   node m

                                    Figure 1














  Zhang, et al.             Expires 23 April 2026                 [Page 3]


  Internet-Draft              Abbreviated Title               October 2025


     During the pre-filling and decoding phases, tokens need to be sent to
     all selected experts, including routed experts and shared experts.
     The tokens dispatching can be intra-node or inter-node.  Different
     LLMs use different numbers of experts.  For example, Mixtral uses 8
     experts and activates 2 experts at a time; LlaMa 4 uses 16 experts
     (Scout) or 128 experts (Maverick), and activates 2 experts at a time;
     DeepSeekV3 uses 256 experts and activates 9 experts at a time.  The
     more routed experts there are, the more distribution there is between
     nodes.  In order to balance the experts, it is difficult to limit the
     number of experts to one node even if only two experts (one routed
     expert and one shared expert) are used.

     The tokens dispatching can be optimized.  For example, in DeepSeekV3,
     LLM first selects the node group and then selects the expert from the
     node.  By implementing the node restricted routing function, a
     maximum of four nodes are selected to reduce the inter-node
     consumption of tokens dispatching.  In addition, in order to maximize
     the usage of the high intra-node bandwidth, after the switch or GPU
     in the node receives the tokens, the switch or GPU needs to
     distribute the tokens to the experts in the same node.  This
     optimization aims to reduce the inter-node distribution, but it
     cannot avoid multicast between nodes.

     It is worth noting that even during an inference or training process,
     the experts selected by the token are not fixed.  Each token may be
     sent to a different combination of experts.  For example, in
     DeepSeekv3, each token may be sent to 9 different experts.

     Therefore, the use of multicast may be intra-node or inter-node.  The
     existing multicast implementation methods are different in intra-node
     and inter-node scenarios, and multicast management is more difficult.

  2.1.  Intra-node multicast

     When tokens need to be sent to multiple GPUs in the same node, the
     GPU or the switch connected to the GPU may send the tokens in a
     multicast manner.  This requires the switch or GPU to support the
     multicast function.  This function can reduce the computational
     burden of the source GPU and reduce the bandwidth consumption between
     nodes.

  2.2.  Inter-node multicast

     When tokens need to be sent to multiple nodes, Leaf switches and even
     Spine switches need to forward tokens.  Due to the limitation of
     inter-node bandwidth, the more packets there are, the greater the
     risk of congestion.  Using multicast technology can reduce the burden
     on the source GPU and reduce the risk of congestion.



  Zhang, et al.             Expires 23 April 2026                 [Page 4]


  Internet-Draft              Abbreviated Title               October 2025


  2.3.  Dynamic Requirements

     Due to the random nature of multicast destination selection, for
     example, in the token dispatching process mentioned above, each token
     may select a different expert combination.  The selection process is
     very short, leaving no time for multicast technologies like PIM to
     establish a multicast tree.  Therefore, multicast technology that can
     meet dynamic needs is needed.

  2.4.  Reliability requirements

     The transmission of all types of data (including tokens) used for LLM
     calculations requires extremely high reliability.  This means that
     packet loss, long delays or jitters, and retransmissions during data
     transmission can impact LLM calculations.  Reliability is a paramount
     requirement when applying multicast technology to data transmission.

     If reliability is insufficient, even if most data reaches its
     destination quickly, if even one destination fails to receive the
     data in time due to packet loss, long latency, or excessive jitter,
     the LLM calculation may need to be restarted, significantly reducing
     computational efficiency.

     So compared to unicast, multicast reliability technology is more
     complex.  Especially in LLM applications, it is necessary to avoid
     packet loss, long latency, excessive jitter, and retransmissions
     caused by individual multicast branches to minimize the impact on LLM
     calculations.

  3.  Multicast technologies analysis

     Protocol Independent Multicast - Sparse Mode (PIM-SM) [RFC7761] is a
     traditional multicast technology.  It relies on PIM signaling to
     build the multicast tree.  When the receivers change, the multicast
     tree may need to be rebuilt.  When PIM is used for intra-node or
     inter-node multicast, the stability of the multicast tree is more
     important.  It may not be applicable when the expert combination is
     flexible.  Even in the intra-node scenario, the number of potential
     multicast trees may be large despite the limited number of GPUs in a
     single node.

     BIER (Bit-Indexed Explicit Replication) [RFC8279] is an architecture
     that provides optimal multicast forwarding through a "multicast
     domain", without requiring intermediate routers to maintain any per-
     flow state or to engage in an explicit tree-building protocol.  BIER
     is more flexible than PIM.  Experts can be numbered and can act as
     ingress or egress BFRs in BIER.  BIER header encapsulation can be a
     function defined in [RFC8296], [I-D.ietf-bier-bierin6], or



  Zhang, et al.             Expires 23 April 2026                 [Page 5]


  Internet-Draft              Abbreviated Title               October 2025


     [I-D.zzhang-bier-unmasked-bier].  By using the BIER function, Leaf
     and Spine switches, and even GPUs or switches connected to GPUs, can
     pre-build expert-based forwarding tables. tokens can be sent to any
     selected expert.

     Other multicast methods, such as PIM DM (dense mode) and ingress
     replication, may consume more bandwidth and may not be a good choice
     for multicast scenarios such as LLM tokens dispatching.

     Considering dynamic requirements like token dispatching, technologies
     like PIM, which require the establishment of a multicast tree, are
     inadequate.  BIER, on the other hand, allows the source GPU (similar
     to BFIR in BIER) to directly specify the destination expert group
     (similar to BFERs in BIER) and encapsulate it into the message,
     eliminating the time for multicast tree establishment.  Therefore,
     BIER is the most suitable multicast technology.  Of course, prior to
     this, control-plane negotiation between the source GPU and the
     experts regarding data transmission is required.

     While the network layer can provide multicast capabilities for
     multicast scenarios, the multicast approach needs to work in
     conjunction with the LLM software.  It may work in conjunction with
     the implementation of collective communication and NIC (network
     interface card).

  4.  IANA Considerations

     There are no IANA consideration introduced by this draft.

  5.  Security Considerations

     There are no security issues introduced by this draft.

  6.  References

  6.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC7761]  Fenner, B., Handley, M., Holbrook, H., Kouvelas, I.,
                Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent
                Multicast - Sparse Mode (PIM-SM): Protocol Specification
                (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, March
                2016, <https://www.rfc-editor.org/info/rfc7761>.




  Zhang, et al.             Expires 23 April 2026                 [Page 6]


  Internet-Draft              Abbreviated Title               October 2025


     [RFC8279]  Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
                Przygienda, T., and S. Aldrin, "Multicast Using Bit Index
                Explicit Replication (BIER)", RFC 8279,
                DOI 10.17487/RFC8279, November 2017,
                <https://www.rfc-editor.org/info/rfc8279>.

     [RFC8296]  Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
                Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation
                for Bit Index Explicit Replication (BIER) in MPLS and Non-
                MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January
                2018, <https://www.rfc-editor.org/info/rfc8296>.

  6.2.  Informative References

     [I-D.ietf-bier-bierin6]
                Zhang, Z., Zhang, Z. J., Wijnands, I., Mishra, M. P.,
                Bidgoli, H., and G. S. Mishra, "Supporting BIER in IPv6
                Networks (BIERin6)", Work in Progress, Internet-Draft,
                draft-ietf-bier-bierin6-12, 25 August 2025,
                <https://datatracker.ietf.org/doc/html/draft-ietf-bier-
                bierin6-12>.

     [I-D.zzhang-bier-unmasked-bier]
                Przygienda, T., Zhang, Z. J., Bigdoli, H., and I.
                Wijnands, "Unmasked BIER Mode", Work in Progress,
                Internet-Draft, draft-zzhang-bier-unmasked-bier-01, 23
                February 2025, <https://datatracker.ietf.org/doc/html/
                draft-zzhang-bier-unmasked-bier-01>.

  Authors' Addresses

     Zheng Zhang
     ZTE Corporation
     China
     Email: zhang.zheng@zte.com.cn


     Wei Duan
     ZTE Corporation
     China
     Email: duan.wei1@zte.com.cn


     Xiaohu Xu
     China Mobile
     China
     Email: xuxiaohu_ietf@hotmail.com




  Zhang, et al.             Expires 23 April 2026                 [Page 7]


name: draft-irtf-nmrg-llm-nm
rev: 00
title: A Framework for LLM-Assisted Network Management with Human-in-the-Loop
time: 2025-09-14T07:09:00Z
group_uri: /api/v1/group/group/38/
expires: 2025-09-14T15:44:40Z
states: ["/api/v1/doc/state/2/", "/api/v1/doc/state/150/", "/api/v1/doc/state/57/"]
categories: []
tags: []
fetched_at: 2026-03-07T06:55:11.565648+00:00
source: ietf
abstract:
  This document defines an interoperable framework that facilitates
     collaborative network management between Large Language Models (LLMs)
     and human operators.  The proposed framework introduces enhanced
     telemetry module, LLM decision module and standardized interaction
     data models between human operators and LLM-driven systems, and
     workflows to enforce human oversight.  The approach ensures
     compatibility with existing network management systems and protocols
     while improving automation and decision-making capabilities in
     network operations.

full_text:
  Network Management Research Group                                 Y. Cui
  Internet-Draft                                       Tsinghua University
  Intended status: Informational                                   M. Xing
  Expires: 4 September 2025                                       L. Zhang
                                                   Zhongguancun Laboratory
                                                              3 March 2025


   A Framework for LLM-Assisted Network Management with Human-in-the-Loop
                         draft-irtf-nmrg-llm-nm-00

  Abstract

     This document defines an interoperable framework that facilitates
     collaborative network management between Large Language Models (LLMs)
     and human operators.  The proposed framework introduces enhanced
     telemetry module, LLM decision module and standardized interaction
     data models between human operators and LLM-driven systems, and
     workflows to enforce human oversight.  The approach ensures
     compatibility with existing network management systems and protocols
     while improving automation and decision-making capabilities in
     network operations.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 4 September 2025.

  Copyright Notice

     Copyright (c) 2025 IETF Trust and the persons identified as the
     document authors.  All rights reserved.







  Cui, et al.             Expires 4 September 2025                [Page 1]


  Internet-Draft                   LLM4Net                      March 2025


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
       1.1.  Motivation  . . . . . . . . . . . . . . . . . . . . . . .   2
       1.2.  Problem Statement . . . . . . . . . . . . . . . . . . . .   3
     2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
       2.1.  Acronyms and Abbreviations  . . . . . . . . . . . . . . .   3
     3.  Framework Overview  . . . . . . . . . . . . . . . . . . . . .   3
       3.1.  Enhanced Telemetry Module . . . . . . . . . . . . . . . .   5
       3.2.  LLM Decision Module . . . . . . . . . . . . . . . . . . .   5
         3.2.1.  RAG Module  . . . . . . . . . . . . . . . . . . . . .   5
         3.2.2.  Task Instance Module  . . . . . . . . . . . . . . . .   6
         3.2.3.  Task Instance Management Module . . . . . . . . . . .   6
         3.2.4.  Config Verify Module  . . . . . . . . . . . . . . . .   7
         3.2.5.  Access Control Module . . . . . . . . . . . . . . . .   7
       3.3.  Operator Audit Module . . . . . . . . . . . . . . . . . .   8
     4.  Data Model  . . . . . . . . . . . . . . . . . . . . . . . . .   9
       4.1.  LLM Response Data Model . . . . . . . . . . . . . . . . .   9
       4.2.  Human Audit Data Model  . . . . . . . . . . . . . . . . .   9
     5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  11
     6.  Security Considerations . . . . . . . . . . . . . . . . . . .  11
     7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  11
       7.1.  Normative References  . . . . . . . . . . . . . . . . . .  11
       7.2.  Informative References  . . . . . . . . . . . . . . . . .  11
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  12

  1.  Introduction

  1.1.  Motivation

     Traditional network automation systems struggle with handling
     unanticipated scenarios and managing complex multi-domain data
     dependencies.  Large Language Models (LLMs) offer advanced multimodal
     data comprehension, adaptive reasoning, and generalization
     capabilities, making them a promising tool for network management and
     autonomous network[TM-IG1230].  However, full automation remains
     impractical due to risks such as model hallucination, operational
     errors, and the lack of accountability in decision-making[Huang25].
     This document proposes a structured framework that integrates LLMs
     into network management through human-in-the-loop collaboration,
     leveraging their strengths while ensuring oversight, reliability, and
     operational safety.



  Cui, et al.             Expires 4 September 2025                [Page 2]


  Internet-Draft                   LLM4Net                      March 2025


  1.2.  Problem Statement

     Network management faces significant challenges, including the
     complexity of multi-vendor configurations, the real-time correlation
     of heterogeneous telemetry data, and the need for rapid responses to
     dynamic security threats.  LLMs offer a promising approach to
     addressing these challenges through their advanced multimodal data
     understanding and adaptive reasoning capabilities.  However, the
     direct application of LLMs in network management introduces several
     technical considerations.  These include the need for semantic
     enrichment of network telemetry to enhance LLM comprehension, a dual-
     channel decision execution mechanism with confidence-based
     escalation, and auditability of LLM-generated decisions through
     provenance tracking.  Addressing these requirements is critical to
     integrating LLMs effectively into network management workflows while
     maintaining reliability, transparency, and interoperability.

  2.  Terminology

  2.1.  Acronyms and Abbreviations

     *  LLM: Large Language Model

     *  RAG: Retrieve Augmented Generation

  3.  Framework Overview

























  Cui, et al.             Expires 4 September 2025                [Page 3]


  Internet-Draft                   LLM4Net                      March 2025


     +-------------------------------------------------------------+
     |             LLM-Assisted Netwrok Management System          |
     +-------------------------------------------------------------+
     |+--------------------LLM Decision Module--------------------+|
     ||                                                           ||
     ||               +--Task Instance Module--+  +-------------+ ||
     ||               | +------+  +----------+ |  |Task Instance<-----+
     ||               | |Prompt|  |Fine-Tuned| <->| Mgt Module  | ||  |
     ||               | | Lib  |  |Weight Lib| |  +-------------+ ||  |
     || +----------+  | +------+  +----------+ |  +-------------+ ||  |
     || |RAG Module|<-> +--------------------+ |  |Config Verify| ||  |
     || +-----^----+  | |Foundation Model Lib| -->|    Module   | ||  |
     ||       |       | +--------------------+ |  +-------|-----+ ||  |
     ||       |       +----^---------------^---+  +-------v------+||  |
     ||       |            |               |      |Access Control|||  |
     ||       |            |               |      |    Module    |||  |
     ||       |            |               |      +-------|------+||  |
     |+-------|------------|---------------|--------------|-------+|  |
     |+-------v------------v----+ +--------v--------------v-------+|  |
     ||Enhanced Telemetry Module| |   Operator Audit Module       ||  |
     |+-----------^-------------+ +--------------|-------------^--+|  |
     +------------|------------------------------|-------------|---+  |
                  |                              |       +-----v---+  |
                  |                              |       |Operator <--+
                  |                              |       +---------+
     +------------v------------------------------v------------------+
     |               Original Netwrok Management System             |
     +------------------------------^-------------------------------+
                                    |
     +------------------------------v-------------------------------+
     |                       Physical Network                       |
     +--------------------------------------------------------------+

     Figure 1: The LLM-Assisted Network Management Framework

     The proposed framework is shown in Figure 1, highlighting the key
     components of LLM-assisted network management.  The human operator
     can generate a specific task instance, e.g., fault analysis or
     topology optimization, using the task instance management module.
     According to the task type, the task instance module can instantiate
     a task instance with specific foundation model, prompt, and
     adaptation fine-tuned parameters[Hu22].  The enahnced telemetry
     module improves the semantics of raw telemetry data from original
     network management system, providing supplementary information to the
     LLM decision module for more informed decision-making.  After the
     decision-making, the generated configuration parameters are validated
     against the YANG model and enforced with access control rules.  The
     operator audit module provides a structured mechanism for human



  Cui, et al.             Expires 4 September 2025                [Page 4]


  Internet-Draft                   LLM4Net                      March 2025


     oversight of LLM-generated configurations, and the configuration can
     be issued to the original network management system for deployment
     once the operator approves.

  3.1.  Enhanced Telemetry Module

     The Enhanced Telemetry Module improves the semantics of raw telemetry
     data, providing supplementary information to the LLM decision module
     for more informed decision-making.  Telemetry data retrieved from
     network devices via NETCONF[RFC6241], e.g., in XML format, often
     lacks field descriptions, structured metadata, and vendor-specific
     details.  Since this information is not included in the pre-trained
     knowledge of LLMs, it can lead to misinterpretation and erroneous
     reasoning.  To address this limitation, an external knowledge base
     should be introduced to store YANG model schema, device manuals, and
     other relevant documentation.  The Enhanced Telemetry Module
     functions as middleware between the network management system and the
     external knowledge base.  Through its southbound interface, it
     retrieves NETCONF data from the NETCONF client of existing network
     management system.  Through its northbound interface, the module
     queries the external knowledge base for the corresponding YANG model
     or device manual.  To enhance semantic richness, the Enhanced
     Telemetry Module processes the retrieved data by simplifying
     formatted content (e.g., removing redundant or closing XML tags) and
     appending XML path and description information from the YANG tree to
     the relevant fields.  This approach ensures that the LLM has access
     to structured, contextually enriched data, improving its ability to
     analyze and reason about network telemetry.

  3.2.  LLM Decision Module

  3.2.1.  RAG Module

     The pre-trained LLM may not encompass specific task requirements or
     vendor-specific knowledge.  To address this kind of limitation, the
     Retrieve-Augmented Generation (RAG)[Lewis20] approach is widely used.
     This module retrieves relevant information from operator-defined
     sources, such as device documentation and expert knowledge, and
     integrates it with the Enhanced Telemetry Module to obtain YANG model
     schema.  The retrieved textual data is stored in a database, either
     as raw text or in a vectorized format for efficient search and
     retrieval.  For a given task context, the module retrieves relevant
     knowledge from the database and incorporates it into the input
     context, improving the understanding and response accuracy of LLM.







  Cui, et al.             Expires 4 September 2025                [Page 5]


  Internet-Draft                   LLM4Net                      March 2025


  3.2.2.  Task Instance Module

     To execute a specific task, such as traffic analysis, traffic
     optimization, or fault remediation, a corresponding task instance
     must be created.  A task instance consists of a selected LLM
     foundation model, an associated prompt and fine-tuned weights.

     *  Foundation Model Library.  Operators must select an appropriate
        foundation model based on the specific task requirements.
        Examples include general-purpose models such as GPT-4, LLaMA, and
        DeepSeek, as well as domain-specific models fine-tuned on private
        datasets.  Since foundation models are trained on different
        datasets using varying methodologies, their performance may differ
        across tasks.

     *  Fine-Tuned Weight Library.  For domain-specific applications,
        fine-tuned weights can be applied on top of a foundation model to
        efficiently adapt it to private datasets.  One commonly used
        approach is to store the fine-tuned weights as the difference
        between the original foundation model and the adapted model, which
        can largely reduce storage requirements.  The Fine-Tuned Weights
        Module supports the selection and loading of an appropriate
        foundation model along with the corresponding fine-tuned weights,
        based on the selection of operators.  This ensures flexibility in
        leveraging both general-purpose and domain-specific knowledge
        while maintaining computational efficiency.

     *  Prompt Library.  For each task, it is essential to accurately
        define the task description, the format of its inputs and outputs.
        These definitions are stored in a structured prompt library.  When
        an operator instantiates a task, the corresponding prompt,
        including placeholders for contextual information, is
        automatically retrieved. operator inputs and device data are then
        incorporated into the prompt at the designated placeholders,
        ensuring a structured and consistent interaction with the language
        model.

  3.2.3.  Task Instance Management Module

     The Task Instance Management Module is responsible for the creation,
     update, and deletion of task instances.  This module ensures that
     each instance is appropriately configured to align with the intended
     network management objective.








  Cui, et al.             Expires 4 September 2025                [Page 6]


  Internet-Draft                   LLM4Net                      March 2025


  3.2.4.  Config Verify Module

     To ensure correctness and policy compliance, LLM-generated
     configurations MUST pass the YANG schema validation steps before
     being queued for human approval.  This module ensures that only
     syntactically correct configurations are presented for operator
     review, thereby reducing errors and enhancing network reliability.

  3.2.5.  Access Control Module

     Although the Configuration Verify Module can guarantee the syntactic
     correction, LLMs may generate unintended or potentially harmful
     operations on critical network devices, it is essential for operators
     to enforce clear permission boundaries for LLM task instance to
     ensure security and operational integrity.  The Network Configuration
     Access Control Model defined in [RFC8341] provides a framework for
     specifying access permissions, which can be used to grant access to
     LLM task instances.  This data model includes the concepts of users,
     groups, access operation types, and action types, which can be
     applied as follows:

     *  User and Group: Each task instance should be registered as a
        specific user, representing an entity with defined access
        permissions for particular devices.  These permissions control the
        types of operations the LLM is authorized to perform.  A task
        instance (i.e., user) is identified by a unique string within the
        system.  Access control can also be applied at the group level,
        where a group consists of zero or more members, and a task
        instance can belong to multiple groups.

     *  Access Operation Types: These define the types of operations
        permitted, including create, read, update, delete, and execute.
        Each task instance may support different sets of operations
        depending on its assigned permissions.

     *  Action Types: These specify whether a given operation is permitted
        or denied.  This mechanism determines whether an LLM request to
        perform an operation is allowed based on predefined access control
        rules.

     *  Rule List: A rule governs access control by specifying the content
        and operations a task instance is authorized to handle within the
        system.

     This module must enforce explicit restrictions on the actions an LLM
     is permitted to perform, ensuring that network configurations remain
     secure and compliant with operational policies.




  Cui, et al.             Expires 4 September 2025                [Page 7]


  Internet-Draft                   LLM4Net                      March 2025


  3.3.  Operator Audit Module

     The Operator Audit Module provides a structured mechanism for human
     oversight of LLM-generated configurations before deployment.  The
     output from the LLM Decision Module should include both the generated
     configuration parameters and an associated confidence score.  The
     configuration parameters are validated for compliance with the YANG
     model and are subject to Access Control Rules enforcement.  The
     confidence score, e.g., ranging from 0 to 100, serves as a reference
     for operators to assess the reliability of the generated
     configuration.  Each audit process must track the input context
     (e.g., input data, RAG query content, model selection, configuration
     files) and the corresponding output results.  The auditing steps are
     as follows:

     *  Result Verification: The operator verifies the LLM-generated
        output to ensure alignment with business objectives and policy
        requirements.

     *  Compliance Check: The operator ensures that the LLM output adheres
        to regulatory standards and legal requirements.

     *  Security Verification: The operator examines the output for
        potential security risks, such as misconfigurations or
        vulnerabilities.

     *  Suggestions and Corrections: If issues are identified, the
        operator documents the findings and proposes corrective actions.

     Upon completing the audit, the system maintains an audit decision
     record to ensure traceability of operator actions.  The audit record
     includes the following information:

     *  Timestamp of the audit action

     *  LLM Task Instance ID associated with the action

     *  Operator decisions, including approval, rejection, modification,
        or pending status

     *  Executed command reflecting the final action taken

     *  Operation type (e.g., configuration update, deletion, or
        execution)

     This structured approach ensures that all LLM-generated
     configurations undergo rigorous human review, maintaining operational
     accountability and security.



  Cui, et al.             Expires 4 September 2025                [Page 8]


  Internet-Draft                   LLM4Net                      March 2025


  4.  Data Model

     This section defines the essential data models for LLM-assisted
     network management, including the LLM decision response and human
     audit records.

  4.1.  LLM Response Data Model

     The LLM decision module should respond with the generated
     configuration parameters along with an associated confidence score.
     If the LLM is unable to generate a valid configuration, it should
     return an error message accompanied by an explanation of the issue.

     module: llm-response-module
       +--rw llm-response
          +--rw config?         string
          +--rw confidence?     uint64
          +--rw error-reason?   enumeration

     The LLM response YANG model is structured as follows:

     module llm-response-module {
       namespace "urn:ietf:params:xml:ns:yang:ietf-nmrg-llmn4et";
       prefix llmresponse;
       container llm-response {
         leaf config {
           type string;
         }
         leaf confidence {
           type uint64;
         }
         leaf error-reason {
           type enumeration {
             enum unsupported-task;
             enum unsupported-vendor;
           }
         }
       }
     }

  4.2.  Human Audit Data Model

     This data model defines the structure for human audit operations and
     record-keeping.  It facilitates collaborative decision-making by
     allowing LLMs to generate actionable insights while ensuring that
     human operators retain final operational authority.





  Cui, et al.             Expires 4 September 2025                [Page 9]


  Internet-Draft                   LLM4Net                      March 2025


     module: human-audit-module
       +--rw human-audit
          +--rw task-id?      string
          +--rw generated-config?   string
          +--rw confidence?         int64
          +--rw human-actions
             +--rw operator?          string
             +--rw action?            enumeration
             +--rw modified-config?   string
             +--rw timestamp?         yang:date-and-time

     The human audit YANG model is structured as follows:

     module human-audit-module {
       namespace "urn:ietf:params:xml:ns:yang:ietf-nmrg-llmn4et";
       prefix llmaudit;
       import ietf-yang-types { prefix yang; }

       container human-audit {
         leaf task-id {
           type string;
           }
         leaf generated-config {
           type string;
           }
         leaf confidence {
           type int64;
           }
         container human-actions {
           leaf operator {
             type string;
             }
           leaf action {
             type enumeration {
               enum approve;
               enum modify;
               enum reject;
               }
             }
           leaf modified-config {
             type string;
             }
           leaf timestamp {
             type yang:date-and-time;
             }
         }
       }
     }



  Cui, et al.             Expires 4 September 2025               [Page 10]


  Internet-Draft                   LLM4Net                      March 2025


  5.  IANA Considerations

     This document includes no request to IANA.

  6.  Security Considerations

     *  Model Hallucination: A key challenge is that, without proper
        constraints, the LLM may produce malformed or invalid
        configurations.  This issue can be mitigated using techniques such
        as Constrained Decoding, which enforces syntactic correctness by
        modeling the configuration syntax and restricting the output to
        conform to predefined rules during the generation process.

     *  Training Data Poisoning: LLMs can be trained on malicious or
        biased data, potentially leading to unintended behavior or
        security vulnerabilities.  To mitigate this risk, LLMs should be
        trained on curated, high-quality datasets with rigorous validation
        and filtering processes.  Periodic retraining and adversarial
        testing should also be conducted to detect and correct anomalies
        before deployment.

  7.  References

  7.1.  Normative References

     [RFC8341]  Bierman, A. and M. Bjorklund, "Network Configuration
                Access Control Model", STD 91, RFC 8341,
                DOI 10.17487/RFC8341, March 2018,
                <https://www.rfc-editor.org/rfc/rfc8341>.

     [RFC6241]  Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed.,
                and A. Bierman, Ed., "Network Configuration Protocol
                (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, June 2011,
                <https://www.rfc-editor.org/rfc/rfc6241>.

  7.2.  Informative References

     [TM-IG1230]
                McDonnell, K., Machwe, A., Milham, D., O’Sullivan, J.,
                Niemöller, J., Varvello, L. F., Devadatta, V., Lei, W.,
                Xu, W., Yuan, X., and Y. Stein, "Autonomous Networks
                Technical Architecture", February 2023.

     [Huang25]  Huang,, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H.,
                Chen, Q., Peng, W., Feng, X., Qin, B., and T. Liu, "A
                Survey on Hallucination in Large Language Models
                Principles, Taxonomy, Challenges, and Open Questions",
                n.d..



  Cui, et al.             Expires 4 September 2025               [Page 11]


  Internet-Draft                   LLM4Net                      March 2025


     [Hu22]     Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y.,
                Wang, S., Wang, L., and W. Chen, "LoRA Low-Rank Adaptation
                of Large Language Models", n.d..

     [Lewis20]  Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin,
                V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t.,
                Rocktäschel, T., and S. Riede, "Retrieval-augmented
                generation for knowledge-intensive NLP tasks", n.d..

  Authors' Addresses

     Yong Cui
     Tsinghua University
     Beijing, 100084
     China
     Email: cuiyong@tsinghua.edu.cn
     URI:   http://www.cuiyong.net/


     Mingzhe Xing
     Zhongguancun Laboratory
     Beijing, 100094
     China
     Email: xingmz@zgclab.edu.cn


     Lei Zhang
     Zhongguancun Laboratory
     Beijing, 100094
     China
     Email: zhanglei@zgclab.edu.cn




















  Cui, et al.             Expires 4 September 2025               [Page 12]


name: draft-liu-nmrg-ai-llm-inference-requirements
rev: 02
title: Requirements Analysis of System and Network for Large Language Model Inference Service
time: 2025-11-04T02:55:18Z
group_uri: /api/v1/group/group/1027/
expires: 2026-05-08T02:55:18Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-08T17:35:45.720771+00:00
source: ietf
source_id: 143440
source_url: https://datatracker.ietf.org/doc/draft-liu-nmrg-ai-llm-inference-requirements/
doc_status: active
abstract:
  With the rise of ChatGPT, DeepSeek, and other Large Language Models,
     which is short for LLMs in the remaining part, as well as the
     proliferation of inference applications, inference serving oriented
     to large-scale users has become increasingly critical.  However, due
     to the extreme demands on computing power and communication during
     inference, the large-scale service deployment of LLMs poses
     significant challenges.  To address these challenges, different
     vendors have adopted diverse inference service architectures, such as
     vLLM, SGLang, Mooncake, etc.  This paper investigates mainstream
     inference frameworks, summarizes their core design principle and
     research question, and analyzes the challenges and requirements they
     impose on network management.  The goal is to lay a foundation for
     defining a unified LLM inference architecture in the future.

full_text:
  Network Management                                                C. Liu
  Internet-Draft                                                    C. Guo
  Intended status: Informational                              China Mobile
  Expires: 7 May 2026                                      3 November 2025


    Requirements Analysis of System and Network for Large Language Model
                             Inference Service
              draft-liu-nmrg-ai-llm-inference-requirements-02

  Abstract

     With the rise of ChatGPT, DeepSeek, and other Large Language Models,
     which is short for LLMs in the remaining part, as well as the
     proliferation of inference applications, inference serving oriented
     to large-scale users has become increasingly critical.  However, due
     to the extreme demands on computing power and communication during
     inference, the large-scale service deployment of LLMs poses
     significant challenges.  To address these challenges, different
     vendors have adopted diverse inference service architectures, such as
     vLLM, SGLang, Mooncake, etc.  This paper investigates mainstream
     inference frameworks, summarizes their core design principle and
     research question, and analyzes the challenges and requirements they
     impose on network management.  The goal is to lay a foundation for
     defining a unified LLM inference architecture in the future.

  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
     document are to be interpreted as described in RFC 2119 [RFC2119].

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 7 May 2026.



  Liu & Guo                  Expires 7 May 2026                   [Page 1]


  Internet-Draft             Network Management              November 2025


  Copyright Notice

     Copyright (c) 2025 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     2.  Service-Oriented Inference Frameworks . . . . . . . . . . . .   3
       2.1.  PD Fusion Architecture  . . . . . . . . . . . . . . . . .   4
       2.2.  PD Disaggregation Architecture  . . . . . . . . . . . . .   5
     3.  Inference-related Metrics . . . . . . . . . . . . . . . . . .   6
     4.  Research Question for Service-Oriented Inference
             Frameworks  . . . . . . . . . . . . . . . . . . . . . . .   7
     5.  Challenges for Service-Oriented Inference Frameworks  . . . .   7
       5.1.  Challenge 1 . . . . . . . . . . . . . . . . . . . . . . .   8
       5.2.  Challenge 2 . . . . . . . . . . . . . . . . . . . . . . .   8
       5.3.  Challenge 3 . . . . . . . . . . . . . . . . . . . . . . .   8
     6.  Network Management Requirements for Service-Oriented Inference
             Frameworks  . . . . . . . . . . . . . . . . . . . . . . .   8
       6.1.  Efficient Load Balancing  . . . . . . . . . . . . . . . .   8
       6.2.  KV Cache Management . . . . . . . . . . . . . . . . . . .   9
       6.3.  KV Cache Transmission . . . . . . . . . . . . . . . . . .   9
     7.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
     8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
     9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
       9.1.  Normative References  . . . . . . . . . . . . . . . . . .   9
       9.2.  Informative References  . . . . . . . . . . . . . . . . .   9
     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  10

  1.  Introduction

     Since the launch of ChatGPT in 2023, more and more product-level LLMs
     have emerged, with GPT-4o, Claude-Sonnet-3.5, Gemini, Kimi, and
     others leading the charge.  In early 2025, DeepSeek-R1 reignited the
     LLM frenzy, and Musk's xAI recently unveiled the powerful Grok3.  It
     is evident that LLMs will continue to reach new heights.





  Liu & Guo                  Expires 7 May 2026                   [Page 2]


  Internet-Draft             Network Management              November 2025


     Major vendors, including OpenAI, Anthropic, DeepSeek, and Google,
     have deployed their LLM applications across mobile and web platforms.
     As the field grows, daily active users (DAUs) for these applications
     are expected to surge, potentially reaching hundreds of millions
     during peak periods.  This presents significant challenges for large-
     scale inference services.  For instance, up to now, DeepSeek still
     struggles with persistent "Service Busy" issues.

     Existing large-scale inference service architectures primarily adopt
     two technical approaches: Prefill-Decoding (PD) Fusion and Prefill-
     Decoding Disaggregation, which is derived from the distinct
     computational characteristics of the Prefill (compute-intensive) and
     Decoding (memory-intensive) phases.  Efficient network management and
     hardware coordination are essential to maximize system throughput and
     minimize user-perceived latency.

     This document first introduces mainstream inference frameworks, then
     optimization metrics, and finally elaborates on the network and
     system requirements for deploying large-scale LLM inference services.

  2.  Service-Oriented Inference Frameworks

     At present, there are two main technical routes of the mainstream LLM
     service systems, namely PD Fusion and PD Disaggregation.  Prefill,
     which is to simultaneously compute all of tokens of user requests,
     also known as prompts, is characterized as computational intensive,
     computing-bound, with extremely high computing force requirements.
     Decoding generates user-required content based on the KV Cache and
     first token generated by Prefill phase.  Due to the reuse of KV Cache
     of the tokens prior to the current token, it is characterized as
     memory-intensive and memory-bound, with higher requirements for
     memory in decoding phase.  A complete LLM inference procedure is
     shown in Figure 1.  Based on whether to decouple two stages with
     obviously different computing requirements, two technical routes of
     LLM inference serving system emerge, namely, PD Fusion and decoupled
     PD Disaggregation.  The rest of this section describes in detail
     about the two technical architectures.














  Liu & Guo                  Expires 7 May 2026                   [Page 3]


  Internet-Draft             Network Management              November 2025


    +-------------+  +-------------+ +-------------+ +-------------+
    |     LLM     |  |     LLM     | |     LLM     | |     LLM     |
    | Iteration 1 +-+| Iteration 2 ++| Iteration 3 ++| Iteration 4 ++
    +-----^-------+ |+---^------^--+|+---^-------^-+|+---^-------^-+|
          |         |    |      |   |    |       |  |    |       |  |
          |         | +--+--+   |   | +--+--+    |  | +--+--+    |  |
   <Prompt:Is apple | | KV  | <Yes> | | KV  |  <It> | | KV  |  <Is> |<EOS>
           a fruit?>| |Cache|   ^   | |Cache|    ^  | |Cache|    ^  |  ^
                    | +--^--+   |   | +--^--+    |  | +--^--+    |  |  |
                    |    |      |   |    |       |  |    |       |  |  |
                    +----+------+   +----+-------+  +----+-------+  +--+

   +-----Prefill----+--------------------Decoding----------------------+

                      Figure 1: LLM Inference Process

     Prefill: Processes all tokens in user prompts (Parallelizable,
     compute-bound, requiring high computing power).

     Decoding: Generates output tokens sequentially based on the KV Cache
     from Prefill (Memory-bound, requiring high GPU memory).

  2.1.  PD Fusion Architecture

     In PD Fusion, LLM instances are deployed within a single cluster,
     managed by a global scheduler responsible for load balancing, KV
     Cache management, and resource allocation.  Most frameworks adopt
     vLLM[vLLM]'s paged KV Cache mechanism, inspired by OS virtual memory
     management.  This approach stores KV Cache into non-contiguous
     physical blocks across nodes and uses a scheduler to map logical
     blocks to physical memory.  Additionally, prefix-sharing strategies
     are employed to reuse KV Cache for prompts with identical prefixes,
     reducing redundant computations.  Remote KV Cache replication across
     nodes is also required to reduce duplicated computing of KV Cache of
     same tokens.  The architecture is shown in Figure 2.
















  Liu & Guo                  Expires 7 May 2026                   [Page 4]


  Internet-Draft             Network Management              November 2025


                               Request1/Prompt1
                               Request2/Prompt2
                                       |
                                       |
                          +------------v------------+
                          |                         |
                          |  Scheduler/Controller   |
                Request1  |                         | Request2
              +-----------+  *********************  +----------+
              |           |  *KV Cache Management*  |          |
              |           |  *  Load Balancing   *  |          |
              |           |  *     ... ...       *  |          |
              |           |  *********************  |          |
              |           +-------------------------+          |
              |                                                |
              |                                                |
              |                                                |
         +----v-----+  Remote    +----------+   Remote    +----v-----+
         |  Model   |KVCache copy|  Model   | KVCache copy|  Model   |
         |Instance 1<----------->|Instance 2|<------------>Instance 3|
         +----------+            +----------+             +----------+

                        Figure 2: PD Fusion Architecture

  2.2.  PD Disaggregation Architecture

     In PD Disaggregation, Prefill and Decoding are decoupled into
     separate instances to optimize hardware utilization.  After Prefill
     computes the full KV Cache for a prompt, the data is transferred to
     Decoding instances for text generation.  This architecture demands
     efficient coordination between Prefill and Decoding instances, as
     well as reliable high-speed data transmission.  The workflow is
     illustrated in Figure 3.


















  Liu & Guo                  Expires 7 May 2026                   [Page 5]


  Internet-Draft             Network Management              November 2025


                               Request1/Prompt1
                               Request2/Prompt2
                                       |
                                       |
                          +------------v------------+
                          |                         |
                          |  Scheduler/Controller   |
                Request1  |                         | Request2
              +-----------+  *********************  +----------+
              |           |  *KV Cache Management*  |          |
              |           |  *  Load Balancing   *  |          |
              |           |  *     ... ...       *  |          |
              |           |  *********************  |          |
              |           +-------------------------+          |
              |                                                |
              |                                                |
              |                                                |
         +----v-----+                                     +----v-----+
         |  Model   |                                     |  Model   |
         |          |       Remote KVCache copy           |          |
         | Prefill  <-------------------------------------> Prefill  |
         |Instance 1|                                     |Instance 2|
         +----+-----+                                     +----+-----+
              |KV Cache                                KV Cache|
              |Transfer                                Transfer|
              |                                                |
         +----v-----+                                     +----v-----+
         |  Model   |                                     |  Model   |
         |          |                                     |          |
         |Decoding  |                                     |Decoding  |
         |Instance 1|                                     |Instance 2|
         +----------+                                     +----------+

                    Figure 3: PD Disaggregation Architecture

  3.  Inference-related Metrics

     The ultimate goals of an inference system are to maximize system
     goodput which reflects the serving volume of user requests and
     minimize user-perceived latency at low cost.  For both PD Fusion and
     PD Disaggregation architectures, three kind of specific key metrics
     are defined as follows:

     Latency-related Metrics:

        TTFT (Time to First Token): The time taken by the Prefill phase to
        generate the first token.




  Liu & Guo                  Expires 7 May 2026                   [Page 6]


  Internet-Draft             Network Management              November 2025


        TBT (Time Between Tokens): The interval between consecutive token
        generations in the Decoding phase.

     Throughput-related Metrics:

        TPS (Tokens Per Second): The number of tokens generated per second
        by the inference system.

        RPS (Requests Per Second): The number of requests processed per
        second by the inference system.

     Cost-related Metrics:

        Cost Per Token: How much it costs for inference system to generate
        each token.

  4.  Research Question for Service-Oriented Inference Frameworks

     From the user's perspective, end-to-end inference latency generated
     during a complete inference process is a key metric directly
     impacting quality of service and user experience.  From the
     perspective of llm deployment, the most critical challenge is how to
     deploy llm services at the lowest possible cost while meeting basic
     user requirements for inference latency, minimizing idle rates of
     computational resources, and ensuring systems operate at near-
     saturation levels.  Industry commonly adopts decoupled architectures
     (e.g., PD Disaggregation) or other decoupled structures to deploy and
     independently optimize distinct model components on heterogeneous
     clusters.  However, maximizing system throughput and reducing
     inference deployment costs inevitably sacrifices user end-to-end
     inference latency.  In other words, latency, throughput, and cost
     form an impossible trinity.  Synthesizing the needs and objectives of
     both users and service providers, the primary challenge for inference
     systems is: How to maximize system throughput and minimize deployment
     costs while adhering to Service Level Objective (SLO) constraints for
     inference services?

  5.  Challenges for Service-Oriented Inference Frameworks

     To address the above key issue, current industry practices (e.g.,
     Mooncake[Mooncake], DeepSeek, vLLM[vLLM], SGLang[SGLang]) employ
     methods such as KV Cache prefix matching, PD Disaggregation
     deployment optimization, efficient KV Cache memory management, and
     flexible load balancing scheduling to enhance resource utilization.
     These approaches also aim to leverage fragmented, low-cost and idle
     resources, thereby increasing user throughput and reducing inference
     service deployment costs.  However, this approach faces several
     significant challenges:



  Liu & Guo                  Expires 7 May 2026                   [Page 7]


  Internet-Draft             Network Management              November 2025


  5.1.  Challenge 1

     Whether using PD Disaggregation or PD Fusion deployment, the industry
     widely adopts KV Cache prefix matching for large-scale inference
     deployment optimization.  This technique aims to reduce redundant
     computation and storage of prefix KV Cache across different inference
     requests, thereby saving computational resources and ultimately
     improving system throughput while lowering deployment costs.  How to
     systematically manage these KV Cache prefixes within the inference
     network—including storage mechanisms, placement strategies,
     scheduling schemes, and replacement policies—remains a critical and
     urgent challenge.

  5.2.  Challenge 2

     To optimize the trade-offs between inference deployment cost, service
     performance, and system throughput, implementing more efficient
     resource- and KV Cache-aware mechanisms within the inference
     network's management and control plane, alongside hardware- and
     network-aware load balancing scheduling, presents an extremely
     crucial challenge.

  5.3.  Challenge 3

     If a PD Disaggregation deployment approach is adopted, ensuring
     efficient transmission of KV Cache between the separated,
     heterogeneous Prefilling and Decoding clusters becomes a vital
     challenge.  This transmission must be designed to either avoid
     impacting end-to-end inference latency entirely or minimize its
     impact as much as possible.

  6.  Network Management Requirements for Service-Oriented Inference
      Frameworks

     To achieve large-scale LLM service deployment, frameworks MUST meet
     the following requirements in both control plane and data plane.

  6.1.  Efficient Load Balancing

     In large-scale inference systems, the efficient scheduling and
     routing of inference requests is a critical requirement.  It
     primarily focuses on two aspects: first, the request scheduling must
     be KV Cache-aware, striving to match cached KV data to minimize
     redundant computations and storage; second, the request scheduling
     must account for hardware load and network conditions, ensuring
     sufficient resources to process inference requests as efficiently as
     possible.




  Liu & Guo                  Expires 7 May 2026                   [Page 8]


  Internet-Draft             Network Management              November 2025


  6.2.  KV Cache Management

     In inference architecture systems, storage capacity is typically
     directly correlated with service concurrency.  In large-scale
     inference service deployment scenarios, distributed storage systems
     are often employed to manage KV Cache.  Additionally, multi-level
     storage structures within single nodes, such as HBM-CPU DRAM-SSD, are
     commonly used to cache KV data.  Beyond the placement mechanisms of
     KV Cache in hierarchical distributed storage systems, the disparity
     in communication bandwidth across cards/nodes and different storage
     tiers has become a bottleneck restricting the efficient utilization
     of KV Cache.  How to balance these communication bandwidths as much
     as possible constitutes another critical issue that needs to be
     addressed.

  6.3.  KV Cache Transmission

     In large-scale inference service deployment scenarios, the separation
     of Prefill and Decode (PD) phases serves as an effective approach to
     enhance system concurrency and reduce deployment costs.  This PD
     separation involves substantial KV Cache data transmission between
     Prefill instances and Decode instances, while demanding extremely
     high real-time requirements for request servicing.  Ensuring real-
     time KV transmission between PD instances represents another
     critically important area that requires focused attention.

  7.  Security Considerations

     TBD.

  8.  IANA Considerations

     TBD.

  9.  References

  9.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

  9.2.  Informative References

     [Mooncake] Qin, R., "Mooncake: A KVCache-centric Disaggregated
                Architecture for LLM Serving", 9 July 2024.




  Liu & Guo                  Expires 7 May 2026                   [Page 9]


  Internet-Draft             Network Management              November 2025


     [SGLang]   Zheng, L., "SGLang: Efficient Execution of Structured
                Language Model Programs", 6 June 2024.

     [vLLM]     Kwon, W., "Efficient Memory Management for Large Language
                Model Serving with PagedAttention", 2023.

  Authors' Addresses

     Chang Liu
     China Mobile
     Beijing
     100053
     China
     Email: liuchangjc@chinamobile.com


     Chuyi Guo
     China Mobile
     Beijing
     100053
     China
     Email: guochuyi@chinamobile.com





























  Liu & Guo                  Expires 7 May 2026                  [Page 10]


name: draft-gaikwad-llm-benchmarking-methodology
rev: 00
title: Benchmarking Methodology for Large Language Model Serving
time: 2026-01-20T15:05:40Z
group_uri: /api/v1/group/group/1027/
expires: 2026-07-24T15:05:40Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-08T17:35:45.883594+00:00
source: ietf
source_id: 153278
source_url: https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-methodology/
doc_status: active
abstract:
  This document defines benchmarking methodologies for Large Language
     Model (LLM) inference serving systems.  It provides test procedures,
     setup parameters, measurement specifications, and reporting formats
     for evaluating latency, throughput, scheduling, and resource
     management characteristics.  This document is a companion to
     "Benchmarking Terminology for Large Language Model Serving" and
     SHOULD be consulted alongside that terminology document.

full_text:
  Network Working Group                                         M. Gaikwad
  Internet-Draft                                    Independent Researcher
  Intended status: Informational                              January 2026
  Expires: 24 July 2026


         Benchmarking Methodology for Large Language Model Serving
               draft-gaikwad-llm-benchmarking-methodology-00

  Abstract

     This document defines benchmarking methodologies for Large Language
     Model (LLM) inference serving systems.  It provides test procedures,
     setup parameters, measurement specifications, and reporting formats
     for evaluating latency, throughput, scheduling, and resource
     management characteristics.  This document is a companion to
     "Benchmarking Terminology for Large Language Model Serving" and
     SHOULD be consulted alongside that terminology document.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 5 July 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.











  Gaikwad                   Expires 24 July 2026                  [Page 1]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   5
     2.  Requirements Language . . . . . . . . . . . . . . . . . . . .   6
     3.  Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . .   6
     4.  Test Setup  . . . . . . . . . . . . . . . . . . . . . . . . .   7
       4.1.  System Under Test Configurations  . . . . . . . . . . . .   7
         4.1.1.  Model Engine Configuration  . . . . . . . . . . . . .   7
         4.1.2.  Application Gateway Configuration . . . . . . . . . .   8
         4.1.3.  Compound System Configuration . . . . . . . . . . . .   9
       4.2.  Load Generator Requirements . . . . . . . . . . . . . . .   9
         4.2.1.  Timing Resolution . . . . . . . . . . . . . . . . . .   9
         4.2.2.  Streaming Support . . . . . . . . . . . . . . . . . .  10
         4.2.3.  Open-Loop Load Generation . . . . . . . . . . . . . .  10
         4.2.4.  Closed-Loop Load Generation . . . . . . . . . . . . .  10
         4.2.5.  Request Isolation . . . . . . . . . . . . . . . . . .  10
         4.2.6.  Output Recording  . . . . . . . . . . . . . . . . . .  10
       4.3.  Reference Workloads . . . . . . . . . . . . . . . . . . .  10
         4.3.1.  Workload Parameters . . . . . . . . . . . . . . . . .  11
         4.3.2.  Standard Workloads  . . . . . . . . . . . . . . . . .  11
         4.3.3.  Workload Reproducibility  . . . . . . . . . . . . . .  13
       4.4.  Tokenization  . . . . . . . . . . . . . . . . . . . . . .  13
         4.4.1.  Tokenizer Specification . . . . . . . . . . . . . . .  13
         4.4.2.  Token Counting Method . . . . . . . . . . . . . . . .  13
         4.4.3.  Special Token Handling  . . . . . . . . . . . . . . .  14
       4.5.  Warm-up Procedures  . . . . . . . . . . . . . . . . . . .  14
         4.5.1.  Warm-up Requirements  . . . . . . . . . . . . . . . .  14
         4.5.2.  Warm-up Verification  . . . . . . . . . . . . . . . .  14
         4.5.3.  Cold Start Measurement  . . . . . . . . . . . . . . .  14
       4.6.  Streaming Protocol  . . . . . . . . . . . . . . . . . . .  15
         4.6.1.  Supported Protocols . . . . . . . . . . . . . . . . .  15
         4.6.2.  Token Chunking  . . . . . . . . . . . . . . . . . . .  15
         4.6.3.  ITL Calculation with Chunked Delivery . . . . . . . .  15
       4.7.  Clock Synchronization . . . . . . . . . . . . . . . . . .  16
         4.7.1.  Single-Machine Testing  . . . . . . . . . . . . . . .  16
         4.7.2.  Distributed Testing . . . . . . . . . . . . . . . . .  16
         4.7.3.  Network Latency Measurement . . . . . . . . . . . . .  16
         4.7.4.  Timestamp Format  . . . . . . . . . . . . . . . . . .  16
       4.8.  Safety and Guardrail Configuration  . . . . . . . . . . .  16



  Gaikwad                   Expires 24 July 2026                  [Page 2]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


         4.8.1.  Guardrail Disclosure  . . . . . . . . . . . . . . . .  17
         4.8.2.  Production-Representative Testing . . . . . . . . . .  17
     5.  Benchmarking Tests  . . . . . . . . . . . . . . . . . . . . .  17
       5.1.  Time to First Token . . . . . . . . . . . . . . . . . . .  17
         5.1.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  17
         5.1.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  17
         5.1.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  18
         5.1.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  19
         5.1.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  20
       5.2.  Output Token Throughput . . . . . . . . . . . . . . . . .  22
         5.2.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  22
         5.2.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  22
         5.2.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  23
         5.2.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  24
         5.2.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  24
       5.3.  Throughput-Latency Tradeoff . . . . . . . . . . . . . . .  25
         5.3.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  25
         5.3.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  25
         5.3.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  25
         5.3.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  26
         5.3.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  27
       5.4.  Inter-Token Latency Distribution  . . . . . . . . . . . .  27
         5.4.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  27
         5.4.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  27
         5.4.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  28
         5.4.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  28
         5.4.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  29
       5.5.  Concurrent Request Capacity . . . . . . . . . . . . . . .  29
         5.5.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  29
         5.5.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  29
         5.5.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  30
         5.5.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  30
         5.5.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  31
       5.6.  Scheduling Fairness . . . . . . . . . . . . . . . . . . .  31
         5.6.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  31
         5.6.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  31
         5.6.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  32
         5.6.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  32
         5.6.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  32
       5.7.  Prefix Cache Effectiveness  . . . . . . . . . . . . . . .  33
         5.7.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  33
         5.7.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  33
         5.7.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  33
         5.7.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  34
         5.7.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  34
       5.8.  Memory Pressure Behavior  . . . . . . . . . . . . . . . .  34
         5.8.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  34
         5.8.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  34



  Gaikwad                   Expires 24 July 2026                  [Page 3]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


         5.8.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  34
         5.8.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  35
         5.8.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  35
       5.9.  Long Context Scaling  . . . . . . . . . . . . . . . . . .  35
         5.9.1.  Objective . . . . . . . . . . . . . . . . . . . . . .  35
         5.9.2.  Setup Parameters  . . . . . . . . . . . . . . . . . .  36
         5.9.3.  Procedure . . . . . . . . . . . . . . . . . . . . . .  36
         5.9.4.  Measurements  . . . . . . . . . . . . . . . . . . . .  36
         5.9.5.  Reporting Format  . . . . . . . . . . . . . . . . . .  36
       5.10. Guardrail Overhead  . . . . . . . . . . . . . . . . . . .  37
         5.10.1.  Objective  . . . . . . . . . . . . . . . . . . . . .  37
         5.10.2.  Setup Parameters . . . . . . . . . . . . . . . . . .  37
         5.10.3.  Procedure  . . . . . . . . . . . . . . . . . . . . .  37
         5.10.4.  Measurements . . . . . . . . . . . . . . . . . . . .  38
         5.10.5.  Reporting Format . . . . . . . . . . . . . . . . . .  38
     6.  Multi-System Comparison Guidelines  . . . . . . . . . . . . .  38
       6.1.  Equivalence Requirements  . . . . . . . . . . . . . . . .  39
       6.2.  Normalization . . . . . . . . . . . . . . . . . . . . . .  39
       6.3.  Statistical Significance  . . . . . . . . . . . . . . . .  39
       6.4.  Fair Comparison Checklist . . . . . . . . . . . . . . . .  39
     7.  Security Considerations . . . . . . . . . . . . . . . . . . .  40
       7.1.  Side-Channel Risks  . . . . . . . . . . . . . . . . . . .  40
       7.2.  Benchmark Gaming  . . . . . . . . . . . . . . . . . . . .  40
       7.3.  Adversarial Workloads . . . . . . . . . . . . . . . . . .  40
       7.4.  Resource Exhaustion . . . . . . . . . . . . . . . . . . .  40
     8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  41
       8.1.  Normative References  . . . . . . . . . . . . . . . . . .  41
       8.2.  Informative References  . . . . . . . . . . . . . . . . .  41
     Appendix A.  Reference Workload Specifications  . . . . . . . . .  42
       A.1.  Synthetic-Uniform Workload  . . . . . . . . . . . . . . .  42
         A.1.1.  Input Specification . . . . . . . . . . . . . . . . .  42
         A.1.2.  Output Specification  . . . . . . . . . . . . . . . .  42
         A.1.3.  Other Parameters  . . . . . . . . . . . . . . . . . .  42
         A.1.4.  Generation Method . . . . . . . . . . . . . . . . . .  42
       A.2.  Synthetic-Skewed Workload . . . . . . . . . . . . . . . .  43
         A.2.1.  Input Specification . . . . . . . . . . . . . . . . .  43
         A.2.2.  Output Specification  . . . . . . . . . . . . . . . .  43
       A.3.  Conversation Workload . . . . . . . . . . . . . . . . . .  44
         A.3.1.  Data Source . . . . . . . . . . . . . . . . . . . . .  44
         A.3.2.  Length Statistics (Reference) . . . . . . . . . . . .  44
       A.4.  Code Completion Workload  . . . . . . . . . . . . . . . .  44
         A.4.1.  Data Source . . . . . . . . . . . . . . . . . . . . .  44
         A.4.2.  Prefix Sharing Pattern  . . . . . . . . . . . . . . .  44
       A.5.  Long Context Workload . . . . . . . . . . . . . . . . . .  45
         A.5.1.  Input Specification . . . . . . . . . . . . . . . . .  45
         A.5.2.  Output Specification  . . . . . . . . . . . . . . . .  45
     Appendix B.  Timing Measurement Reference . . . . . . . . . . . .  45
       B.1.  TTFT Measurement Points . . . . . . . . . . . . . . . . .  45



  Gaikwad                   Expires 24 July 2026                  [Page 4]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


         B.1.1.  HTTP/SSE Measurement  . . . . . . . . . . . . . . . .  45
         B.1.2.  gRPC Streaming Measurement  . . . . . . . . . . . . .  45
         B.1.3.  Server-Side Measurement . . . . . . . . . . . . . . .  46
       B.2.  ITL Measurement with SSE  . . . . . . . . . . . . . . . .  46
         B.2.1.  Recommended Approach  . . . . . . . . . . . . . . . .  46
       B.3.  Clock Synchronization Methods . . . . . . . . . . . . . .  46
         B.3.1.  NTP Synchronization . . . . . . . . . . . . . . . . .  46
         B.3.2.  PTP Synchronization . . . . . . . . . . . . . . . . .  46
         B.3.3.  Single-Machine Alternative  . . . . . . . . . . . . .  47
     Appendix C.  Reporting Templates  . . . . . . . . . . . . . . . .  47
       C.1.  Minimum Viable Report . . . . . . . . . . . . . . . . . .  47
       C.2.  Full Report Template  . . . . . . . . . . . . . . . . . .  48
     Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  48
     Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  48

  1.  Introduction

     This document provides benchmarking methodologies for Large Language
     Model inference serving systems.  It defines test procedures,
     measurement specifications, and reporting formats that enable
     meaningful performance comparison.

     A companion document, "Benchmarking Terminology for Large Language
     Model Serving" [LLM-TERMS], defines the metrics referenced in this
     methodology.  That terminology document SHOULD be consulted before
     attempting to make use of this document.

     LLM serving systems present unique benchmarking challenges:

     Streaming responses:  Output tokens arrive incrementally over seconds
        or minutes, requiring timing measurements at multiple points
        within a single request.

     Phase separation:  The prefill phase (processing input) and decode
        phase (generating output) have distinct computational profiles and
        optimization targets.

     Memory-bound decoding:  The decode phase is limited by memory
        bandwidth rather than compute, creating different bottlenecks than
        traditional neural network inference.

     Dynamic batching:  Continuous batching systems interleave requests,
        causing per-request performance to depend on concurrent load.

     Context-dependent performance:  Request latency varies with input
        length, output length, and cache state, making workload
        specification critical.




  Gaikwad                   Expires 24 July 2026                  [Page 5]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     These characteristics require methodology beyond traditional
     throughput and latency measurement.  This document addresses these
     challenges by specifying:

     *  Test configurations for different system boundaries

     *  Reference workloads with defined characteristics

     *  Measurement procedures for streaming responses

     *  Statistical requirements for reliable percentile estimation

     *  Reporting formats enabling meaningful comparison

     This document does not specify acceptance thresholds or recommend
     particular systems.  It provides methodology for fair comparison.

  2.  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in BCP
     14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

     An implementation is not compliant if it fails to satisfy one or more
     of the MUST requirements for a given test.  An implementation that
     satisfies all the MUST and all the SHOULD requirements for a test is
     said to be "unconditionally compliant" for that test; one that
     satisfies all the MUST requirements but not all the SHOULD
     requirements is said to be "conditionally compliant."

  3.  Scope

     This document covers benchmarking methodology for transformer-based
     autoregressive language models deployed as network services.  The
     methodology applies to:

     *  Inference engines executing model forward passes

     *  Application gateways providing API endpoints

     *  Compound systems with retrieval or tool execution

     The following are out of scope:

     *  Model training or fine-tuning performance




  Gaikwad                   Expires 24 July 2026                  [Page 6]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     *  Model quality or accuracy evaluation

     *  Non-autoregressive models (diffusion, encoder-only)

     *  Edge deployment or on-device inference

     *  Specific vendor implementations or products

  4.  Test Setup

  4.1.  System Under Test Configurations

     The System Under Test (SUT) boundary MUST be declared before
     benchmarking.  This document defines three standard configurations.

  4.1.1.  Model Engine Configuration

     The Model Engine configuration measures raw inference capability.

                     +------------------+
                     |   Load Generator |
                     +--------+---------+
                              |
                   Internal API (gRPC/HTTP)
                              |
                     +--------v---------+
                     |   Model Engine   |
                     |  (SUT Boundary)  |
                     +------------------+

                      Figure 1: Model Engine Configuration

     Included components:

     *  Model weights and inference runtime

     *  Batching and scheduling logic

     *  KV cache management

     *  Tensor operations and kernels

     Excluded components:

     *  External network transport

     *  Authentication and authorization




  Gaikwad                   Expires 24 July 2026                  [Page 7]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     *  Rate limiting

     *  Input/output safety filtering

     *  Load balancing

     This configuration is appropriate for comparing inference engines
     (vLLM, TensorRT-LLM, SGLang) independent of deployment stack.

  4.1.2.  Application Gateway Configuration

     The Application Gateway configuration measures user-observable API
     performance.

                     +------------------+
                     |   Load Generator |
                     +--------+---------+
                              |
                   External API (HTTPS)
                              |
                     +--------v---------+
                     | Application GW   |
                     |  (SUT Boundary)  |
                     |  +------------+  |
                     |  |   Engine   |  |
                     |  +------------+  |
                     +------------------+

                  Figure 2: Application Gateway Configuration

     Included components (in addition to Model Engine):

     *  TLS termination

     *  Authentication and session management

     *  Rate limiting and quota enforcement

     *  Input validation and output filtering

     *  Safety guardrails

     This configuration is appropriate for comparing API providers or
     evaluating production deployment performance.







  Gaikwad                   Expires 24 July 2026                  [Page 8]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.1.3.  Compound System Configuration

     The Compound System configuration measures end-to-end task completion
     for agentic or retrieval-augmented workloads.

                     +------------------+
                     |   Task Driver    |
                     +--------+---------+
                              |
                     +--------v---------+
                     |  Compound System |
                     |  (SUT Boundary)  |
                     |  +------------+  |
                     |  | Retrieval  |  |
                     |  +------------+  |
                     |  +------------+  |
                     |  |   Tools    |  |
                     |  +------------+  |
                     |  +------------+  |
                     |  |  Gateway   |  |
                     |  +------------+  |
                     +------------------+

                    Figure 3: Compound System Configuration

     Included components (in addition to Application Gateway):

     *  Retrieval pipeline (embedding, vector search, reranking)

     *  Tool execution environment

     *  Orchestration logic

     *  Multi-turn conversation state

     This configuration is appropriate for evaluating RAG systems or
     agentic applications.

  4.2.  Load Generator Requirements

     The load generator produces requests and measures responses.  It MUST
     satisfy the following requirements.

  4.2.1.  Timing Resolution

     The load generator MUST measure time with resolution of 1 millisecond
     or better.  Microsecond resolution is RECOMMENDED for ITL
     measurement.



  Gaikwad                   Expires 24 July 2026                  [Page 9]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.2.2.  Streaming Support

     The load generator MUST support streaming response protocols (SSE,
     WebSocket, or gRPC streaming).  It MUST record the arrival time of
     each token or chunk, not only the complete response.

  4.2.3.  Open-Loop Load Generation

     The load generator MUST support open-loop load generation where
     request arrival times are determined by a specified distribution
     independent of response times.  Poisson arrivals MUST be supported.
     Uniform and bursty arrival patterns are RECOMMENDED.

  4.2.4.  Closed-Loop Load Generation

     The load generator MUST support closed-loop load generation where a
     fixed number of concurrent requests are maintained.  When a request
     completes, a new request is immediately submitted.

  4.2.5.  Request Isolation

     The load generator MUST NOT allow slow responses to delay the
     submission of subsequent requests in open-loop mode.  Asynchronous or
     multi-threaded implementation is REQUIRED.

  4.2.6.  Output Recording

     The load generator MUST record for each request:

     *  Request submission timestamp

     *  First token arrival timestamp

     *  Each subsequent token arrival timestamp

     *  Final token arrival timestamp

     *  Total input token count

     *  Total output token count

     *  Request success/failure status

  4.3.  Reference Workloads

     Workload specification is critical for reproducible benchmarking.
     This document defines reference workloads with fixed characteristics.
     Testers MAY use custom workloads but MUST fully specify them.



  Gaikwad                   Expires 24 July 2026                 [Page 10]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.3.1.  Workload Parameters

     Each workload MUST specify:

     Input length distribution:  Distribution type (fixed, uniform,
        normal, empirical), parameters (mean, std, min, max, or
        histogram), and unit (tokens using specified tokenizer).

     Output length distribution:  Distribution type (fixed, uniform,
        normal, empirical), parameters (mean, std, min, max, or
        histogram), control method (max_tokens parameter, stop sequence,
        or both), and unit (tokens using specified tokenizer).

     Content characteristics:  Domain (general, code, conversation,
        instruction), language (English, multilingual, code languages),
        and system prompt presence and typical length.

     Prefix sharing:  Fraction of requests sharing common prefix and
        shared prefix length distribution.

  4.3.2.  Standard Workloads

     This document defines five standard workloads.  Full specifications
     appear in Appendix A.

  4.3.2.1.  Synthetic-Uniform

     Purpose: Baseline comparison with controlled variability

     *  Input length: Uniform(128, 512) tokens

     *  Output length: Uniform(64, 256) tokens

     *  Content: Random token sequences (no semantic meaning)

     *  Prefix sharing: None

     This workload isolates inference performance from content effects.
     It is REQUIRED for Model Engine benchmarking.

  4.3.2.2.  Synthetic-Skewed

     Purpose: Test behavior under realistic length variation

     *  Input length: Log-normal(mu=5.5, sigma=1.0) tokens, capped at 4096

     *  Output length: Log-normal(mu=4.5, sigma=1.2) tokens, capped at
        2048



  Gaikwad                   Expires 24 July 2026                 [Page 11]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     *  Content: Random token sequences

     *  Prefix sharing: None

     This workload tests scheduling fairness with high length variance.

  4.3.2.3.  Conversation

     Purpose: Simulate interactive chat workloads

     *  Input length: Empirical distribution from ShareGPT dataset

     *  Output length: Empirical distribution from ShareGPT dataset

     *  Content: Natural language conversation

     *  Prefix sharing: 50% share 200-token system prompt

     This workload is RECOMMENDED for Application Gateway benchmarking.

  4.3.2.4.  Code Completion

     Purpose: Simulate coding assistant workloads

     *  Input length: Empirical from code completion datasets

     *  Output length: Log-normal(mu=4.0, sigma=1.5) tokens

     *  Content: Source code in Python, JavaScript, TypeScript

     *  Prefix sharing: 80% share repository context prefix

     This workload tests prefix caching effectiveness.

  4.3.2.5.  Long Context

     Purpose: Test long-context behavior

     *  Input length: Uniform(8192, 32768) tokens

     *  Output length: Fixed at 256 tokens

     *  Content: Document + question format

     *  Prefix sharing: None

     This workload is REQUIRED for Long Context Scaling tests.




  Gaikwad                   Expires 24 July 2026                 [Page 12]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.3.3.  Workload Reproducibility

     For reproducible benchmarking:

     *  Testers MUST use deterministic random seeds for workload
        generation.  The seed MUST be reported.

     *  Testers SHOULD publish the exact request sequences used, or
        provide generation code with fixed seeds.

     *  When using dataset-derived workloads (ShareGPT, HumanEval),
        testers MUST specify the dataset version, subset selection method,
        and any preprocessing applied.

  4.4.  Tokenization

     Token counts depend on the tokenizer.  Different tokenizers produce
     different counts for identical text, making cross-system comparison
     challenging.

  4.4.1.  Tokenizer Specification

     The test report MUST specify:

     *  Tokenizer name and version (e.g., "cl100k_base", "Llama-3
        tokenizer")

     *  Vocabulary size

     *  Source (Hugging Face model ID, tiktoken name, or custom)

  4.4.2.  Token Counting Method

     For cross-system comparison where systems use different tokenizers:

     Option A - Native tokenizer:  Count tokens using each system's native
        tokenizer.  Report results separately with tokenizer identified.
        This method reflects actual system behavior but complicates
        comparison.

     Option B - Reference tokenizer:  Count tokens using a declared
        reference tokenizer for all systems.  This enables direct
        comparison but may not reflect actual system token counts.

     The test report MUST declare which option is used.  Option B with
     cl100k_base (GPT-4 tokenizer) as reference is RECOMMENDED for cross-
     system comparison.




  Gaikwad                   Expires 24 July 2026                 [Page 13]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.4.3.  Special Token Handling

     The test report MUST specify handling of:

     *  BOS/EOS tokens (included or excluded from counts)

     *  System prompt tokens (counted separately or included)

     *  Tool/function call formatting tokens

  4.5.  Warm-up Procedures

     LLM serving systems require warm-up before reaching steady-state
     performance.  Warm-up effects include JIT compilation, memory
     allocator initialization, prefix cache population, and batch size
     ramp-up.

  4.5.1.  Warm-up Requirements

     Before measurement begins, testers MUST:

     1.  Load the model fully into accelerator memory

     2.  Process at least 100 requests or 10,000 output tokens, whichever
         is greater

     3.  Wait for request queue to drain completely

     4.  If prefix caching is enabled and being tested, populate the cache
         with representative prefixes

  4.5.2.  Warm-up Verification

     Testers SHOULD verify warm-up completion by:

     1.  Measuring latency for a probe request before and after warm-up

     2.  Confirming latency stabilization (less than 10% variation across
         consecutive probe requests)

  4.5.3.  Cold Start Measurement

     When cold start performance is being measured (Model Load Time, Cold
     Start Latency), warm-up MUST be skipped.  The test report MUST
     clearly indicate cold start measurement.






  Gaikwad                   Expires 24 July 2026                 [Page 14]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.6.  Streaming Protocol

     LLM serving systems deliver tokens via streaming protocols.  The
     choice of protocol affects timing measurement.

  4.6.1.  Supported Protocols

     This methodology supports:

     Server-Sent Events (SSE):  HTTP-based streaming.  Each event contains
        one or more tokens.  RECOMMENDED for Application Gateway testing.

     WebSocket:  Bidirectional streaming.  Each message contains one or
        more tokens.

     gRPC streaming:  Binary streaming protocol.  Each message contains
        one or more tokens.  RECOMMENDED for Model Engine testing.

  4.6.2.  Token Chunking

     Streaming protocols may deliver multiple tokens per chunk due to
     batching or network buffering.  The test report MUST specify:

     *  Protocol used

     *  Whether each chunk contains exactly one token or potentially
        multiple tokens

     *  How multi-token chunks are handled for ITL calculation

  4.6.3.  ITL Calculation with Chunked Delivery

     When chunks contain multiple tokens:

     Option A - Chunk timing:  Measure inter-chunk latency.  Report as
        "Time Between Chunks" rather than ITL.  Note chunk size
        distribution.

     Option B - Distributed timing:  Distribute chunk arrival time across
        tokens.  If a chunk with N tokens arrives at time T, assign
        arrival time T to all N tokens.  This understates ITL variance.

     Option C - Server-side timing:  Use server-reported per-token
        timestamps if available.  This measures ITL independent of network
        effects.

     The test report MUST declare which option is used.  Option C is
     RECOMMENDED when available.



  Gaikwad                   Expires 24 July 2026                 [Page 15]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.7.  Clock Synchronization

     Accurate timing requires synchronized clocks between load generator
     and SUT, and between distributed SUT components.

  4.7.1.  Single-Machine Testing

     When load generator and SUT run on the same machine, clock
     synchronization is inherent.  This configuration is RECOMMENDED for
     Model Engine testing.

  4.7.2.  Distributed Testing

     When load generator and SUT are on different machines:

     *  NTP synchronization MUST achieve accuracy of 10ms or better

     *  PTP synchronization SHOULD be used when sub-millisecond accuracy
        is required

     *  The test report MUST state the synchronization method and
        estimated accuracy

  4.7.3.  Network Latency Measurement

     For Application Gateway testing where network latency is significant:

     *  Testers SHOULD measure and report network RTT separately

     *  Testers MAY subtract estimated network latency from TTFT to
        isolate server-side processing time

     *  Any latency adjustment MUST be documented in the test report

  4.7.4.  Timestamp Format

     All timestamps MUST be recorded in a format with at least millisecond
     precision.  ISO 8601 with milliseconds (YYYY-MM-DDTHH:MM:SS.sssZ) or
     Unix epoch with milliseconds is RECOMMENDED.

  4.8.  Safety and Guardrail Configuration

     Production LLM deployments include safety systems that affect
     performance.  Benchmarking MUST account for these systems.







  Gaikwad                   Expires 24 July 2026                 [Page 16]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  4.8.1.  Guardrail Disclosure

     The test report MUST disclose:

     *  Whether input content filtering is enabled

     *  Whether output content filtering is enabled

     *  Names of safety systems if known (e.g., "Llama Guard")

     *  Whether any requests were refused during testing

  4.8.2.  Production-Representative Testing

     For Application Gateway benchmarking intended to represent production
     performance:

     *  Safety systems SHOULD be enabled in their default configuration

     *  The test report MUST note if safety systems are disabled

     *  Testers SHOULD run comparative tests with safety enabled and
        disabled to quantify overhead

  5.  Benchmarking Tests

     This section defines benchmarking tests.  Each test includes:
     objective, setup parameters, procedure, measurements, and reporting
     format.

  5.1.  Time to First Token

  5.1.1.  Objective

     To determine the latency from request submission to first token
     receipt under varying load conditions.  TTFT measures perceived
     responsiveness for interactive applications.

  5.1.2.  Setup Parameters

     The following parameters MUST be defined:

  5.1.2.1.  Workload Parameters

     Workload:  One of the standard workloads (Section 4.3.2) or a fully
        specified custom workload.

     Request count:  Total number of requests to execute.  MUST be at



  Gaikwad                   Expires 24 July 2026                 [Page 17]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


        least 1000 for P99 measurement, 10000 for P99.9.

  5.1.2.2.  Load Parameters

     Load model:  Open-loop or closed-loop.

     For open-loop:

     Arrival rate:  Requests per second.

     Arrival distribution:  Poisson (REQUIRED), uniform, or bursty.

     For closed-loop:

     Concurrency:  Number of concurrent requests maintained.

  5.1.2.3.  System Parameters

     SUT configuration:  Model Engine, Application Gateway, or Compound
        System.

     Model identifier:  Model name, version, and quantization if
        applicable.

     Hardware:  Accelerator type, count, and memory.

     Prefix caching:  Enabled or disabled.

  5.1.3.  Procedure

     1.  Configure the SUT with specified parameters.

     2.  Complete warm-up procedure (Section 4.5).

     3.  Begin load generation at the specified arrival rate or
         concurrency.

     4.  For each request:

         a.  Record submission timestamp (T_submit)

         b.  Record first token arrival timestamp (T_first)

         c.  Calculate TTFT = T_first - T_submit

         d.  Record input token count

     5.  Continue until request count is reached.



  Gaikwad                   Expires 24 July 2026                 [Page 18]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     6.  Compute distribution statistics.

  5.1.3.1.  First Token Definition

     The first token is defined as the first content token received,
     excluding:

     *  Empty tokens or whitespace-only tokens

     *  Protocol overhead (SSE event markers, JSON framing)

     *  Metadata tokens (token IDs, logprobs if requested separately)

     If the system emits non-content tokens before content, the test
     report MUST note this and specify whether TTFT measures time to any
     token or time to first content token.

  5.1.4.  Measurements

  5.1.4.1.  Primary Measurements

     TTFT Percentiles:  P50, P90, P95, P99, and P99.9 of TTFT
        distribution.  All percentiles MUST be reported.

     TTFT Mean:  Arithmetic mean of TTFT values.

     TTFT Minimum:  Smallest TTFT observed.

     TTFT Maximum:  Largest TTFT observed.

  5.1.4.2.  Conditional Measurements

     TTFT by input length:  When workload has variable input length,
        report TTFT percentiles bucketed by input length ranges.
        RECOMMENDED buckets: [0-256), [256-512), [512-1024), [1024-2048),
        [2048-4096), [4096+) tokens.

     Queue wait time:  If measurable (server instrumentation), report the
        queue wait component of TTFT separately.

     Prefill latency:  If measurable, report the prefill computation
        component of TTFT separately.

  5.1.4.3.  Statistical Requirements

     For P99 accuracy within 10% relative error at 95% confidence, at
     least 1000 samples are required.  For P99.9, at least 10000 samples.
     The test report MUST state the sample count.



  Gaikwad                   Expires 24 July 2026                 [Page 19]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.1.5.  Reporting Format

     The test report MUST include:

  5.1.5.1.  Configuration Summary

     *  SUT configuration and boundary

     *  Model identifier and hardware

     *  Workload name or full specification

     *  Load model and parameters

     *  Request count and test duration

     *  Warm-up procedure followed

     *  Prefix caching state

     *  Guardrail configuration

  5.1.5.2.  Results Table

     The results SHOULD be reported in tabular format:


























  Gaikwad                   Expires 24 July 2026                 [Page 20]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


                            +============+=========+
                            | Metric     | Value   |
                            +============+=========+
                            | Requests   | 10000   |
                            +------------+---------+
                            | TTFT P50   | 127 ms  |
                            +------------+---------+
                            | TTFT P90   | 245 ms  |
                            +------------+---------+
                            | TTFT P95   | 312 ms  |
                            +------------+---------+
                            | TTFT P99   | 524 ms  |
                            +------------+---------+
                            | TTFT P99.9 | 891 ms  |
                            +------------+---------+
                            | TTFT Mean  | 156 ms  |
                            +------------+---------+
                            | TTFT Min   | 89 ms   |
                            +------------+---------+
                            | TTFT Max   | 1243 ms |
                            +------------+---------+

                             Table 1: TTFT Results
                                    Example

  5.1.5.3.  TTFT by Input Length

     If applicable:

               +==============+==========+==========+==========+
               | Input Tokens | P50 (ms) | P95 (ms) | P99 (ms) |
               +==============+==========+==========+==========+
               | 0-256        | 95       | 198      | 312      |
               +--------------+----------+----------+----------+
               | 256-512      | 142      | 287      | 445      |
               +--------------+----------+----------+----------+
               | 512-1024     | 198      | 412      | 623      |
               +--------------+----------+----------+----------+
               | 1024-2048    | 312      | 587      | 891      |
               +--------------+----------+----------+----------+
               | 2048+        | 523      | 912      | 1243     |
               +--------------+----------+----------+----------+

                     Table 2: TTFT by Input Length Example







  Gaikwad                   Expires 24 July 2026                 [Page 21]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.1.5.4.  Distribution Visualization

     Testers SHOULD include a histogram or CDF plot of the TTFT
     distribution.

  5.2.  Output Token Throughput

  5.2.1.  Objective

     To determine the maximum rate at which the SUT can generate output
     tokens while maintaining acceptable latency.  This test measures
     system capacity under load.

  5.2.2.  Setup Parameters

     The following parameters MUST be defined:

  5.2.2.1.  Workload Parameters

     Workload:  One of the standard workloads or fully specified custom
        workload.

     Test duration:  Minimum 60 seconds.  RECOMMENDED 300 seconds for
        stable measurement.

  5.2.2.2.  Load Parameters

     Load model:  Open-loop or closed-loop.

     For open-loop:

     Arrival rate range:  Minimum and maximum request rates to test.

     Rate increment:  Step size for iterative search.

     For closed-loop:

     Concurrency range:  Minimum and maximum concurrent requests.

     Concurrency increment:  Step size for iterative search.

  5.2.2.3.  Latency Constraint (Optional)

     TTFT SLO:  Maximum acceptable P99 TTFT.

     TPOT SLO:  Maximum acceptable P99 TPOT.





  Gaikwad                   Expires 24 July 2026                 [Page 22]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     When specified, throughput is measured as the maximum rate achieving
     these SLOs.

  5.2.3.  Procedure

     This test employs an iterative search to find maximum throughput.

     1.  Configure the SUT with specified parameters.

     2.  Complete warm-up procedure.

     3.  For each load level (arrival rate or concurrency):

         a.  Run load for the specified test duration.

         b.  Record all request timings.

         c.  Compute throughput as total output tokens divided by test
             duration.

         d.  Compute TTFT and TPOT percentiles.

         e.  If latency constraint specified, check SLO compliance.

     4.  Use binary search to find maximum throughput:

         a.  If no latency constraint: find load level where queue grows
             unboundedly (system saturation).

         b.  If latency constraint: find highest load level meeting SLO.

     5.  Report throughput at the maximum sustainable load level.

  5.2.3.1.  Saturation Detection

     System saturation is detected when:

     *  Queue depth grows continuously during test duration, OR

     *  Request completion rate is less than 90% of arrival rate, OR

     *  P99 latency exceeds 10x the P50 latency at lower load

  5.2.3.2.  Steady State Verification

     At each load level, verify steady state by:

     *  Confirming queue depth is stable (not growing)



  Gaikwad                   Expires 24 July 2026                 [Page 23]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     *  Confirming throughput is stable across test duration

     *  Excluding initial ramp-up period (first 10% of duration)

  5.2.4.  Measurements

  5.2.4.1.  Primary Measurements

     Maximum output token throughput:  Output tokens per second at maximum
        sustainable load.  Report with or without latency constraint as
        specified.

     Request throughput:  Requests completed per second at maximum load.

     Input token throughput:  Input tokens processed per second (measures
        prefill capacity).

  5.2.4.2.  Efficiency Measurements

     Tokens per GPU-second:  Output tokens per second divided by GPU
        count.  Enables comparison across different hardware
        configurations.

     Batch utilization:  If measurable, report average batch size divided
        by maximum batch size.

  5.2.4.3.  Latency at Maximum Throughput

     At the maximum sustainable load level, report:

     *  TTFT P50, P95, P99

     *  TPOT P50, P95, P99

     *  End-to-end latency P50, P95, P99

  5.2.5.  Reporting Format

  5.2.5.1.  Summary Results

                  +========================+===============+
                  | Metric                 | Value         |
                  +========================+===============+
                  | Max Output Throughput  | 2847 tok/s    |
                  +------------------------+---------------+
                  | Max Request Throughput | 18.2 req/s    |
                  +------------------------+---------------+
                  | Max Input Throughput   | 5123 tok/s    |



  Gaikwad                   Expires 24 July 2026                 [Page 24]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


                  +------------------------+---------------+
                  | Sustainable Load       | 20 req/s      |
                  +------------------------+---------------+
                  | Tokens per GPU-second  | 356 tok/s/GPU |
                  +------------------------+---------------+

                     Table 3: Throughput Summary Example

  5.2.5.2.  Latency at Maximum Throughput

                  +============+========+========+=========+
                  | Metric     | P50    | P95    | P99     |
                  +============+========+========+=========+
                  | TTFT       | 312 ms | 687 ms | 1124 ms |
                  +------------+--------+--------+---------+
                  | TPOT       | 42 ms  | 78 ms  | 134 ms  |
                  +------------+--------+--------+---------+
                  | End-to-End | 6.2 s  | 11.4 s | 18.7 s  |
                  +------------+--------+--------+---------+

                    Table 4: Latency at Maximum Throughput
                                   Example

  5.3.  Throughput-Latency Tradeoff

  5.3.1.  Objective

     To characterize the relationship between throughput and latency
     across the operating range of the SUT.  This test produces a
     throughput-latency curve revealing system behavior better than point
     measurements.

  5.3.2.  Setup Parameters

     Workload:  One of the standard workloads or fully specified custom
        workload.

     Test duration per point:  Minimum 60 seconds per load level.

     Load levels:  At least 10 load levels spanning from low load (10% of
        estimated capacity) to saturation.

     Load model:  Open-loop is REQUIRED for this test.  Closed-loop cannot
        reveal behavior beyond capacity.

  5.3.3.  Procedure





  Gaikwad                   Expires 24 July 2026                 [Page 25]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     1.  Estimate system capacity using a preliminary throughput test or
         published specifications.

     2.  Define load levels: 10%, 20%, 30%, ..., 100%, 110%, 120% of
         estimated capacity.

     3.  For each load level in ascending order:

         a.  Run load for specified duration.

         b.  Record all request timings.

         c.  Compute achieved throughput (may differ from offered load at
             saturation).

         d.  Compute latency percentiles.

     4.  Plot throughput vs latency curves.

  5.3.4.  Measurements

     For each load level, record:

     *  Offered load (request rate)

     *  Achieved throughput (output tokens per second)

     *  TTFT: P50, P95, P99

     *  TPOT: P50, P95, P99

     *  End-to-end latency: P50, P95, P99

     *  Request success rate

     *  Queue growth indicator (stable/growing)

     Derived metrics:

     Optimal operating point:  Load level achieving highest throughput
        while meeting specified SLO.

     Knee point:  Load level where P99 latency exceeds 2x the minimum P99
        latency observed.

     Saturation point:  Load level where achieved throughput first
        decreases from previous level.




  Gaikwad                   Expires 24 July 2026                 [Page 26]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.3.5.  Reporting Format

      +===============+==========+======+======+======+======+=========+
      | Offered (r/s) | Achieved | TTFT | TTFT | TPOT | TPOT | Success |
      |               | (tok/s)  | P50  | P99  | P50  | P99  |         |
      +===============+==========+======+======+======+======+=========+
      | 2             | 284      | 95   | 142  | 32   | 41   | 100%    |
      +---------------+----------+------+------+------+------+---------+
      | 6             | 852      | 102  | 178  | 34   | 48   | 100%    |
      +---------------+----------+------+------+------+------+---------+
      | 10            | 1420     | 128  | 267  | 38   | 62   | 100%    |
      +---------------+----------+------+------+------+------+---------+
      | 14            | 1988     | 198  | 512  | 48   | 98   | 100%    |
      +---------------+----------+------+------+------+------+---------+
      | 18            | 2534     | 378  | 1234 | 72   | 198  | 99.8%   |
      +---------------+----------+------+------+------+------+---------+
      | 22            | 2712     | 823  | 3456 | 142  | 523  | 94.1%   |
      +---------------+----------+------+------+------+------+---------+

                  Table 5: Throughput-Latency Table Example

     Knee point: 14 req/s (TTFT P99 exceeds 2x minimum)

     Saturation point: 22 req/s (throughput peaks)

  5.4.  Inter-Token Latency Distribution

  5.4.1.  Objective

     To characterize the variability of token delivery during the decode
     phase.  ITL distribution determines streaming smoothness experienced
     by users.

  5.4.2.  Setup Parameters

     Workload:  Synthetic-Uniform or Conversation workload RECOMMENDED.

     Minimum output length:  Requests MUST generate at least 50 output
        tokens to provide meaningful ITL samples.

     Request count:  At least 100 requests for per-request statistics,
        yielding 5000+ ITL samples.

     Load level:  Specify as percentage of maximum throughput.  Multiple
        load levels RECOMMENDED: 25%, 50%, 75%, 90% of saturation.

     Measurement method:  Specify per Section 4.6.3 (chunk timing,
        distributed timing, or server-side timing).



  Gaikwad                   Expires 24 July 2026                 [Page 27]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.4.3.  Procedure

     1.  Configure SUT and complete warm-up.

     2.  For each load level:

         a.  Generate requests at specified load.

         b.  For each request, record arrival time of each token after the
             first.

         c.  Calculate ITL_i = T(token_i) - T(token_{i-1}) for each
             consecutive token pair.

         d.  Aggregate ITL samples across all requests.

         e.  Calculate per-request jitter (standard deviation of ITL
             within each request).

         f.  Record maximum pause duration per request.

     The interval between request submission and first token (TTFT) MUST
     NOT be included in ITL calculation.

  5.4.4.  Measurements

  5.4.4.1.  Aggregate ITL Statistics

     ITL Percentiles:  P50, P90, P95, P99, P99.9 across all ITL samples.

     ITL Mean:  Arithmetic mean of all ITL samples.

     ITL Standard Deviation:  Standard deviation across all samples.

  5.4.4.2.  Per-Request Statistics

     Jitter Distribution:  P50, P95, P99 of per-request standard
        deviation.

     Maximum Pause Distribution:  P50, P95, P99 of per-request maximum
        ITL.

  5.4.4.3.  Distribution Shape

     Modality:  Whether ITL distribution is unimodal or multimodal.
        Multimodal distributions indicate distinct operating regimes
        (e.g., batching effects).




  Gaikwad                   Expires 24 July 2026                 [Page 28]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     Tail behavior:  Characterize tail (exponential, heavy-tailed).
        Report the ratio P99/P50 as a tail heaviness indicator.

  5.4.5.  Reporting Format

                          +===============+========+
                          | Metric        | Value  |
                          +===============+========+
                          | ITL Samples   | 15234  |
                          +---------------+--------+
                          | ITL P50       | 38 ms  |
                          +---------------+--------+
                          | ITL P90       | 52 ms  |
                          +---------------+--------+
                          | ITL P95       | 67 ms  |
                          +---------------+--------+
                          | ITL P99       | 124 ms |
                          +---------------+--------+
                          | ITL P99.9     | 312 ms |
                          +---------------+--------+
                          | ITL Mean      | 42 ms  |
                          +---------------+--------+
                          | ITL Std Dev   | 28 ms  |
                          +---------------+--------+
                          | P99/P50 Ratio | 3.26   |
                          +---------------+--------+

                             Table 6: ITL Results
                                   Example

  5.5.  Concurrent Request Capacity

  5.5.1.  Objective

     To determine the maximum number of concurrent requests the SUT can
     maintain while meeting latency objectives.  This test measures memory
     capacity and scheduling limits.

  5.5.2.  Setup Parameters

     Workload:  Synthetic-Uniform RECOMMENDED for controlled testing.

     Fixed output length:  Use fixed output length (e.g., 256 tokens) to
        ensure all requests have similar duration.

     Initial concurrency:  Starting number of concurrent requests (e.g.,
        8).




  Gaikwad                   Expires 24 July 2026                 [Page 29]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     Maximum concurrency:  Upper bound for search (e.g., 512).

     Success criteria:  Request completion rate >= 99%, TTFT P99 <=
        specified threshold, and no out-of-memory errors.

  5.5.3.  Procedure

     This test employs binary search to find maximum concurrent capacity.

     1.  Configure SUT and complete warm-up.

     2.  Set concurrency = initial concurrency.

     3.  For each concurrency level:

         a.  Submit [concurrency] requests simultaneously.

         b.  Maintain concurrency: when a request completes, immediately
             submit a replacement.

         c.  Run for at least 60 seconds or 100 request completions per
             slot, whichever is longer.

         d.  Record completion rate, latency percentiles, and any errors.

         e.  Check success criteria.

     4.  Binary search:

         a.  If success criteria met: increase concurrency toward maximum.

         b.  If success criteria not met: decrease concurrency.

         c.  Continue until convergence.

     5.  Report maximum concurrency meeting success criteria.

  5.5.4.  Measurements

     Maximum concurrent requests:  Highest concurrency meeting success
        criteria.

     Achieved throughput at maximum:  Output tokens per second at maximum
        concurrency.

     Tokens in flight at maximum:  Approximate total tokens (input +
        output so far) across all concurrent requests.




  Gaikwad                   Expires 24 July 2026                 [Page 30]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.5.5.  Reporting Format

     +=============+============+==========+==========+========+========+
     | Concurrency | Completion | TTFT P99 | TPOT P99 | Errors | Status |
     +=============+============+==========+==========+========+========+
     | 8           | 100%       | 142 ms   | 38 ms    | 0      | Pass   |
     +-------------+------------+----------+----------+--------+--------+
     | 16          | 100%       | 178 ms   | 42 ms    | 0      | Pass   |
     +-------------+------------+----------+----------+--------+--------+
     | 32          | 100%       | 267 ms   | 52 ms    | 0      | Pass   |
     +-------------+------------+----------+----------+--------+--------+
     | 64          | 99.7%      | 523 ms   | 78 ms    | 0      | Pass   |
     +-------------+------------+----------+----------+--------+--------+
     | 128         | 97.2%      | 1234 ms  | 156 ms   | 3      | Fail   |
     +-------------+------------+----------+----------+--------+--------+

                   Table 7: Capacity Search Results Example

     Maximum concurrent requests meeting criteria: 64

  5.6.  Scheduling Fairness

  5.6.1.  Objective

     To evaluate how equitably the SUT allocates resources across
     concurrent requests with different characteristics.  This test
     reveals head-of-line blocking, starvation, and priority effects.

  5.6.2.  Setup Parameters

     Workload:  Synthetic-Skewed REQUIRED.  The high length variance
        creates fairness-sensitive conditions.

     Request classes:  Define two or more request classes:

        *  Short requests: Input [64, 256] tokens, output [32, 128] tokens

        *  Long requests: Input [1024, 4096] tokens, output [256, 1024]
           tokens

     Class mix:  Ratio of request classes (e.g., 80% short, 20% long).

     Load level:  70-90% of saturation throughput RECOMMENDED to create
        contention.

     Request count:  At least 500 requests per class.





  Gaikwad                   Expires 24 July 2026                 [Page 31]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.6.3.  Procedure

     1.  Configure SUT and complete warm-up.

     2.  Measure baseline: performance of each class in isolation at same
         total load.

     3.  Generate mixed workload with specified class ratio.

     4.  Run at specified load level for at least 300 seconds.

     5.  For each request, record class membership, submission time, first
         token time, completion time.

     6.  Compute per-class statistics and fairness metrics.

  5.6.4.  Measurements

     Per-class latency:  TTFT P50, P95, P99 for each request class.

     Latency inflation:  (Mixed workload TTFT) / (Isolated TTFT) per
        class.

     Jain's Fairness Index:  J = (sum(x_i))^2 / (n * sum(x_i^2)) where x_i
        is normalized latency.  J = 1.0 indicates perfect fairness.  J <
        0.9 indicates significant unfairness.

     Starvation rate:  Fraction of requests waiting longer than 5x the
        median wait time for their class.

  5.6.5.  Reporting Format

         +=======+=======+==========+==========+==========+==========+
         | Class | Count | TTFT P50 | TTFT P99 | TPOT P50 | TPOT P99 |
         +=======+=======+==========+==========+==========+==========+
         | Short | 4012  | 89 ms    | 234 ms   | 35 ms    | 67 ms    |
         +-------+-------+----------+----------+----------+----------+
         | Long  | 988   | 312 ms   | 1234 ms  | 42 ms    | 89 ms    |
         +-------+-------+----------+----------+----------+----------+

                       Table 8: Per-Class Results Example










  Gaikwad                   Expires 24 July 2026                 [Page 32]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


                      +========================+=======+
                      | Metric                 | Value |
                      +========================+=======+
                      | Jain's Fairness Index  | 0.87  |
                      +------------------------+-------+
                      | Short Class Starvation | 0.3%  |
                      +------------------------+-------+
                      | Long Class Starvation  | 2.1%  |
                      +------------------------+-------+

                      Table 9: Fairness Metrics Example

  5.7.  Prefix Cache Effectiveness

  5.7.1.  Objective

     To evaluate the performance benefit of prefix caching under workloads
     with shared prefixes.  This test quantifies TTFT reduction from cache
     hits.

  5.7.2.  Setup Parameters

     Workload:  Code Completion workload RECOMMENDED (high prefix
        sharing).

     Shared prefix:  Define a prefix shared across requests.

     Prefix length:  Length in tokens of shared prefix.

     Sharing fraction:  Percentage of requests sharing the prefix.

     Comparison mode:  Test MUST run in two configurations: cache disabled
        (baseline) and cache enabled.

  5.7.3.  Procedure

     1.  Configure SUT with cache disabled.

     2.  Complete warm-up (without populating prefix cache).

     3.  Run workload, record TTFT for all requests.

     4.  Enable prefix cache.

     5.  Optionally pre-populate cache with shared prefix.

     6.  Run identical workload, record TTFT for all requests.




  Gaikwad                   Expires 24 July 2026                 [Page 33]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     7.  Compare results.

  5.7.4.  Measurements

     TTFT without cache:  P50, P95, P99 with caching disabled.

     TTFT with cache:  P50, P95, P99 with caching enabled.

     TTFT reduction:  (TTFT_no_cache - TTFT_cache) / TTFT_no_cache as
        percentage.

     Cache hit rate:  Fraction of prefix tokens served from cache.

     Throughput improvement:  Percentage increase from caching.

  5.7.5.  Reporting Format

              +================+==========+==========+==========+
              | Configuration  | TTFT P50 | TTFT P95 | TTFT P99 |
              +================+==========+==========+==========+
              | Cache Disabled | 312 ms   | 423 ms   | 534 ms   |
              +----------------+----------+----------+----------+
              | Cache (Cold)   | 134 ms   | 198 ms   | 267 ms   |
              +----------------+----------+----------+----------+
              | Cache (Warm)   | 98 ms    | 156 ms   | 212 ms   |
              +----------------+----------+----------+----------+

                     Table 10: Cache Effectiveness Example

  5.8.  Memory Pressure Behavior

  5.8.1.  Objective

     To characterize SUT behavior when memory resources are constrained,
     including preemption, swapping, and degradation patterns.

  5.8.2.  Setup Parameters

     Workload:  Long Context workload RECOMMENDED to create memory
        pressure.

     Oversubscription level:  Percentage above maximum capacity (e.g.,
        110%, 125%, 150%).

  5.8.3.  Procedure

     1.  Determine maximum concurrent capacity from Section 5.5.




  Gaikwad                   Expires 24 July 2026                 [Page 34]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     2.  Configure SUT and complete warm-up.

     3.  For each oversubscription level:

         a.  Submit requests at concurrency exceeding capacity.

         b.  Run for at least 120 seconds.

         c.  Monitor request completions, preemption events, latency.

         d.  Record any OOM errors or system failures.

     4.  Analyze degradation patterns.

  5.8.4.  Measurements

     Completion rate:  Percentage of requests completing successfully at
        each level.

     Preemption rate:  Fraction of requests preempted at least once.

     Preemption recovery rate:  Fraction of preempted requests that
        eventually complete.

     Preemption loss:  Average tokens discarded per preemption event.

  5.8.5.  Reporting Format

         +===============+==========+=========+===========+==========+
         | Oversub Level | Complete | Preempt | Fail Rate | TTFT P99 |
         +===============+==========+=========+===========+==========+
         | 100% (base)   | 99.7%    | 0%      | 0.3%      | 523 ms   |
         +---------------+----------+---------+-----------+----------+
         | 110%          | 98.2%    | 5.2%    | 1.8%      | 789 ms   |
         +---------------+----------+---------+-----------+----------+
         | 125%          | 94.5%    | 18.7%   | 5.5%      | 1456 ms  |
         +---------------+----------+---------+-----------+----------+
         | 150%          | 82.3%    | 42.1%   | 17.7%     | 3234 ms  |
         +---------------+----------+---------+-----------+----------+

                 Table 11: Memory Pressure Degradation Example

  5.9.  Long Context Scaling

  5.9.1.  Objective

     To characterize how latency and throughput scale with context length.




  Gaikwad                   Expires 24 July 2026                 [Page 35]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.9.2.  Setup Parameters

     Workload:  Long Context workload REQUIRED.

     Context length range:  Sequence of lengths to test (e.g., 1K, 2K, 4K,
        8K, 16K, 32K, 64K, 128K tokens).

     Fixed output length:  Use consistent short output (256 tokens) to
        isolate prefill impact.

     Load model:  Closed-loop with low concurrency (1-4).

     Requests per length:  At least 20 requests per context length.

  5.9.3.  Procedure

     1.  Configure SUT and complete warm-up with short-context requests.

     2.  For each context length in ascending order:

         a.  Generate requests with specified input length.

         b.  Submit requests at low concurrency.

         c.  Record TTFT and total latency for each request.

     3.  Analyze scaling behavior and fit to scaling models.

  5.9.4.  Measurements

     Per-length latency:  TTFT Mean, P50, P95 for each context length.

     Prefill scaling:  Time per input token (TTFT / input_length).

     Scaling exponent:  Fit exponent k where TTFT proportional to
        context_length^k.

     Throughput at length:  Maximum throughput achievable at each context
        length.

  5.9.5.  Reporting Format

          +==================+===========+==========+==============+
          | Context (tokens) | TTFT Mean | TTFT P95 | ms/1K tokens |
          +==================+===========+==========+==============+
          | 1024             | 89 ms     | 112 ms   | 76           |
          +------------------+-----------+----------+--------------+
          | 4096             | 289 ms    | 367 ms   | 63           |



  Gaikwad                   Expires 24 July 2026                 [Page 36]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


          +------------------+-----------+----------+--------------+
          | 16384            | 1023 ms   | 1287 ms  | 59           |
          +------------------+-----------+----------+--------------+
          | 65536            | 4234 ms   | 5123 ms  | 62           |
          +------------------+-----------+----------+--------------+
          | 131072           | 9123 ms   | 11234 ms | 68           |
          +------------------+-----------+----------+--------------+

                    Table 12: Long Context Scaling Example

     Best fit: Linear (R^2 = 0.9987), ~68 microseconds per input token

  5.10.  Guardrail Overhead

  5.10.1.  Objective

     To quantify the latency impact of safety systems and content
     filtering.

  5.10.2.  Setup Parameters

     Workload:  Conversation workload RECOMMENDED.

     Content mix:  Use benign content to measure processing overhead.

     Configurations to compare:  The following configurations should be
        tested:

        *  Baseline: All guardrails disabled (if possible)

        *  Input filtering only

        *  Output filtering only

        *  Full filtering: All production guardrails enabled

     Load levels:  Test at 25%, 50%, 75% of capacity.

  5.10.3.  Procedure

     1.  Configure SUT with baseline (no guardrails).

     2.  Complete warm-up and run workload at each load level.

     3.  Enable each guardrail configuration and repeat.

     4.  Compare results across configurations.




  Gaikwad                   Expires 24 July 2026                 [Page 37]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  5.10.4.  Measurements

     Per-configuration latency:  TTFT P50, P95, P99 and End-to-end latency
        for each configuration.

     Input filter overhead:  TTFT(input_filter) - TTFT(baseline)

     Total guardrail overhead:  End-to-end(full) - End-to-end(baseline)

     Throughput reduction:  Percentage reduction from guardrails.

  5.10.5.  Reporting Format

          +===============+==========+==========+=========+=========+
          | Configuration | TTFT P50 | TTFT P99 | E2E P50 | E2E P99 |
          +===============+==========+==========+=========+=========+
          | Baseline      | 98 ms    | 234 ms   | 4.2 s   | 8.7 s   |
          +---------------+----------+----------+---------+---------+
          | Input Filter  | 112 ms   | 267 ms   | 4.3 s   | 8.9 s   |
          +---------------+----------+----------+---------+---------+
          | Output Filter | 101 ms   | 242 ms   | 4.8 s   | 9.8 s   |
          +---------------+----------+----------+---------+---------+
          | Full Filter   | 118 ms   | 289 ms   | 5.0 s   | 10.2 s  |
          +---------------+----------+----------+---------+---------+

                      Table 13: Guardrail Overhead Example

                +===============+================+===========+
                | Configuration | Max Throughput | Reduction |
                +===============+================+===========+
                | Baseline      | 2867 tok/s     | -         |
                +---------------+----------------+-----------+
                | Input Filter  | 2756 tok/s     | -3.9%     |
                +---------------+----------------+-----------+
                | Output Filter | 2412 tok/s     | -15.9%    |
                +---------------+----------------+-----------+
                | Full Filter   | 2289 tok/s     | -20.2%    |
                +---------------+----------------+-----------+

                     Table 14: Throughput Impact Example

  6.  Multi-System Comparison Guidelines

     When comparing multiple SUTs:







  Gaikwad                   Expires 24 July 2026                 [Page 38]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  6.1.  Equivalence Requirements

     Testers MUST ensure:

     *  Identical workload (same requests in same order with same seeds)

     *  Equivalent SUT boundary (all systems at same boundary)

     *  Comparable hardware (or normalize by hardware capability)

     *  Same load model and parameters

  6.2.  Normalization

     When hardware differs:

     *  Report tokens per GPU-second (normalized by GPU count)

     *  Report cost-normalized throughput (tokens per dollar-hour)

     *  Clearly state normalization method

  6.3.  Statistical Significance

     For comparative claims:

     *  Report confidence intervals for key metrics

     *  Conduct multiple independent runs (at least 3)

     *  Use appropriate statistical tests for comparison

  6.4.  Fair Comparison Checklist

     Before publishing comparative results, verify:

     *  Same workload specification

     *  Same test duration

     *  Same warm-up procedure

     *  Same success criteria

     *  Both systems tested at same time (if using shared resources)

     *  Both systems in production-representative configuration




  Gaikwad                   Expires 24 July 2026                 [Page 39]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     *  Differences in configuration explicitly noted

  7.  Security Considerations

     Benchmarking methodology intersects with security in several ways.

  7.1.  Side-Channel Risks

     Benchmark results may reveal:

     *  System capacity limits useful for DoS planning

     *  Timing patterns enabling cache probing attacks

     *  Memory pressure thresholds for resource exhaustion

     Operators SHOULD consider whether to publish detailed capacity
     information publicly.

  7.2.  Benchmark Gaming

     Systems may be optimized specifically for benchmark workloads in ways
     that do not generalize:

     *  Detecting benchmark patterns and applying special handling

     *  Caching benchmark-specific prefixes

     *  Prioritizing benchmark-like requests

     Testers SHOULD vary workloads and verify results with production
     traffic samples.

  7.3.  Adversarial Workloads

     This methodology uses benign workloads.  Adversarial inputs
     (jailbreak attempts, prompt injections) may have different
     performance characteristics due to guardrail processing.

     Testing with adversarial workloads requires additional ethical and
     safety considerations not covered here.

  7.4.  Resource Exhaustion

     Memory pressure tests (Section 5.8) intentionally push systems beyond
     capacity.  Testers SHOULD:

     *  Conduct such tests on isolated systems



  Gaikwad                   Expires 24 July 2026                 [Page 40]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     *  Have recovery procedures ready

     *  Monitor for cascading failures

  8.  References

  8.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/info/rfc8174>.

     [LLM-TERMS]
                Gaikwad, M., "Benchmarking Terminology for Large Language
                Model Serving", Work in Progress, Internet-Draft, draft-
                gaikwad-llm-benchmarking-terminology-00, January 2026,
                <https://datatracker.ietf.org/doc/draft-gaikwad-llm-
                benchmarking-terminology/>.

  8.2.  Informative References

     [RFC1242]  Bradner, S., "Benchmarking Terminology for Network
                Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242,
                July 1991, <https://www.rfc-editor.org/info/rfc1242>.

     [RFC2544]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
                Network Interconnect Devices", RFC 2544,
                DOI 10.17487/RFC2544, March 1999,
                <https://www.rfc-editor.org/info/rfc2544>.

     [RFC3511]  Hickman, B., Newman, D., Tadjudin, S., and T. Martin,
                "Benchmarking Methodology for Firewall Performance",
                RFC 3511, DOI 10.17487/RFC3511, April 2003,
                <https://www.rfc-editor.org/info/rfc3511>.

     [VLLM]     Kwon, W., "Efficient Memory Management for Large Language
                Model Serving with PagedAttention", Proceedings of SOSP
                2023, DOI 10.1145/3600006.3613165, 2023,
                <https://doi.org/10.1145/3600006.3613165>.

     [SARATHI]  Agrawal, A., "Taming Throughput-Latency Tradeoff in LLM
                Inference with Sarathi-Serve", Proceedings of OSDI 2024,
                2024.



  Gaikwad                   Expires 24 July 2026                 [Page 41]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  Appendix A.  Reference Workload Specifications

     This appendix provides complete specifications for standard
     workloads.

  A.1.  Synthetic-Uniform Workload

     Purpose: Controlled baseline with minimal variance

  A.1.1.  Input Specification

     Distribution:  Uniform

     Minimum:  128 tokens

     Maximum:  512 tokens

     Mean:  320 tokens

     Content:  Random token IDs from vocabulary

  A.1.2.  Output Specification

     Distribution:  Uniform

     Minimum:  64 tokens

     Maximum:  256 tokens

     Mean:  160 tokens

     Control:  max_tokens parameter

  A.1.3.  Other Parameters

     System prompt:  None

     Prefix sharing:  None

     Temperature:  0.0 (deterministic)

     Stop sequences:  None

  A.1.4.  Generation Method

     Python pseudocode:





  Gaikwad                   Expires 24 July 2026                 [Page 42]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     def generate_synthetic_uniform(n_requests, seed=42):
         rng = random.Random(seed)
         requests = []
         for i in range(n_requests):
             input_len = rng.randint(128, 512)
             output_len = rng.randint(64, 256)
             input_tokens = [rng.randint(0, 100255)
                             for _ in range(input_len)]
             requests.append({
                 'input_tokens': input_tokens,
                 'max_tokens': output_len,
                 'temperature': 0.0
             })
         return requests

  A.2.  Synthetic-Skewed Workload

     Purpose: Test scheduling with high length variance

  A.2.1.  Input Specification

     Distribution:  Log-normal

     mu:  5.5 (in log space)

     sigma:  1.0 (in log space)

     Minimum:  32 tokens (floor)

     Maximum:  4096 tokens (cap)

     Median:  ~245 tokens

     Mean:  ~405 tokens

  A.2.2.  Output Specification

     Distribution:  Log-normal

     mu:  4.5 (in log space)

     sigma:  1.2 (in log space)

     Minimum:  16 tokens (floor)

     Maximum:  2048 tokens (cap)





  Gaikwad                   Expires 24 July 2026                 [Page 43]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  A.3.  Conversation Workload

     Purpose: Realistic interactive chat patterns

  A.3.1.  Data Source

     Dataset:  ShareGPT (vicuna_cleaned subset)

     Version:  2023-04-12

     Preprocessing:  Filter conversations with 1+ assistant turns

  A.3.2.  Length Statistics (Reference)

     Input tokens:

     *  P50: 156

     *  P95: 892

     *  P99: 2134

     Output tokens:

     *  P50: 234

     *  P95: 789

     *  P99: 1567

  A.4.  Code Completion Workload

     Purpose: Test prefix caching with code context

  A.4.1.  Data Source

     Dataset:  The Stack (Python, JavaScript, TypeScript subset)

     Preprocessing:  Extract function-level completions

  A.4.2.  Prefix Sharing Pattern

     *  10 unique repository contexts

     *  Each 512-1024 tokens

     *  80% of requests share one of these prefixes




  Gaikwad                   Expires 24 July 2026                 [Page 44]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


     *  Distribution: Zipf with s=1.5

  A.5.  Long Context Workload

     Purpose: Test long-context handling

  A.5.1.  Input Specification

     Distribution:  Uniform over target lengths

     Target lengths:  [8192, 16384, 32768, 65536, 131072] tokens

     Structure:  [document][question]

     Document:  Fills target length minus 100 tokens

     Question:  Fixed ~100 token question about document

  A.5.2.  Output Specification

     Distribution:  Fixed

     Length:  256 tokens

     Control:  max_tokens = 256

  Appendix B.  Timing Measurement Reference

     This appendix provides detailed guidance for timing measurements.

  B.1.  TTFT Measurement Points

  B.1.1.  HTTP/SSE Measurement

     Client-side TTFT:

     T_submit:  time of sending final byte of HTTP request

     T_first:  time of receiving first data event with content token

     T_first is when the complete "data:" line is received and parsed, not
     when the first byte of the response arrives.

  B.1.2.  gRPC Streaming Measurement

     T_submit:  time of sending request message

     T_first:  time of receiving first response message with token



  Gaikwad                   Expires 24 July 2026                 [Page 45]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  B.1.3.  Server-Side Measurement

     If server instrumentation available:

     T_submit:  time request enters inference queue

     T_first:  time first token exits model forward pass

     Server-side excludes network latency but may include internal queue
     time.

  B.2.  ITL Measurement with SSE

     SSE delivery may batch multiple tokens per event due to server-side
     batching, TCP buffering, or client-side buffering.

  B.2.1.  Recommended Approach

     1.  First, characterize delivery pattern (tokens per chunk)

     2.  If single-token chunks dominate (>90%): use direct measurement

     3.  If multi-token chunks common: prefer server timestamps

     4.  If server timestamps unavailable: use chunk timing and document

  B.3.  Clock Synchronization Methods

  B.3.1.  NTP Synchronization

     1.  Both machines sync to same NTP server

     2.  Verify offset: ntpq -p (check offset column)

     3.  Acceptable offset: < 10ms for most LLM benchmarking

     4.  Document NTP server and measured offset

  B.3.2.  PTP Synchronization

     For sub-millisecond accuracy:

     1.  Use PTP-capable network hardware

     2.  Configure ptp4l on Linux systems

     3.  Acceptable offset: < 1 microsecond




  Gaikwad                   Expires 24 July 2026                 [Page 46]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  B.3.3.  Single-Machine Alternative

     Recommended for Model Engine testing:

     1.  Run load generator on same machine as SUT

     2.  Use loopback network interface

     3.  Clock synchronization inherent

     4.  Eliminates network latency from measurement

  Appendix C.  Reporting Templates

  C.1.  Minimum Viable Report

     For quick comparisons, include at minimum:

     === LLM Benchmark Report (Minimum) ===

     System Identification:
     - Model: [model name and version]
     - Hardware: [GPU type] x [count]
     - Software: [inference engine and version]
     - SUT Boundary: [Model Engine | Gateway | Compound]

     Test Configuration:
     - Workload: [workload name]
     - Load Model: [open-loop rate | closed-loop concurrency]
     - Request Count: [N]
     - Test Duration: [seconds]

     Key Results:
     - TTFT P50: [value] ms
     - TTFT P99: [value] ms
     - TPOT P50: [value] ms
     - TPOT P99: [value] ms
     - Max Throughput: [value] tok/s
     - Throughput at P99 TTFT < 500ms: [value] tok/s

     Notes:
     - [Any deviations from methodology]
     - [Guardrail configuration]

     === End Report ===






  Gaikwad                   Expires 24 July 2026                 [Page 47]


  Internet-Draft        LLM Benchmarking Methodology          January 2026


  C.2.  Full Report Template

     A complete benchmark report should include the following sections:

     1.  System Identification (model, hardware, software)

     2.  Test Configuration (workload, load, execution parameters)

     3.  Results (latency summary, throughput summary, success metrics)

     4.  Detailed Results (per-test tables and visualizations)

     5.  Methodology Compliance (tests performed, deviations, limitations)

     6.  Reproduction Information (test harness, configuration, data)

  Acknowledgements

     This document draws on the structure and approach established by RFC
     3511 for firewall benchmarking methodology.  The author thanks the
     Benchmarking Methodology Working Group for their foundational work in
     network device benchmarking.

  Author's Address

     Madhava Gaikwad
     Independent Researcher
     Email: gaikwad.madhav@gmail.com























  Gaikwad                   Expires 24 July 2026                 [Page 48]


name: draft-gaikwad-llm-benchmarking-profiles
rev: 00
title: Performance Benchmarking Profiles for Large Language Model Serving Systems
time: 2026-01-20T18:16:53Z
group_uri: /api/v1/group/group/1027/
expires: 2026-07-24T18:16:53Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-07T06:55:11.565670+00:00
source: ietf
abstract:
  This document defines performance benchmarking profiles for Large
     Language Model (LLM) serving systems.  Profiles bind the terminology
     defined in draft-gaikwad-llm-benchmarking-terminology and the
     procedures described in draft-gaikwad-llm-benchmarking-methodology to
     concrete architectural roles and workload patterns.  Each profile
     clarifies the System Under Test (SUT) boundary, measurement points,
     and interpretation constraints required for reproducible and
     comparable benchmarking.

     This document specifies profiles only.  It does not define new
     metrics, benchmark workloads, or acceptance thresholds.

full_text:
  Network Working Group                                         M. Gaikwad
  Internet-Draft                                    Independent Researcher
  Intended status: Informational                              January 2026
  Expires: 24 July 2026


     Performance Benchmarking Profiles for Large Language Model Serving
                                  Systems
                 draft-gaikwad-llm-benchmarking-profiles-00

  Abstract

     This document defines performance benchmarking profiles for Large
     Language Model (LLM) serving systems.  Profiles bind the terminology
     defined in draft-gaikwad-llm-benchmarking-terminology and the
     procedures described in draft-gaikwad-llm-benchmarking-methodology to
     concrete architectural roles and workload patterns.  Each profile
     clarifies the System Under Test (SUT) boundary, measurement points,
     and interpretation constraints required for reproducible and
     comparable benchmarking.

     This document specifies profiles only.  It does not define new
     metrics, benchmark workloads, or acceptance thresholds.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 5 July 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.






  Gaikwad                   Expires 24 July 2026                  [Page 1]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   4
       1.1.  Terminology Alignment . . . . . . . . . . . . . . . . . .   5
     2.  Requirements Language . . . . . . . . . . . . . . . . . . . .   6
     3.  Profile Taxonomy  . . . . . . . . . . . . . . . . . . . . . .   6
       3.1.  Infrastructure Profiles . . . . . . . . . . . . . . . . .   6
       3.2.  Workload Profiles . . . . . . . . . . . . . . . . . . . .   7
       3.3.  Profile Selection Guidance  . . . . . . . . . . . . . . .   8
     4.  Infrastructure Profiles . . . . . . . . . . . . . . . . . . .   8
       4.1.  Model Engine Profile  . . . . . . . . . . . . . . . . . .   9
         4.1.1.  Definition and Concepts . . . . . . . . . . . . . . .   9
         4.1.2.  Boundary Specification  . . . . . . . . . . . . . . .   9
         4.1.3.  Architecture Variants . . . . . . . . . . . . . . . .  10
         4.1.4.  Configuration Disclosure  . . . . . . . . . . . . . .  14
         4.1.5.  Primary Metrics . . . . . . . . . . . . . . . . . . .  16
         4.1.6.  Secondary Metrics . . . . . . . . . . . . . . . . . .  16
         4.1.7.  Benchmarking Constraints  . . . . . . . . . . . . . .  17
       4.2.  AI Gateway Profile  . . . . . . . . . . . . . . . . . . .  17
         4.2.1.  Definition and Concepts . . . . . . . . . . . . . . .  17
         4.2.2.  Boundary Specification  . . . . . . . . . . . . . . .  17
         4.2.3.  Baseline Requirement  . . . . . . . . . . . . . . . .  18
         4.2.4.  Load Balancing Disclosure . . . . . . . . . . . . . .  19
         4.2.5.  Multi-Model Gateway . . . . . . . . . . . . . . . . .  19
         4.2.6.  Semantic Cache  . . . . . . . . . . . . . . . . . . .  20
       4.3.  AI Firewall Profile . . . . . . . . . . . . . . . . . . .  22
         4.3.1.  Definition and Concepts . . . . . . . . . . . . . . .  22
         4.3.2.  Boundary Specification  . . . . . . . . . . . . . . .  22
         4.3.3.  Enforcement Directions  . . . . . . . . . . . . . . .  23
         4.3.4.  Inspection Architecture . . . . . . . . . . . . . . .  24
         4.3.5.  Required Metrics  . . . . . . . . . . . . . . . . . .  25
         4.3.6.  Workload Specification  . . . . . . . . . . . . . . .  27
         4.3.7.  Multi-Layer Firewall  . . . . . . . . . . . . . . . .  28
         4.3.8.  Benchmarking Constraints  . . . . . . . . . . . . . .  28
       4.4.  Compound System Profile . . . . . . . . . . . . . . . . .  29
         4.4.1.  Definition and Concepts . . . . . . . . . . . . . . .  29
         4.4.2.  Boundary Specification  . . . . . . . . . . . . . . .  29
         4.4.3.  Primary Metrics . . . . . . . . . . . . . . . . . . .  30
         4.4.4.  Evaluation Oracle . . . . . . . . . . . . . . . . . .  31



  Gaikwad                   Expires 24 July 2026                  [Page 2]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


         4.4.5.  Secondary Metrics . . . . . . . . . . . . . . . . . .  32
         4.4.6.  RAG Sub-Profile . . . . . . . . . . . . . . . . . . .  33
         4.4.7.  Agentic System Boundaries . . . . . . . . . . . . . .  34
         4.4.8.  Exclusions  . . . . . . . . . . . . . . . . . . . . .  36
     5.  Workload Profiles . . . . . . . . . . . . . . . . . . . . . .  36
       5.1.  Chatbot Workload Profile  . . . . . . . . . . . . . . . .  36
         5.1.1.  Characteristics . . . . . . . . . . . . . . . . . . .  36
         5.1.2.  Required Parameters . . . . . . . . . . . . . . . . .  37
         5.1.3.  Example . . . . . . . . . . . . . . . . . . . . . . .  37
       5.2.  Compound Workflow Workload Profile  . . . . . . . . . . .  37
         5.2.1.  Characteristics . . . . . . . . . . . . . . . . . . .  37
         5.2.2.  Required Parameters . . . . . . . . . . . . . . . . .  38
         5.2.3.  External Dependency Handling  . . . . . . . . . . . .  38
     6.  Delta Measurement Model . . . . . . . . . . . . . . . . . . .  39
       6.1.  Timestamp Reference . . . . . . . . . . . . . . . . . . .  39
       6.2.  Timestamp Definitions . . . . . . . . . . . . . . . . . .  40
       6.3.  Component Deltas  . . . . . . . . . . . . . . . . . . . .  41
       6.4.  End-to-End Metrics  . . . . . . . . . . . . . . . . . . .  42
       6.5.  Clock Synchronization . . . . . . . . . . . . . . . . . .  42
     7.  Profile Composition . . . . . . . . . . . . . . . . . . . . .  42
       7.1.  Composite SUT Declaration . . . . . . . . . . . . . . . .  43
       7.2.  Composition Validation  . . . . . . . . . . . . . . . . .  43
       7.3.  Interaction Effects . . . . . . . . . . . . . . . . . . .  44
     8.  Access Logging Requirements . . . . . . . . . . . . . . . . .  45
       8.1.  Minimum Fields  . . . . . . . . . . . . . . . . . . . . .  45
       8.2.  Model Engine Fields . . . . . . . . . . . . . . . . . . .  46
       8.3.  AI Firewall Fields  . . . . . . . . . . . . . . . . . . .  46
       8.4.  Compound System Fields  . . . . . . . . . . . . . . . . .  47
       8.5.  AI Gateway Fields . . . . . . . . . . . . . . . . . . . .  47
       8.6.  OpenTelemetry Integration . . . . . . . . . . . . . . . .  47
     9.  Measurement Considerations  . . . . . . . . . . . . . . . . .  48
       9.1.  Baseline and Delta Reporting  . . . . . . . . . . . . . .  48
       9.2.  Warm-up and Steady State  . . . . . . . . . . . . . . . .  48
       9.3.  Clock Synchronization . . . . . . . . . . . . . . . . . .  48
       9.4.  Streaming Protocol Considerations . . . . . . . . . . . .  49
     10. Security Considerations . . . . . . . . . . . . . . . . . . .  49
       10.1.  Bidirectional Enforcement Gaps . . . . . . . . . . . . .  49
       10.2.  Adversarial Workload Handling  . . . . . . . . . . . . .  50
       10.3.  Side-Channel Considerations  . . . . . . . . . . . . . .  50
     11. References  . . . . . . . . . . . . . . . . . . . . . . . . .  50
       11.1.  Normative References . . . . . . . . . . . . . . . . . .  50
       11.2.  Informative References . . . . . . . . . . . . . . . . .  51
     Appendix A.  Example Benchmark Report Structure . . . . . . . . .  52
     Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  52







  Gaikwad                   Expires 24 July 2026                  [Page 3]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  1.  Introduction

     LLM serving systems are rarely monolithic.  Production deployments
     typically compose multiple infrastructural intermediaries before a
     request reaches a Model Engine.  A request may pass through an API
     gateway for authentication, an AI firewall for prompt inspection, a
     load balancer for routing, and finally arrive at an inference engine.
     Each component adds latency and affects throughput.

     Performance metrics such as Time to First Token (TTFT) or throughput
     are boundary dependent.  A TTFT measurement taken at the client
     includes network latency, gateway processing, firewall inspection,
     queue wait time, and prefill computation.  The same measurement taken
     at the engine boundary includes only queue wait and prefill.  Without
     explicit boundary declaration, reported results cannot be compared.

     This document addresses this ambiguity by defining benchmarking
     profiles: standardized descriptions of SUT boundaries and their
     associated performance interpretation rules.  Section 4 defines four
     infrastructure profiles that specify what component is being
     measured.  Section 5 defines workload profiles that specify how that
     component is tested.  Section 6 then shows how to attribute latency
     across composed systems using delta measurement.




























  Gaikwad                   Expires 24 July 2026                  [Page 4]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


                  Client
                    |
                    v
          +------------------+
          |   AI Gateway     |<-- Auth, routing, caching
          +--------+---------+
                   |
                   v
          +------------------+
          |   AI Firewall    |<-- Prompt inspection
          +--------+---------+
                   |
                   v
          +------------------+
          |  Model Engine    |<-- Inference
          +--------+---------+
                   |
                   v
          +------------------+
          |   AI Firewall    |<-- Output inspection
          +--------+---------+
                   |
                   v
          +------------------+
          |   AI Gateway     |<-- Response normalization
          +--------+---------+
                   |
                   v
                 Client

  Each layer adds latency. Benchmarks must declare which layers are included.

                   Figure 1: Typical LLM Serving Stack

  1.1.  Terminology Alignment

     This document uses metrics defined in
     [I-D.gaikwad-llm-benchmarking-terminology].  The following table maps
     profile-specific terms to their normative definitions.












  Gaikwad                   Expires 24 July 2026                  [Page 5]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


          +=========================+===============================+
          | Term Used in Profiles   | Terminology Draft Reference   |
          +=========================+===============================+
          | TTFT                    | Time to First Token           |
          +-------------------------+-------------------------------+
          | ITL                     | Inter-Token Latency           |
          +-------------------------+-------------------------------+
          | TPOT                    | Time per Output Token         |
          +-------------------------+-------------------------------+
          | Queue Residence Time    | Queue Wait Time               |
          +-------------------------+-------------------------------+
          | FRR                     | False Refusal Rate            |
          +-------------------------+-------------------------------+
          | Guardrail Overhead      | Guardrail Processing Overhead |
          +-------------------------+-------------------------------+
          | Task Completion Latency | Task Completion Latency       |
          +-------------------------+-------------------------------+
          | Goodput                 | Goodput                       |
          +-------------------------+-------------------------------+

                          Table 1: Terminology Mapping

  2.  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in BCP
     14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

  3.  Profile Taxonomy

     Profiles divide into two categories that serve orthogonal purposes.
     Conflating them produces misleading benchmarks.

  3.1.  Infrastructure Profiles

     Infrastructure Profiles define what is being tested.  They specify
     the SUT boundary: where measurements start and end, what components
     are included, and what is excluded.











  Gaikwad                   Expires 24 July 2026                  [Page 6]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     +==============+========================+===========================+
     | Profile      | SUT Boundary           | Primary Question Answered |
     +==============+========================+===========================+
     | Model Engine | Inference              | How fast can this engine  |
     |              | runtime only           | generate tokens?          |
     +--------------+------------------------+---------------------------+
     | AI Gateway   | API intermediary       | What overhead does the    |
     |              | layer                  | gateway add?              |
     +--------------+------------------------+---------------------------+
     | AI Firewall  | Security               | What latency and accuracy |
     |              | inspection layer       | does inspection cost?     |
     +--------------+------------------------+---------------------------+
     | Compound     | End-to-end             | How long does it take to  |
     | System       | orchestration          | complete a task?          |
     +--------------+------------------------+---------------------------+

                        Table 2: Infrastructure Profiles

     The choice of infrastructure profile determines which metrics are
     meaningful.  Measuring "AI Firewall throughput" in tokens per second
     conflates firewall performance with downstream engine performance.
     The firewall does not generate tokens; it inspects them.  Appropriate
     firewall metrics include inspection latency, detection rate, and
     false positive rate.

  3.2.  Workload Profiles

     Workload Profiles define how the SUT is tested.  They specify traffic
     patterns, request characteristics, and arrival models.  Workload
     profiles are independent of infrastructure profiles.

      +==================+========================+====================+
      | Profile          | Traffic Pattern        | Applicable To      |
      +==================+========================+====================+
      | Chatbot Workload | Multi-turn, streaming, | Engine, Gateway,   |
      |                  | human-paced            | Firewall, Compound |
      +------------------+------------------------+--------------------+
      | Compound         | Multi-step, tool-      | Compound System    |
      | Workflow         | using, machine-paced   | primarily          |
      +------------------+------------------------+--------------------+

                          Table 3: Workload Profiles









  Gaikwad                   Expires 24 July 2026                  [Page 7]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     A Chatbot Workload can be applied to a Model Engine (measuring raw
     inference speed), an AI Gateway (measuring gateway overhead under
     conversational traffic), or a Compound System (measuring end-to-end
     chat latency including retrieval).  The infrastructure profile
     determines the measurement boundary; the workload profile determines
     the traffic shape.

     Conflating infrastructure and workload profiles produces non-
     comparable results.  "Chatbot benchmark on Gateway A" versus "Chatbot
     benchmark on Engine B" compares different things.  The former
     includes gateway overhead; the latter does not.  Valid comparison
     requires either:

     *  Same infrastructure profile, different implementations (Gateway A
        vs Gateway B)

     *  Same implementation, different workload profiles (Chatbot vs
        Compound Workflow on Engine A)

     Cross-profile comparisons require explicit delta decomposition
     (Section 6).

  3.3.  Profile Selection Guidance

     +===========================+====================+==================+
     | If you want to            | Use Infrastructure | Apply Workload   |
     | measure...                | Profile            | Profile          |
     +===========================+====================+==================+
     | Raw model inference       | Model Engine       | Chatbot or       |
     | speed                     |                    | synthetic        |
     +---------------------------+--------------------+------------------+
     | Gateway routing           | AI Gateway         | Match production |
     | overhead                  |                    | traffic          |
     +---------------------------+--------------------+------------------+
     | Security inspection       | AI Firewall        | Mixed benign/    |
     | cost                      |                    | adversarial      |
     +---------------------------+--------------------+------------------+
     | End-to-end agent          | Compound System    | Compound         |
     | latency                   |                    | Workflow         |
     +---------------------------+--------------------+------------------+
     | Full-stack production     | Composite (see     | Match production |
     | performance               | Section 7)         | traffic          |
     +---------------------------+--------------------+------------------+

                        Table 4: Profile Selection Guide

  4.  Infrastructure Profiles




  Gaikwad                   Expires 24 July 2026                  [Page 8]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  4.1.  Model Engine Profile

  4.1.1.  Definition and Concepts

     A Model Engine is the runtime responsible for executing LLM
     inference.  Before specifying the benchmark boundary, understanding
     three core operations is necessary:

     *Prefill* (also called prompt processing): The engine processes all
     input tokens in parallel to build initial hidden states.  Prefill is
     compute-bound and benefits from parallelism.  Prefill latency scales
     with input length but can be reduced by adding more compute.

     *Decode* (also called autoregressive generation): The engine
     generates output tokens one at a time, each depending on all previous
     tokens.  Decode is memory-bandwidth-bound because each token requires
     reading the full model weights.  Decode latency per token is
     relatively constant regardless of batch size, but throughput
     increases with batching.

     *KV Cache*: To avoid recomputing attention over previous tokens, the
     engine stores key-value pairs from prior tokens.  The KV cache grows
     with sequence length and consumes GPU memory.  Cache management
     (allocation, eviction, swapping to CPU) directly affects how many
     concurrent sequences the engine can handle.

     These three operations determine the fundamental performance
     characteristics:

     *  TTFT depends primarily on prefill time plus any queue wait

     *  ITL depends on decode time per token

     *  Maximum concurrency depends on KV cache capacity

     *  Throughput depends on batching efficiency during decode

  4.1.2.  Boundary Specification

     Included in SUT:

     *  Model weights and inference kernels

     *  Prefill and decode computation

     *  Batch formation and scheduling logic

     *  KV cache allocation, eviction, and swapping



  Gaikwad                   Expires 24 July 2026                  [Page 9]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     *  Speculative decoding (if enabled)

     *  Quantization and precision handling

     Excluded from SUT:

     *  Network transport beyond local interface

     *  Authentication and authorization

     *  Policy enforcement and content inspection

     *  Request routing between multiple engines

     *  Protocol translation (handled by gateway)

  4.1.3.  Architecture Variants

     Model Engines exist in several architectural configurations that
     affect measurement interpretation.

  4.1.3.1.  Monolithic Architecture

     Prefill and decode execute on the same hardware.  This is the
     simplest configuration and the most common in single-GPU deployments.

                 MONOLITHIC ENGINE

     +------------------------------------------+
     |               GPU / Accelerator          |
     |                                          |
     |  Request -> [Prefill] -> [Decode] -> Out |
     |               |            |             |
     |            KV Cache <------+             |
     |                                          |
     +------------------------------------------+

     Timeline for single request:
     |---- Queue ----|---- Prefill ----|---- Decode (N tokens) ----|
           t6             t6a -> t7            t7 -> t8

                    Figure 2: Monolithic Engine Architecture

     Timestamp mapping:







  Gaikwad                   Expires 24 July 2026                 [Page 10]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


         +========+==================================================+
         | Symbol | Event                                            |
         +========+==================================================+
         | t6     | Request enters engine queue                      |
         +--------+--------------------------------------------------+
         | t6a    | Prefill computation begins (batch slot acquired) |
         +--------+--------------------------------------------------+
         | t7     | First output token generated                     |
         +--------+--------------------------------------------------+
         | t8     | Last output token generated                      |
         +--------+--------------------------------------------------+

                                    Table 5

     Derived metrics:

     Queue residence = t6a - t6
     Prefill latency = t7 - t6a
     Engine TTFT = t7 - t6
     Generation time = t8 - t7

  4.1.3.2.  Disaggregated Architecture

     Prefill and decode execute on separate hardware pools.  Prefill nodes
     are optimized for compute throughput; decode nodes are optimized for
     memory bandwidth.  After prefill completes, the KV cache must
     transfer across the network to the decode pool.

     This architecture appears in published systems including DistServe
     [DISTSERVE] and Mooncake [MOONCAKE], and in open-source projects such
     as llm-d.




















  Gaikwad                   Expires 24 July 2026                 [Page 11]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


                         DISAGGREGATED SERVING

     +------------------+                      +------------------+
     |  PREFILL POOL    |                      |  DECODE POOL     |
     |                  |                      |                  |
     |  High compute    |    KV Cache          |  High memory BW  |
     |  utilization     |    Transfer          |  utilization     |
     |                  |  ================>   |                  |
     |  +-----------+   |   Network link       |  +-----------+   |
     |  | GPU 0     |   |   (RDMA or TCP)      |  | GPU 0     |   |
     |  +-----------+   |                      |  +-----------+   |
     |  | GPU 1     |   |   Bottleneck at      |  | GPU 1     |   |
     |  +-----------+   |   high context       |  +-----------+   |
     |  | ...       |   |   lengths            |  | ...       |   |
     |  +-----------+   |                      |  +-----------+   |
     +------------------+                      +------------------+

     Timeline:
     |-- Queue --|-- Prefill --|-- KV Transfer --|-- Decode --|
          t6          t6a            t7a              t7 -> t8

                  Figure 3: Disaggregated Serving Architecture

     The KV transfer phase (t7a) does not exist in monolithic deployments.
     This phase can become the bottleneck for long contexts.

     KV Transfer Constraint:

     Transfer time depends on context length and network bandwidth:

  KV_transfer_time = (context_length * kv_bytes_per_token) / effective_bandwidth

     Where:

     *  context_length = input tokens processed

     *  kv_bytes_per_token = 2 * num_layers * head_dim * num_heads *
        bytes_per_element

     *  effective_bandwidth = min(network_bandwidth, memory_bandwidth) *
        efficiency

     Bandwidth Saturation Threshold: The context length at which KV
     transfer time exceeds prefill compute time.  Beyond this threshold,
     adding more prefill compute does not reduce TTFT.






  Gaikwad                   Expires 24 July 2026                 [Page 12]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Configuration:
       Model: 70B parameters
       KV cache: 80 layers, 128 heads, 128 dim, BF16
       KV bytes per token: 2 * 80 * 128 * 128 * 2 = 5.24 MB
       Inter-pool bandwidth: 400 Gbps = 50 GB/s effective

     At 4K context:
       KV transfer = 4096 * 5.24 MB / 50 GB/s = 430 ms

     At 32K context:
       KV transfer = 32768 * 5.24 MB / 50 GB/s = 3.44 s

     If prefill compute takes 500ms for 32K tokens:
       Bottleneck shifts to KV transfer at ~4.8K tokens

                   Figure 4: KV Transfer Example Calculation

     Testers benchmarking disaggregated architectures MUST report:

            +======================+=============================+
            | Parameter            | Description                 |
            +======================+=============================+
            | Pool configuration   | Number and type of prefill  |
            |                      | vs decode accelerators      |
            +----------------------+-----------------------------+
            | KV transfer          | RDMA, TCP, or other;        |
            | mechanism            | theoretical bandwidth       |
            +----------------------+-----------------------------+
            | KV bytes per token   | Calculated from model       |
            |                      | architecture                |
            +----------------------+-----------------------------+
            | Observed transfer    | Measured, not calculated    |
            | latency              |                             |
            +----------------------+-----------------------------+
            | Bandwidth saturation | Context length where        |
            | threshold            | transfer becomes bottleneck |
            +----------------------+-----------------------------+
            | TTFT boundary        | Whether reported TTFT       |
            |                      | includes KV transfer        |
            +----------------------+-----------------------------+

                                   Table 6

     Results from disaggregated and monolithic deployments MUST NOT be
     directly compared without explicit architectural notation.






  Gaikwad                   Expires 24 July 2026                 [Page 13]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


             MONOLITHIC                         DISAGGREGATED

     +----------------------+      +-----------+    +-----------+
     |     Single Pool      |      |  Prefill  |    |  Decode   |
     |                      |      |   Pool    |    |   Pool    |
     |  Prefill --> Decode  |      |           |    |           |
     |      |         |     |      | +-------+ |    | +-------+ |
     |      +> KV Cache <---+      | | GPU 0 | |    | | GPU 0 | |
     |                      |      | +-------+ |    | +-------+ |
     |  Same memory space   |      | +-------+ |    | +-------+ |
     |  No transfer needed  |      | | GPU 1 |======>| GPU 1 | |
     |                      |      | +-------+ |    | +-------+ |
     +----------------------+      |   KV transfer  |           |
                                   +-----------+    +-----------+

     TTFT = Queue + Prefill        TTFT = Queue + Prefill + KV_Transfer

     Best for:                     Best for:
     - Smaller models              - Large models (70B+)
     - Lower latency               - Higher throughput
     - Simpler deployment          - Independent scaling

                Figure 5: Monolithic vs Disaggregated Comparison

  4.1.3.3.  Distributed Architecture

     Model sharded across multiple accelerators using tensor parallelism
     (TP), pipeline parallelism (PP), or expert parallelism (EP for
     mixture-of-experts models).

     Testers MUST report:

     *  Parallelism strategy and degree (e.g., TP=8, PP=2)

     *  Interconnect type (NVLink, PCIe, InfiniBand)

     *  Collective communication overhead if measurable

  4.1.4.  Configuration Disclosure

     Testers MUST disclose:










  Gaikwad                   Expires 24 July 2026                 [Page 14]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


       +=================+=======================+=====================+
       | Configuration   | Example Values        | Why It Matters      |
       +=================+=======================+=====================+
       | Model precision | FP16, BF16, INT8, FP8 | Affects throughput, |
       |                 |                       | memory, and quality |
       +-----------------+-----------------------+---------------------+
       | Quantization    | GPTQ, AWQ,            | Different speed/    |
       | method          | SmoothQuant           | quality tradeoffs   |
       +-----------------+-----------------------+---------------------+
       | Batch strategy  | Static, continuous,   | Affects latency     |
       |                 | chunked prefill       | distribution        |
       +-----------------+-----------------------+---------------------+
       | Max batch size  | 64 requests           | Limits concurrency  |
       +-----------------+-----------------------+---------------------+
       | Max sequence    | 8192 tokens           | Limits context      |
       | length          |                       | window              |
       +-----------------+-----------------------+---------------------+
       | KV cache memory | 24 GB                 | Limits concurrent   |
       |                 |                       | sequences           |
       +-----------------+-----------------------+---------------------+

                                    Table 7

  4.1.4.1.  Speculative Decoding

     Speculative decoding uses a smaller draft model to propose multiple
     tokens, then verifies them in parallel with the target model.  When
     draft tokens are accepted, generation is faster.  When rejected,
     compute is wasted.

     If speculative decoding is enabled, testers MUST report:

      +========================+========================================+
      | Parameter              | Description                            |
      +========================+========================================+
      | Draft model            | Identifier and parameter count         |
      +------------------------+----------------------------------------+
      | Speculation window (k) | Tokens proposed per verification step  |
      +------------------------+----------------------------------------+
      | Acceptance rate        | Fraction of draft tokens accepted      |
      +------------------------+----------------------------------------+
      | Verification overhead  | Latency when draft tokens are rejected |
      +------------------------+----------------------------------------+

                                    Table 8

     Acceptance rate directly affects efficiency:




  Gaikwad                   Expires 24 July 2026                 [Page 15]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     High acceptance (80%), k=5:
       Expected accepted per step = 4 tokens
       Verification passes per output token = 0.25

     Low acceptance (30%), k=5:
       Expected accepted per step = 1.5 tokens
       Verification passes per output token = 0.67

     Result: 2.7x more verification overhead at low acceptance

     Results with speculative decoding MUST be labeled separately and
     include observed acceptance rate.

  4.1.4.2.  Chunked Prefill

     Chunked prefill splits long prompts into smaller pieces, processing
     each chunk and potentially interleaving with decode iterations from
     other requests.  This reduces head-of-line blocking but increases
     total prefill time for the chunked request.

     If chunked prefill is enabled, testers MUST report:

     *  Chunk size in tokens

     *  Whether chunks interleave with other requests

     *  Impact on TTFT for long prompts

  4.1.5.  Primary Metrics

     From [I-D.gaikwad-llm-benchmarking-terminology]:

     *  Time to First Token (TTFT)

     *  Inter-Token Latency (ITL)

     *  Time per Output Token (TPOT)

     *  Output Token Throughput

  4.1.6.  Secondary Metrics

     *  Request Throughput

     *  Queue Depth over time

     *  Queue Residence Time




  Gaikwad                   Expires 24 July 2026                 [Page 16]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     *  Prefill Latency (TTFT minus queue residence)

     *  Batch Utilization

  4.1.7.  Benchmarking Constraints

     Request rate saturation differs from token saturation.  A system
     might handle 2000 output tokens per second but only 50 requests per
     second if scheduling overhead dominates.  Testers SHOULD measure both
     dimensions.

     Mixed-length workloads increase tail latency under continuous
     batching.  Short requests arriving behind long prefills experience
     head-of-line blocking.  When workload includes high length variance,
     measure fairness: the ratio of actual latency to expected latency
     based on request size.

  4.2.  AI Gateway Profile

  4.2.1.  Definition and Concepts

     An AI Gateway is a network-facing intermediary that virtualizes
     access to one or more Model Engines.  Gateways handle cross-cutting
     concerns that do not belong in the inference engine itself.

     Gateways perform several functions that affect latency:

     *Request Processing:* TLS termination, authentication, schema
     validation, and protocol translation.  These operations add fixed
     overhead per request.

     *Routing:* Selection of backend engine based on load, capability, or
     policy.  Intelligent routing (e.g., KV-cache-aware) adds decision
     latency but may reduce overall latency by improving cache hit rates.

     *Caching:* Gateways may implement response caching.  Traditional
     exact-match caching has limited utility for LLM traffic due to low
     query repetition.  Semantic caching (matching similar queries)
     improves hit rates but introduces quality risk from approximate
     matches.

     *Admission Control:* Rate limiting and quota enforcement.  Under
     load, admission control adds queuing delay or rejects requests.

  4.2.2.  Boundary Specification

     Included in SUT:




  Gaikwad                   Expires 24 July 2026                 [Page 17]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     *  TLS termination

     *  Authentication and authorization

     *  Schema validation and protocol translation

     *  Load balancing across engines or model replicas

     *  Semantic cache lookup and population

     *  Admission control and rate limiting

     *  Retry and fallback logic

     *  Response normalization

     Excluded from SUT:

     *  Model inference computation (handled by downstream engine)

     *  Content inspection for safety (handled by AI Firewall)

  4.2.3.  Baseline Requirement

     Gateway overhead is meaningful only relative to direct engine access.
     Gateway benchmarks MUST declare measurement type:

            +==================+=================================+
            | Measurement Type | What It Includes                |
            +==================+=================================+
            | Aggregate        | Gateway processing plus         |
            |                  | downstream engine latency       |
            +------------------+---------------------------------+
            | Differential     | Gateway overhead only, relative |
            |                  | to direct engine access         |
            +------------------+---------------------------------+

                                   Table 9

     To measure differential latency:

     1.  Benchmark the Model Engine directly (baseline)

     2.  Benchmark through the Gateway to the same engine (same workload,
         same conditions)

     3.  Compute delta: Gateway_overhead = Gateway_TTFT - Engine_TTFT




  Gaikwad                   Expires 24 July 2026                 [Page 18]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Report both absolute values and delta.

  4.2.4.  Load Balancing Disclosure

     Load balancing strategy affects tail latency.  Testers MUST report:

      +===============+=================================+==============+
      | Configuration | Options                         | Impact       |
      +===============+=================================+==============+
      | Algorithm     | Round-robin, least-connections, | Tail latency |
      |               | weighted, adaptive              | variance     |
      +---------------+---------------------------------+--------------+
      | Health checks | Interval, timeout, failure      | Failover     |
      |               | threshold                       | speed        |
      +---------------+---------------------------------+--------------+
      | Sticky        | Enabled/disabled, key type      | Cache        |
      | sessions      |                                 | locality     |
      +---------------+---------------------------------+--------------+
      | Retry policy  | Max retries, backoff strategy   | Failure      |
      |               |                                 | handling     |
      +---------------+---------------------------------+--------------+

                                   Table 10

     For intelligent routing (KV-cache-aware, cost-optimized, latency-
     optimized):

     *  Routing signals used (queue depth, cache locality, model cost)

     *  Decision latency overhead

     *  Routing effectiveness (e.g., cache hit improvement from routing)

  4.2.5.  Multi-Model Gateway

     Modern gateways route to multiple backend models based on capability,
     cost, or latency.

     When gateway routes to heterogeneous backends, testers MUST report:

     *  Model selection logic: Rule-based, cost-optimized, capability-
        based

     *  Backend composition: List of models and their roles

     *  Fallback behavior: Conditions triggering model switching

     Per-model metrics SHOULD be reported separately.



  Gaikwad                   Expires 24 July 2026                 [Page 19]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Cross-gateway comparison requires backend normalization.  Comparing
     Gateway A (routing to GPT-4) against Gateway B (routing to Llama-70B)
     conflates gateway performance with model performance.

  4.2.6.  Semantic Cache

     Semantic caching matches queries by meaning rather than exact text.
     A cache hit on "What is the capital of France?" might serve a
     response cached from "France's capital city?"  This improves hit
     rates but risks serving inappropriate responses for queries that are
     similar but not equivalent.

     Configuration Disclosure:

        +============+========================+=======================+
        | Parameter  | Example                | Why It Matters        |
        +============+========================+=======================+
        | Similarity | Cosine >= 0.92         | Lower threshold: more |
        | threshold  |                        | hits, more mismatches |
        +------------+------------------------+-----------------------+
        | Embedding  | text-embedding-3-small | Affects similarity    |
        | model      |                        | quality               |
        +------------+------------------------+-----------------------+
        | Cache      | 100,000 entries        | Hit rate ceiling      |
        | capacity   |                        |                       |
        +------------+------------------------+-----------------------+
        | Eviction   | LRU, frequency-based   | Long-term hit rate    |
        | policy     |                        |                       |
        +------------+------------------------+-----------------------+
        | Cache      | Global, per-tenant,    | Security and hit rate |
        | scope      | per-user               | tradeoff              |
        +------------+------------------------+-----------------------+
        | TTL        | 1 hour                 | Staleness vs hit rate |
        +------------+------------------------+-----------------------+

                                    Table 11

     Required Metrics:













  Gaikwad                   Expires 24 July 2026                 [Page 20]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


      +=======================+========================================+
      | Metric                | Definition                             |
      +=======================+========================================+
      | Hit rate              | Fraction of requests served from cache |
      +-----------------------+----------------------------------------+
      | Hit rate distribution | P50, P95, P99 of per-session hit rates |
      +-----------------------+----------------------------------------+
      | Latency on hit        | TTFT when cache serves response        |
      +-----------------------+----------------------------------------+
      | Latency on miss       | TTFT when engine generates             |
      +-----------------------+----------------------------------------+
      | Cache delta           | Latency_miss minus Latency_hit         |
      +-----------------------+----------------------------------------+
      | Mismatch rate         | Fraction of hits where cached response |
      |                       | was inappropriate                      |
      +-----------------------+----------------------------------------+

                                   Table 12

     Mismatch rate requires evaluation.  Testers SHOULD disclose
     evaluation methodology (human review, automated comparison, or LLM-
     as-judge).

     Session Definition: For per-session metrics, define what constitutes
     a session: requests sharing a session identifier, requests from the
     same user within a time window, or another definition.  Testers MUST
     disclose session definition.

     Staleness in RAG Systems: When semantic cache operates with a RAG
     system, cached responses may reference documents that have since been
     updated.

      +========================+========================================+
      | Parameter              | Description                            |
      +========================+========================================+
      | Index update frequency | How often RAG index refreshes          |
      +------------------------+----------------------------------------+
      | Cache TTL              | Maximum age of cached entries          |
      +------------------------+----------------------------------------+
      | Staleness risk         | Estimated fraction of stale cache hits |
      +------------------------+----------------------------------------+

                                    Table 13

     Staleness risk estimate:

  staleness_risk = (average_cache_age / index_update_interval) * corpus_change_rate




  Gaikwad                   Expires 24 July 2026                 [Page 21]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Benchmarking Constraints: Workload diversity determines hit rate.
     Testers MUST report:

     *  Number of distinct query clusters in workload

     *  Cache state at test start (cold or warm)

     *  Time until hit rate stabilizes

  4.3.  AI Firewall Profile

  4.3.1.  Definition and Concepts

     An AI Firewall is a bidirectional security intermediary that inspects
     LLM inputs and outputs to detect and prevent policy violations.

     Unlike traditional firewalls that examine packet headers or match
     byte patterns, AI Firewalls analyze semantic content.  They must
     understand what a prompt is asking and what a response is saying.
     This requires ML models, making firewall latency fundamentally
     different from network firewall latency.

     The firewall sits on the request path and adds latency to every
     request.  The core tradeoff: more thorough inspection catches more
     threats but costs more time.

  4.3.2.  Boundary Specification

     Included in SUT:

     *  Prompt analysis and classification

     *  Output content inspection

     *  Policy decision engine

     *  Block, allow, or modify actions

     Excluded from SUT:

     *  Model inference (upstream or downstream)

     *  Network-layer firewalling (traditional WAF)

     *  Authentication (handled by gateway)






  Gaikwad                   Expires 24 July 2026                 [Page 22]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  4.3.3.  Enforcement Directions

     AI Firewalls operate bidirectionally.  Each direction addresses
     different threats.

     Inbound Enforcement inspects user prompts before they reach the
     model:

            +=========================+===========================+
            | Threat                  | Description               |
            +=========================+===========================+
            | Direct prompt injection | User attempts to override |
            |                         | system instructions       |
            +-------------------------+---------------------------+
            | Indirect prompt         | Malicious content in      |
            | injection               | retrieved documents       |
            +-------------------------+---------------------------+
            | Jailbreak attempts      | Techniques to bypass      |
            |                         | model safety training     |
            +-------------------------+---------------------------+
            | Context poisoning       | Adversarial content to    |
            |                         | manipulate model behavior |
            +-------------------------+---------------------------+

                                    Table 14

     Outbound Enforcement inspects model outputs before they reach the
     user:

        +===================+=========================================+
        | Threat            | Description                             |
        +===================+=========================================+
        | PII leakage       | Model outputs personal information      |
        +-------------------+-----------------------------------------+
        | Policy violation  | Output violates content policies        |
        +-------------------+-----------------------------------------+
        | Tool misuse       | Model attempts unauthorized actions     |
        +-------------------+-----------------------------------------+
        | Data exfiltration | Sensitive information encoded in output |
        +-------------------+-----------------------------------------+

                                    Table 15

     Testers MUST declare which directions are enforced.  A benchmark
     testing inbound-only enforcement MUST NOT claim protection against
     outbound threats.





  Gaikwad                   Expires 24 July 2026                 [Page 23]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  4.3.4.  Inspection Architecture

     Firewalls use different inspection strategies with distinct latency
     characteristics.

     INSPECTION ARCHITECTURE COMPARISON

     BUFFERED (adds to TTFT, no ITL impact):

       Input:   ================........................
                |-- collect --|-- analyze --|-- forward -->

       Output:  ..................========........
                |- engine generates -|- buffer -|- analyze -|-->
                                     t7                     t12
                                     |-- inspection delay --|

     STREAMING (no TTFT impact, adds to ITL):

       Output:  ..o..o..o..o..o..o..o..o..o..o..o..o..o..o..o.
                   |     |     |     |     |     |     |
                inspect  |  inspect  |  inspect  |  inspect
                   |     |     |     |     |     |     |
                --o-----o-----o-----o-----o-----o-----o----->
                Variable delays, increased jitter

                  Figure 6: Inspection Architecture Comparison

     Buffered Inspection: The firewall collects complete input (or output)
     before analysis.

     Characteristics:

     *  Adds to TTFT (inbound) or delays token delivery (outbound)

     *  No impact on ITL once streaming starts

     *  Enables deep analysis requiring full context

     For outbound buffered inspection, the client receives the first token
     later than the engine generates it.  This distinction matters:

     Engine TTFT (t7 - t6): 200ms
     Outbound inspection: 50ms
     Client-observed TTFT (t12 - t0): 250ms + network

     Streaming Inspection: The firewall analyzes content as tokens flow
     through.



  Gaikwad                   Expires 24 July 2026                 [Page 24]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Characteristics:

     *  Adds per-token overhead to ITL

     *  May batch or pause tokens during analysis

     *  Introduces jitter in token delivery

     Required measurements:

     +============================+======================================+
     | Metric                     | Definition                           |
     +============================+======================================+
     | Per-token inspection delay | Average latency added per token      |
     +----------------------------+--------------------------------------+
     | Maximum pause duration     | Longest delay during streaming       |
     +----------------------------+--------------------------------------+
     | Pause frequency            | How often inspection causes          |
     |                            | batching                             |
     +----------------------------+--------------------------------------+
     | Jitter contribution        | Standard deviation of delays         |
     +----------------------------+--------------------------------------+

                                    Table 16

     Hybrid Inspection: Initial buffering followed by streaming.  Common
     pattern: buffer first N tokens for context, then stream with spot-
     checks.

     Configuration to disclose:

     *  Buffer threshold (tokens before streaming starts)

     *  Spot-check frequency

     *  Escalation triggers (patterns that switch to full buffering)

  4.3.5.  Required Metrics

     Accuracy Metrics:











  Gaikwad                   Expires 24 July 2026                 [Page 25]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


      +================+================================================+
      | Metric         | Definition                                     |
      +================+================================================+
      | Detection Rate | Fraction of malicious inputs correctly blocked |
      +----------------+------------------------------------------------+
      | False Positive | Fraction of benign inputs blocked by firewall  |
      | Rate (FPR)     |                                                |
      +----------------+------------------------------------------------+
      | False Refusal  | Fraction of policy-compliant requests refused  |
      | Rate (FRR)     | at system boundary                             |
      +----------------+------------------------------------------------+
      | Over-Defense   | FPR conditional on trigger-word presence in    |
      | Rate           | benign inputs                                  |
      +----------------+------------------------------------------------+

                                    Table 17

     FPR vs FRR: FPR measures firewall classifier errors on a benign test
     set.  FRR measures all refusals observed at the system boundary,
     which may include:

     *  Firewall blocks (captured in FPR)

     *  Model refusals (model's own safety behavior)

     *  Policy engine blocks (business rules)

     *  Rate limiting (capacity rejection)

     Therefore: FRR >= FPR when other refusal sources exist.

     When reporting both, attribute refusals by source when possible.

     Over-Defense Rate: Measures false positives on benign inputs that
     contain words commonly associated with attacks.

     Over-Defense Rate = P(Block | Benign AND Contains_Trigger_Words)

     Examples of benign inputs that may trigger over-defense:

     *  "Explain how prompt injection attacks work" (security education)

     *  "What does 'ignore previous instructions' mean?" (linguistic
        question)

     *  "How do I kill a process in Linux?" (technical query)





  Gaikwad                   Expires 24 July 2026                 [Page 26]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     The test corpus for over-defense MUST contain semantically benign
     inputs that happen to include trigger words.  Testing with trivially
     benign inputs does not measure real over-defense risk.

     Latency Metrics:

      +========================+=======================================+
      | Metric                 | Definition                            |
      +========================+=======================================+
      | Passing latency        | Overhead when firewall allows request |
      +------------------------+---------------------------------------+
      | Blocking latency       | Time to reach block decision          |
      +------------------------+---------------------------------------+
      | Throughput degradation | Reduction in requests per second      |
      +------------------------+---------------------------------------+

                                   Table 18

     Latency may vary by decision path:

     Example:
       Allow (no flags): 8ms
       Allow (flagged, deep analysis): 45ms
       Block (pattern match): 3ms
       Block (semantic analysis): 67ms

     Report latency distribution by decision type.

  4.3.6.  Workload Specification

     AI Firewall benchmarks require careful workload design.

     Benign Workload: Normal traffic with no policy violations.  Measures
     passing latency, FRR, and throughput impact on legitimate use.
     Source: Sanitized production samples or standard datasets.

     Adversarial Workload: Known attack patterns.  Measures detection
     rate, blocking latency, and FPR.  Source: Published datasets (BIPIA
     [BIPIA], JailbreakBench, PromptInject) or red team generated.  Do not
     publish working exploits.

     Mixed Workload (recommended): Combines benign and adversarial at
     declared ratio.








  Gaikwad                   Expires 24 July 2026                 [Page 27]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


      +========================+=======================================+
      | Parameter              | Example                               |
      +========================+=======================================+
      | Mix ratio              | 95% benign, 5% adversarial            |
      +------------------------+---------------------------------------+
      | Adversarial categories | 40% injection, 30% jailbreak, 30% PII |
      +------------------------+---------------------------------------+
      | Arrival pattern        | Uniform or bursty                     |
      +------------------------+---------------------------------------+

                                   Table 19

  4.3.7.  Multi-Layer Firewall

     Production deployments often stack multiple inspection layers.

  Request -> Quick Filter -> ML Classifier -> Model -> Semantic Check -> PII Scan -> Response
                |               |                        |              |
             regex           embedding               output           entity
             + rules         classifier             analysis         detection

     When multiple layers exist, report:

     *  Number and position of layers

     *  Per-layer latency

     *  Execution model: Series (latencies add) or parallel

     *  Short-circuit behavior: Does blocking at layer N skip later
        layers?

     Delta decomposition:

  Total overhead = Quick(2ms) + ML(12ms) + Semantic(34ms) + PII(8ms) = 56ms

  With short-circuit on input block:
    Overhead = Quick(2ms) + ML(12ms) = 14ms

  4.3.8.  Benchmarking Constraints

     Blocking speed alone is meaningless.  A firewall blocking all
     requests in 1ms is useless.  Always measure impact on benign traffic
     alongside detection effectiveness.

     Disclose integration with WAF, rate limiting, or DDoS protection.
     These add latency.




  Gaikwad                   Expires 24 July 2026                 [Page 28]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Different attack categories have different detection latencies.
     Pattern-based detection is faster than semantic analysis.  Report
     detection latency by category.

  4.4.  Compound System Profile

  4.4.1.  Definition and Concepts

     A Compound System executes multiple inference, retrieval, and tool-
     use steps to satisfy a user intent.  The system orchestrates these
     steps, manages state across them, and produces a final response.

     Examples: RAG pipelines, multi-agent systems, tool-using assistants,
     coding agents.

     Unlike single-inference benchmarks, compound system benchmarks
     measure task completion, not token generation.  The primary question
     is "Did it accomplish the goal?" not "How fast did it generate
     tokens?"

  4.4.2.  Boundary Specification

     Included in SUT:

     *  Orchestration and planning logic

     *  Multiple LLM inference calls

     *  Retrieval pipeline (embedding, search, reranking)

     *  Tool execution environment

     *  Conversation state management

     *  Agent-to-agent communication

     Excluded from SUT:

     *  External APIs outside the system boundary (latency measured but
        not controlled)

     *  User interface rendering

     *  Arbitrary user-supplied code







  Gaikwad                   Expires 24 July 2026                 [Page 29]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Boundary Rule: The Compound System boundary includes only components
     deployed and controlled as part of the serving system.  User-provided
     plugins or custom code at runtime are excluded.  This prevents
     ambiguity when comparing systems with different extensibility models.

        +========================+===========+========================+
        | Component              | Included? | Rationale              |
        +========================+===========+========================+
        | Built-in retrieval     | Yes       | Part of serving system |
        +------------------------+-----------+------------------------+
        | Standard tool library  | Yes       | Shipped with system    |
        +------------------------+-----------+------------------------+
        | User-uploaded plugin   | No        | User-supplied          |
        +------------------------+-----------+------------------------+
        | External API (weather) | Latency   | Outside boundary       |
        |                        | measured  |                        |
        +------------------------+-----------+------------------------+

                                    Table 20

  4.4.3.  Primary Metrics

             +=========================+========================+
             | Metric                  | Definition             |
             +=========================+========================+
             | Task Completion Latency | Time from user request |
             |                         | to final response      |
             +-------------------------+------------------------+
             | Task Success Rate       | Fraction of tasks      |
             |                         | completed correctly    |
             +-------------------------+------------------------+

                                   Table 21

     Task Success has two dimensions:

         +==============+========================+==================+
         | Type         | Definition             | Evaluation       |
         +==============+========================+==================+
         | Hard Success | Structural correctness | Automated (valid |
         |              |                        | JSON, no errors) |
         +--------------+------------------------+------------------+
         | Soft Success | Semantic correctness   | Requires         |
         |              |                        | evaluation       |
         +--------------+------------------------+------------------+

                                   Table 22




  Gaikwad                   Expires 24 July 2026                 [Page 30]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  4.4.4.  Evaluation Oracle

     When using automated evaluation for Task Success Rate, disclose
     oracle methodology.

     LLM-as-Judge:

      +=====================+===========================================+
      | Parameter           | Report                                    |
      +=====================+===========================================+
      | Judge model         | Identifier and version                    |
      +---------------------+-------------------------------------------+
      | Judge prompt        | Full prompt or published rubric reference |
      +---------------------+-------------------------------------------+
      | Ground truth access | Whether judge sees reference answers      |
      +---------------------+-------------------------------------------+
      | Sampling            | Temperature, judgments per task           |
      +---------------------+-------------------------------------------+

                                    Table 23

     Report inter-rater agreement if using multiple judges.

     Rule-Based Evaluation:

     +====================+==============================================+
     | Parameter          | Report                                       |
     +====================+==============================================+
     | Rule specification | Formal definition                            |
     +--------------------+----------------------------------------------+
     | Coverage           | Fraction of criteria                         |
     |                    | that are rule-checkable                      |
     +--------------------+----------------------------------------------+
     | Edge case handling | How ambiguous cases                          |
     |                    | resolve                                      |
     +--------------------+----------------------------------------------+

                                    Table 24

     Human Evaluation:











  Gaikwad                   Expires 24 July 2026                 [Page 31]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


      +=================+===============================================+
      | Parameter       | Report                                        |
      +=================+===============================================+
      | Evaluator count | Number of humans                              |
      +-----------------+-----------------------------------------------+
      | Rubric          | Criteria and scoring                          |
      +-----------------+-----------------------------------------------+
      | Agreement       | Inter-rater reliability (e.g., Cohen's Kappa) |
      +-----------------+-----------------------------------------------+
      | Blinding        | Whether evaluators knew system identity       |
      +-----------------+-----------------------------------------------+

                                    Table 25

  4.4.5.  Secondary Metrics

          +=====================+===================================+
          | Metric              | Definition                        |
          +=====================+===================================+
          | Trace Depth         | Sequential steps in execution     |
          +---------------------+-----------------------------------+
          | Fan-out Factor      | Maximum parallel sub-requests     |
          +---------------------+-----------------------------------+
          | Sub-Request Count   | Total LLM calls per user request  |
          +---------------------+-----------------------------------+
          | Loop Incidence Rate | Fraction of tasks with repetitive |
          |                     | non-progressing actions           |
          +---------------------+-----------------------------------+
          | Stalled Task Rate   | Fraction of tasks hitting step    |
          |                     | limit without resolution          |
          +---------------------+-----------------------------------+
          | State Management    | Latency and memory for multi-turn |
          | Overhead            | context                           |
          +---------------------+-----------------------------------+

                                    Table 26

     Stalled Task Rate:

     Stalled Task Rate = Tasks_reaching_max_steps / Total_tasks

     Stalled tasks differ from loops.  A loop repeats similar actions.  A
     stalled task may try diverse actions but fail to converge.  Both
     indicate problems but different ones.







  Gaikwad                   Expires 24 July 2026                 [Page 32]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  4.4.6.  RAG Sub-Profile

     When Compound System includes Retrieval-Augmented Generation:

                             RAG PIPELINE LATENCY

         Query --> Embed --> Search --> Rerank --> Inject --> Generate
           |         |          |          |          |          |
           Q         E          S          R          I          G
           |         |          |          |          |          |
         ----------------------------------------------------------->
         0ms      15ms       60ms       180ms      185ms      385ms
                   |          |          |          |          |
                   +--15ms----+          |          |          |
                        +-----45ms-------+          |          |
                                   +-----120ms------+          |
                                             +--5ms-+          |
                                                   +--200ms----+

         TTFT = E + S + R + I + Prefill + Queue = 385ms

                         Figure 7: RAG Pipeline Latency

     Configuration Disclosure:

            +==============+=====================================+
            | Component    | Parameters                          |
            +==============+=====================================+
            | Embedding    | Model, dimensions, batch size       |
            +--------------+-------------------------------------+
            | Vector store | Type, index configuration           |
            +--------------+-------------------------------------+
            | Search       | Top-k, similarity metric, filters   |
            +--------------+-------------------------------------+
            | Reranking    | Model (if used), top-n after rerank |
            +--------------+-------------------------------------+
            | Context      | Max tokens, formatting template     |
            +--------------+-------------------------------------+

                                   Table 27

     RAG-Specific Metrics:









  Gaikwad                   Expires 24 July 2026                 [Page 33]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     +============================+=====================================+
     | Metric                     | Definition                          |
     +============================+=====================================+
     | Embedding Latency          | Query to vector conversion          |
     +----------------------------+-------------------------------------+
     | Retrieval Latency          | Search and fetch time               |
     +----------------------------+-------------------------------------+
     | Retrieval Recall           | Fraction of relevant docs retrieved |
     +----------------------------+-------------------------------------+
     | Context Injection Overhead | Additional prefill from retrieved   |
     |                            | content                             |
     +----------------------------+-------------------------------------+

                                   Table 28

     Corpus Constraints:

          +====================+====================================+
          | Characteristic     | Impact                             |
          +====================+====================================+
          | Corpus size        | Larger means longer search         |
          +--------------------+------------------------------------+
          | Document length    | Longer means more context overhead |
          +--------------------+------------------------------------+
          | Semantic diversity | More diverse reduces precision     |
          +--------------------+------------------------------------+

                                    Table 29

     Report corpus statistics: document count, average length, domain.

     Vector index must be fully built before measurement.

  4.4.7.  Agentic System Boundaries

     For multi-agent or tool-using systems:















  Gaikwad                   Expires 24 July 2026                 [Page 34]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


                         AGENTIC EXECUTION TRACE

     User Request
          |
          v
     +---------+     +---------+     +---------+
     | Planner |---->| Agent A |---->| Agent B |
     |  (LLM)  |     |  (LLM)  |     |  (LLM)  |
     +----+----+     +----+----+     +----+----+
          |               |               |
          |          +----+----+          |
          |          v         v          |
          |     +-------+ +-------+       |
          |     | Tool  | | Tool  |       |
          |     |  API  | |  DB   |       |
          |     +-------+ +-------+       |
          |          |         |          |
          |          +----+----+          |
          |               v               |
          |          +---------+          |
          +--------->|  Final  |<---------+
                     | Response|
                     +---------+

     Trace depth: 4 (Planner -> A -> Tools -> B)
     Fan-out: 2 (parallel tool calls)
     Sub-requests: 3 LLM calls

                       Figure 8: Agentic Execution Trace

     Definitions:

        +====================+========================================+
        | Term               | Definition                             |
        +====================+========================================+
        | Agent invocation   | Single LLM call with specific role     |
        +--------------------+----------------------------------------+
        | Tool call          | External capability invocation         |
        +--------------------+----------------------------------------+
        | Orchestration step | Planning/routing decision              |
        +--------------------+----------------------------------------+
        | Trace              | Complete sequence for one user request |
        +--------------------+----------------------------------------+

                                    Table 30

     Measurement Points:




  Gaikwad                   Expires 24 July 2026                 [Page 35]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     +===================+======================+=======================+
     | Metric            | Start                | End                   |
     +===================+======================+=======================+
     | Per-agent latency | Agent receives input | Agent produces output |
     +-------------------+----------------------+-----------------------+
     | Per-tool latency  | Tool call initiated  | Response received     |
     +-------------------+----------------------+-----------------------+
     | Orchestration     | Previous step        | Next step starts      |
     | overhead          | complete             |                       |
     +-------------------+----------------------+-----------------------+
     | Task completion   | User request         | Final response        |
     |                   | received             | delivered             |
     +-------------------+----------------------+-----------------------+

                                   Table 31

  4.4.8.  Exclusions

     Custom user application logic and bespoke agent frameworks are out of
     scope.  This profile covers general patterns, not specific
     implementations.

  5.  Workload Profiles

     Workload profiles specify traffic patterns applied to infrastructure
     profiles.  They do not define measurement boundaries.

  5.1.  Chatbot Workload Profile

  5.1.1.  Characteristics

         +================+=========================================+
         | Characteristic | Description                             |
         +================+=========================================+
         | Interaction    | Stateful, multi-turn                    |
         +----------------+-----------------------------------------+
         | Delivery       | Streaming                               |
         +----------------+-----------------------------------------+
         | Arrival        | Closed-loop (user thinks between turns) |
         +----------------+-----------------------------------------+
         | Session length | Variable, typically 3-20 turns          |
         +----------------+-----------------------------------------+

                                   Table 32







  Gaikwad                   Expires 24 July 2026                 [Page 36]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  5.1.2.  Required Parameters

     +===============+==========================+========================+
     | Parameter     | Description              | Example                |
     +===============+==========================+========================+
     | Arrival model | Open or closed loop      | Closed-loop            |
     +---------------+--------------------------+------------------------+
     | Think-time    | User delay between turns | Exponential, mean=5s   |
     +---------------+--------------------------+------------------------+
     | Input length  | Tokens per user message  | Log-normal, median=50  |
     +---------------+--------------------------+------------------------+
     | Output length | Tokens per response      | Log-normal,            |
     |               |                          | median=150             |
     +---------------+--------------------------+------------------------+
     | Context       | History handling         | Sliding window, 4K     |
     | retention     |                          | tokens                 |
     +---------------+--------------------------+------------------------+
     | Session       | Turns per conversation   | Geometric, mean=8      |
     | length        |                          |                        |
     +---------------+--------------------------+------------------------+

                                    Table 33

  5.1.3.  Example

     Chatbot Workload: Customer Support

     Arrival: Closed-loop, 100 concurrent sessions
     Think-time: Exponential(mean=8s)
     Input: Log-normal(mu=4.0, sigma=0.8), range [10, 500]
     Output: Log-normal(mu=5.0, sigma=1.0), range [20, 1000]
     Context: Sliding window, last 4000 tokens
     Session: Geometric(p=0.12), mean ~8 turns
     System prompt: 200 tokens, shared

  5.2.  Compound Workflow Workload Profile

  5.2.1.  Characteristics

     +================+=================================================+
     | Characteristic | Description                                     |
     +================+=================================================+
     | Execution      | Multi-step, may include parallel branches       |
     +----------------+-------------------------------------------------+
     | Tool usage     | API calls, code execution, database queries     |
     +----------------+-------------------------------------------------+
     | Dependencies   | Steps may depend on previous outputs            |
     +----------------+-------------------------------------------------+



  Gaikwad                   Expires 24 July 2026                 [Page 37]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     | Failure modes  | Steps may fail, requiring retry or alternatives |
     +----------------+-------------------------------------------------+

                                   Table 34

  5.2.2.  Required Parameters

      +=================+=====================+=========================+
      | Parameter       | Description         | Example                 |
      +=================+=====================+=========================+
      | Task complexity | Steps per task      | Fixed=5 or distribution |
      +-----------------+---------------------+-------------------------+
      | Fan-out pattern | Parallel vs         | Max parallel=3          |
      |                 | sequential          |                         |
      +-----------------+---------------------+-------------------------+
      | Tool latency    | External dependency | Real, mocked, simulated |
      |                 | behavior            |                         |
      +-----------------+---------------------+-------------------------+
      | Failure         | Simulated failures  | 5% tool failure rate    |
      | injection       |                     |                         |
      +-----------------+---------------------+-------------------------+
      | Retry behavior  | Failure handling    | Max 2 retries,          |
      |                 |                     | exponential backoff     |
      +-----------------+---------------------+-------------------------+

                                    Table 35

  5.2.3.  External Dependency Handling

     Compound workflows depend on external systems.  Disclose handling:

         +===========+===================+===========================+
         | Approach  | Description       | When                      |
         +===========+===================+===========================+
         | Real      | Actual API calls  | Production-representative |
         +-----------+-------------------+---------------------------+
         | Mocked    | Fixed responses   | Controlled experiments    |
         +-----------+-------------------+---------------------------+
         | Simulated | Statistical model | Reproducible benchmarks   |
         +-----------+-------------------+---------------------------+

                                    Table 36

     Report observed latency and failure rate for real dependencies.
     Report configured values for mocked dependencies.






  Gaikwad                   Expires 24 July 2026                 [Page 38]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  6.  Delta Measurement Model

     Section 4 and Section 5 defined individual profiles.  Production
     systems compose multiple profiles.  A request may pass through
     Gateway, Firewall, and Engine before response generation.

     Meaningful comparison across composed systems requires attributing
     latency to each component.  This section defines the delta
     measurement model.

  6.1.  Timestamp Reference

     Consider a request flowing through a full stack:






































  Gaikwad                   Expires 24 July 2026                 [Page 39]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Client
       |
       |  t0: request sent
       v
     +-------------------+
     |    AI Gateway     |
     |  t1: arrives      |
     |  t2: exits        |
     +-------------------+
       |
       v
     +-------------------+
     |   AI Firewall     |
     |  t3: arrives      |
     |  t4: decision     |
     |  t5: exits        |
     +-------------------+
       |
       v
     +-------------------+
     |   Model Engine    |
     |  t6: queue entry  |
     |  t6a: exec start  |
     |  t7: first token  |
     |  t8: last token   |
     +-------------------+
       |
       v
     +-------------------+
     |   Output Path     |
     |  t9: fw receives  |
     |  t10: fw releases |
     |  t11: gw releases |
     +-------------------+
       |
       v
       t12: client receives first token

                       Figure 9: Request Flow Timestamps

  6.2.  Timestamp Definitions

           +===========+==========+===============================+
           | Timestamp | Location | Event                         |
           +===========+==========+===============================+
           | t0        | Client   | Request transmission begins   |
           +-----------+----------+-------------------------------+
           | t1        | Gateway  | Request arrives               |



  Gaikwad                   Expires 24 July 2026                 [Page 40]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


           +-----------+----------+-------------------------------+
           | t2        | Gateway  | Request exits toward firewall |
           +-----------+----------+-------------------------------+
           | t3        | Firewall | Request arrives               |
           +-----------+----------+-------------------------------+
           | t4        | Firewall | Inbound decision reached      |
           +-----------+----------+-------------------------------+
           | t5        | Firewall | Request exits toward engine   |
           +-----------+----------+-------------------------------+
           | t6        | Engine   | Request enters queue          |
           +-----------+----------+-------------------------------+
           | t6a       | Engine   | Prefill computation begins    |
           +-----------+----------+-------------------------------+
           | t7        | Engine   | First output token generated  |
           +-----------+----------+-------------------------------+
           | t8        | Engine   | Last output token generated   |
           +-----------+----------+-------------------------------+
           | t9        | Firewall | First token arrives for       |
           |           |          | outbound inspection           |
           +-----------+----------+-------------------------------+
           | t10       | Firewall | First token released after    |
           |           |          | inspection                    |
           +-----------+----------+-------------------------------+
           | t11       | Gateway  | First token exits toward      |
           |           |          | client                        |
           +-----------+----------+-------------------------------+
           | t12       | Client   | Client receives first token   |
           +-----------+----------+-------------------------------+

                                   Table 37

  6.3.  Component Deltas

     +==========================+===========+===========================+
     | Component                | Formula   | Measures                  |
     +==========================+===========+===========================+
     | Gateway inbound          | t2 - t1   | Auth, validation, routing |
     +--------------------------+-----------+---------------------------+
     | Firewall inbound (pass)  | t5 - t3   | Prompt inspection         |
     +--------------------------+-----------+---------------------------+
     | Firewall inbound (block) | t4 - t3   | Time to block             |
     +--------------------------+-----------+---------------------------+
     | Engine queue             | t6a - t6  | Wait before execution     |
     +--------------------------+-----------+---------------------------+
     | Engine prefill           | t7 - t6a  | Prefill computation       |
     +--------------------------+-----------+---------------------------+
     | Engine TTFT              | t7 - t6   | Queue plus prefill        |
     +--------------------------+-----------+---------------------------+



  Gaikwad                   Expires 24 July 2026                 [Page 41]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     | Firewall outbound        | t10 - t9  | Output inspection         |
     +--------------------------+-----------+---------------------------+
     | Gateway outbound         | t11 - t10 | Response processing       |
     +--------------------------+-----------+---------------------------+

                                   Table 38

  6.4.  End-to-End Metrics

         +======================+==========+========================+
         | Metric               | Formula  | Notes                  |
         +======================+==========+========================+
         | Engine TTFT          | t7 - t6  | At engine boundary     |
         +----------------------+----------+------------------------+
         | System TTFT          | t12 - t0 | Client-observed        |
         +----------------------+----------+------------------------+
         | Output path overhead | t12 - t7 | Delay from engine emit |
         |                      |          | to client receive      |
         +----------------------+----------+------------------------+

                                   Table 39

  6.5.  Clock Synchronization

     Delta metrics within a single component (t2 - t1, both from gateway
     clock) are reliable.  Cross-component deltas (t6 - t5) require clock
     synchronization.

     For end-to-end metrics involving client timestamps (t0, t12), clock
     skew introduces error.

     Options:

     1.  Single-machine measurement (client and server share clock)

     2.  Measure and report skew bounds

     3.  Report server-side metrics only when skew is too large

     Recommended practice: Calculate deltas within components rather than
     across boundaries when possible.

     See Section 9.3 for synchronization requirements.

  7.  Profile Composition






  Gaikwad                   Expires 24 July 2026                 [Page 42]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  7.1.  Composite SUT Declaration

     When SUT includes multiple profiles, testers MUST:

     1.  Enumerate all components in request path:

  Client -> AI Gateway -> AI Firewall -> Model Engine -> AI Firewall -> Client

     2.  Declare measurement boundary:

         +===============+==========================================+
         | Type          | Description                              |
         +===============+==========================================+
         | Full-stack    | Client to response, all components       |
         +---------------+------------------------------------------+
         | Per-component | Separate measurement at each boundary    |
         +---------------+------------------------------------------+
         | Partial       | Specific subset (e.g., Gateway + Engine) |
         +---------------+------------------------------------------+

                                   Table 40

     3.  Provide delta decomposition:

     Component          | TTFT Contribution | Throughput Impact
     -------------------|-------------------|------------------
     AI Gateway         | +15ms             | -3%
     AI Firewall (in)   | +45ms             | -8%
     Model Engine       | 180ms (baseline)  | baseline
     AI Firewall (out)  | +12ms*            | -12%
     -------------------|-------------------|------------------
     Total              | 252ms             | -22%

     *Outbound adds to client-observed TTFT, not engine TTFT

  7.2.  Composition Validation

     Measure components independently before measuring composite:

     1.  Engine alone: TTFT_engine = 180ms

     2.  Gateway + Engine: TTFT_gw = 195ms, Gateway_delta = 15ms

     3.  Firewall + Engine: TTFT_fw = 225ms, Firewall_delta = 45ms

     4.  Full stack: TTFT_full = 252ms

     5.  Validate: TTFT_engine + deltas approximately equals TTFT_full



  Gaikwad                   Expires 24 July 2026                 [Page 43]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     If validation fails, interaction effects exist.  Document them.

  7.3.  Interaction Effects

     Components may interact beyond simple addition:

     +==============+==============================+=====================+
     | Effect       | Description                  | Example             |
     +==============+==============================+=====================+
     | Batching     | Gateway batching conflicts   | Gateway batches 8,  |
     | interference | with engine                  | engine max is 4     |
     +--------------+------------------------------+---------------------+
     | Cache        | High gateway cache hit means | Biased difficulty   |
     | interaction  | engine sees hard queries     |                     |
     +--------------+------------------------------+---------------------+
     | Backpressure | Slow component causes        | Firewall slowdown   |
     |              | upstream queuing             | grows gateway       |
     |              |                              | queue               |
     +--------------+------------------------------+---------------------+
     | Timeout      | Mismatched timeouts waste    | See below           |
     | cascades     | resources                    |                     |
     +--------------+------------------------------+---------------------+

                                    Table 41

     Timeout Cascades:

























  Gaikwad                   Expires 24 July 2026                 [Page 44]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     TIMEOUT CASCADE (mismatched configurations)

     Gateway timeout:  10s  -------------+
     Firewall timeout: 15s  ---------------------+
     Engine timeout:   30s  -------------------------------------+
                            |            |       |               |
     Time: 0s              10s          15s     30s
           |                |            |       |
           +-- Request ---->|            |       |
           |                |            |       |
           |   Gateway -----X timeout    |       |
           |   (returns error to client) |       |
           |                |            |       |
           |   Firewall -----------------+       |
           |   (still waiting)           |       |
           |                             |       |
           |   Engine ---------------------------+
           |   (completes at 12s, result discarded)
           |                             |       |
           +-------------------------------------+

     Result: Client gets error at 10s. Engine wastes 12s of compute.

                       Figure 10: Timeout Cascade Example

     Report timeout configurations and note mismatches.

  8.  Access Logging Requirements

  8.1.  Minimum Fields

     All profiles MUST log:



















  Gaikwad                   Expires 24 July 2026                 [Page 45]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


              +============+===================================+
              | Field      | Description                       |
              +============+===================================+
              | timestamp  | Request start time                |
              +------------+-----------------------------------+
              | request_id | Unique identifier                 |
              +------------+-----------------------------------+
              | profile    | Infrastructure profile under test |
              +------------+-----------------------------------+
              | workload   | Workload profile applied          |
              +------------+-----------------------------------+
              | latency_ms | Total request latency             |
              +------------+-----------------------------------+
              | status     | Success, error, timeout           |
              +------------+-----------------------------------+

                                   Table 42

  8.2.  Model Engine Fields

              +=================+==============================+
              | Field           | Description                  |
              +=================+==============================+
              | queue_time_ms   | Time in queue                |
              +-----------------+------------------------------+
              | prefill_time_ms | Prefill latency              |
              +-----------------+------------------------------+
              | decode_time_ms  | Generation time              |
              +-----------------+------------------------------+
              | batch_size      | Concurrent requests in batch |
              +-----------------+------------------------------+
              | token_count_in  | Input tokens                 |
              +-----------------+------------------------------+
              | token_count_out | Output tokens                |
              +-----------------+------------------------------+

                                   Table 43

  8.3.  AI Firewall Fields

                 +====================+======================+
                 | Field              | Description          |
                 +====================+======================+
                 | direction          | Inbound or outbound  |
                 +--------------------+----------------------+
                 | decision           | Allow, block, modify |
                 +--------------------+----------------------+
                 | policy_triggered   | Which policy matched |



  Gaikwad                   Expires 24 July 2026                 [Page 46]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


                 +--------------------+----------------------+
                 | confidence         | Detection confidence |
                 +--------------------+----------------------+
                 | inspection_time_ms | Analysis time        |
                 +--------------------+----------------------+

                                    Table 44

  8.4.  Compound System Fields

                +==============+==============================+
                | Field        | Description                  |
                +==============+==============================+
                | trace_id     | Identifier linking all steps |
                +--------------+------------------------------+
                | step_count   | Total orchestration steps    |
                +--------------+------------------------------+
                | tool_calls   | List of tools invoked        |
                +--------------+------------------------------+
                | success_type | Hard, soft, or failure       |
                +--------------+------------------------------+

                                    Table 45

  8.5.  AI Gateway Fields

                  +=================+======================+
                  | Field           | Description          |
                  +=================+======================+
                  | cache_status    | Hit, miss, or bypass |
                  +-----------------+----------------------+
                  | route_target    | Selected backend     |
                  +-----------------+----------------------+
                  | token_count_in  | Input tokens         |
                  +-----------------+----------------------+
                  | token_count_out | Output tokens        |
                  +-----------------+----------------------+

                                   Table 46

  8.6.  OpenTelemetry Integration

     OpenTelemetry integration SHOULD be supported.  Reference GenAI
     semantic conventions when available.







  Gaikwad                   Expires 24 July 2026                 [Page 47]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


  9.  Measurement Considerations

  9.1.  Baseline and Delta Reporting

     For intermediary components (Gateway, Firewall), provide differential
     measurements:

     1.  Measure downstream directly (baseline)

     2.  Measure through intermediary

     3.  Compute delta

     4.  Report both absolute and delta

  9.2.  Warm-up and Steady State

     Declare whether results include cold start.

                 +==============+===========================+
                 | Profile      | Cold Start Factors        |
                 +==============+===========================+
                 | Model Engine | JIT compilation, KV cache |
                 |              | allocation, batch ramp-up |
                 +--------------+---------------------------+
                 | AI Gateway   | Connection pool, cache    |
                 |              | population                |
                 +--------------+---------------------------+
                 | AI Firewall  | Model loading, rule       |
                 |              | compilation               |
                 +--------------+---------------------------+
                 | Compound     | All above plus retrieval  |
                 | System       | index loading             |
                 +--------------+---------------------------+

                                   Table 47

     If excluding cold start, report warm-up procedure and duration.

  9.3.  Clock Synchronization

      +==========================+==================+===================+
      | Configuration            | Minimum Accuracy | Method            |
      +==========================+==================+===================+
      | Single-machine           | Inherent         | N/A               |
      +--------------------------+------------------+-------------------+
      | Same rack                | 1ms              | NTP               |
      +--------------------------+------------------+-------------------+



  Gaikwad                   Expires 24 July 2026                 [Page 48]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


      | Distributed              | 100us            | PTP               |
      +--------------------------+------------------+-------------------+
      | Sub-millisecond analysis | 10us             | PTP with hardware |
      |                          |                  | timestamps        |
      +--------------------------+------------------+-------------------+

                                    Table 48

     Reports MUST declare:

     *  Synchronization method

     *  Estimated maximum skew

     *  Single-point or distributed measurement

  9.4.  Streaming Protocol Considerations

      +==============+===========================+======================+
      | Profile      | Recommended Protocol      | Notes                |
      +==============+===========================+======================+
      | Model Engine | gRPC streaming            | Lower overhead       |
      +--------------+---------------------------+----------------------+
      | AI Gateway   | SSE over HTTP             | Broad compatibility  |
      +--------------+---------------------------+----------------------+
      | AI Firewall  | Match upstream/downstream | Minimize translation |
      +--------------+---------------------------+----------------------+
      | Compound     | SSE or WebSocket          | Client dependent     |
      | System       |                           |                      |
      +--------------+---------------------------+----------------------+

                                    Table 49

     Report chunk size distribution when measuring ITL.

  10.  Security Considerations

  10.1.  Bidirectional Enforcement Gaps

     AI Firewalls enforcing only one direction leave systems exposed.

     Inbound-only gaps:

     *  Cannot prevent PII leakage

     *  Cannot catch policy violations from model

     *  Cannot stop tool misuse



  Gaikwad                   Expires 24 July 2026                 [Page 49]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Outbound-only gaps:

     *  Cannot prevent prompt injection

     *  Cannot stop jailbreak attempts

     *  Malicious content reaches model

     Declare which directions are enforced.  "AI Firewall protection"
     without direction is incomplete.

  10.2.  Adversarial Workload Handling

     Security requirements for adversarial benchmarks:

     *  Samples MUST NOT contain working exploits

     *  Use sanitized patterns or synthetic constructs

     *  Reference published taxonomies (OWASP LLM Top 10, MITRE ATLAS)

     *  Do not publish novel attacks discovered during testing

  10.3.  Side-Channel Considerations

     Performance characteristics may leak information:

        +=========+======================================+============+
        | Channel | Risk                                 | Mitigation |
        +=========+======================================+============+
        | Timing  | Decision time reveals classification | Add noise  |
        +---------+--------------------------------------+------------+
        | Cache   | Hit patterns reveal similarity       | Per-tenant |
        |         |                                      | isolation  |
        +---------+--------------------------------------+------------+
        | Routing | Balancing reveals backend state      | Randomize  |
        +---------+--------------------------------------+------------+

                                    Table 50

     Multi-tenant benchmarks SHOULD measure side-channel exposure.

  11.  References

  11.1.  Normative References






  Gaikwad                   Expires 24 July 2026                 [Page 50]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     [I-D.gaikwad-llm-benchmarking-terminology]
                Gaikwad, M., "Benchmarking Terminology for Large Language
                Model Serving", Work in Progress, Internet-Draft, draft-
                gaikwad-llm-benchmarking-terminology, 2026,
                <https://datatracker.ietf.org/doc/html/draft-gaikwad-llm-
                benchmarking-terminology>.

     [I-D.gaikwad-llm-benchmarking-methodology]
                Gaikwad, M., "Benchmarking Methodology for Large Language
                Model Serving", Work in Progress, Internet-Draft, draft-
                gaikwad-llm-benchmarking-methodology, 2026,
                <https://datatracker.ietf.org/doc/html/draft-gaikwad-llm-
                benchmarking-methodology>.

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/info/rfc8174>.

  11.2.  Informative References

     [RFC2647]  Newman, D., "Benchmarking Terminology for Firewall
                Performance", RFC 2647, August 1999,
                <https://www.rfc-editor.org/info/rfc2647>.

     [RFC3511]  Hickman, B., Newman, D., Tadjudin, S., and T. Martin,
                "Benchmarking Methodology for Firewall Performance",
                RFC 3511, April 2003,
                <https://www.rfc-editor.org/info/rfc3511>.

     [OWASP-LLM]
                OWASP Foundation, "OWASP Top 10 for Large Language Model
                Applications", 2023, <https://owasp.org/www-project-top-
                10-for-large-language-model-applications/>.

     [BIPIA]    Yi, J., Xie, Y., Zhu, B., Hines, K., Kiciman, E., Sun, G.,
                and X. Xie, "Benchmarking and Defending Against Indirect
                Prompt Injection Attacks on Large Language Models", 2023,
                <https://arxiv.org/abs/2312.14197>.

     [DISTSERVE]
                Zhong, Y., Liu, S., Chen, J., Hu, J., Zhu, Y., Liu, X.,
                Jin, X., and H. Zhang, "DistServe: Disaggregating Prefill
                and Decoding for Goodput-optimized Large Language Model



  Gaikwad                   Expires 24 July 2026                 [Page 51]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


                Serving", OSDI 2024, 2024,
                <https://www.usenix.org/conference/osdi24/presentation/
                zhong-yinmin>.

     [MOONCAKE] Qin, R., Li, Z., He, W., Zhang, M., Wu, Y., Zheng, W., and
                L. Zhou, "Mooncake: A KVCache-centric Disaggregated
                Architecture for LLM Serving", 2024,
                <https://arxiv.org/abs/2407.00079>.

  Appendix A.  Example Benchmark Report Structure

     1. Executive Summary
        - SUT and profile(s) used
        - Key results

     2. System Configuration
        - Hardware
        - Software versions
        - Profile-specific config (per Section 4)

     3. Workload Specification
        - Workload profile
        - Parameters (per Section 5)
        - Dataset sources

     4. Methodology
        - Measurement boundary
        - Clock synchronization
        - Warm-up procedure
        - Duration and request counts

     5. Results
        - Primary metrics with percentiles
        - Secondary metrics
        - Delta decomposition (if composite)

     6. Analysis
        - Observations
        - Interaction effects
        - Limitations

     7. Reproduction
        - Config files
        - Scripts
        - Random seeds

  Author's Address




  Gaikwad                   Expires 24 July 2026                 [Page 52]


  Internet-Draft          LLM Benchmarking Profiles           January 2026


     Madhava Gaikwad
     Independent Researcher
     Email: gaikwad.madhav@gmail.com
















































  Gaikwad                   Expires 24 July 2026                 [Page 53]


name: draft-levy-llm-uri-scheme
rev: 00
title: The "llm" URI Scheme
time: 2026-02-06T17:55:33Z
group_uri: /api/v1/group/group/1027/
expires: 2026-08-10T17:55:32Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-05-22T09:24:52.065882+00:00
source: ietf
abstract:
  This document defines the "llm" Uniform Resource Identifier (URI)
     scheme for identifying Large Language Model (LLM) endpoints and their
     configuration.  The scheme encodes provider host, model identifier,
     authentication credentials, and inference parameters into a single,
     portable connection string, analogous to database connection URIs
     such as those used by PostgreSQL and MongoDB.

     This document registers the scheme name in the IANA "Uniform Resource
     Identifier (URI) Schemes" registry as a provisional registration per
     RFC 7595.

full_text:
  Individual Submission                                            D. Levy
  Internet-Draft                                        DanLevy Consulting
  Intended status: Informational                           6 February 2026
  Expires: 10 August 2026


                            The "llm" URI Scheme
                        draft-levy-llm-uri-scheme-00

  Abstract

     This document defines the "llm" Uniform Resource Identifier (URI)
     scheme for identifying Large Language Model (LLM) endpoints and their
     configuration.  The scheme encodes provider host, model identifier,
     authentication credentials, and inference parameters into a single,
     portable connection string, analogous to database connection URIs
     such as those used by PostgreSQL and MongoDB.

     This document registers the scheme name in the IANA "Uniform Resource
     Identifier (URI) Schemes" registry as a provisional registration per
     RFC 7595.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 10 August 2026.

  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights



  Levy                     Expires 10 August 2026                 [Page 1]


  Internet-Draft               LLM URI Scheme                February 2026


     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
       1.1.  Motivation  . . . . . . . . . . . . . . . . . . . . . . .   3
       1.2.  Relationship to Other Work  . . . . . . . . . . . . . . .   4
     2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   4
     3.  URI Syntax  . . . . . . . . . . . . . . . . . . . . . . . . .   4
       3.1.  The "llm" Scheme  . . . . . . . . . . . . . . . . . . . .   4
       3.2.  ABNF  . . . . . . . . . . . . . . . . . . . . . . . . . .   5
     4.  URI Components  . . . . . . . . . . . . . . . . . . . . . . .   5
       4.1.  Scheme  . . . . . . . . . . . . . . . . . . . . . . . . .   5
       4.2.  Authority . . . . . . . . . . . . . . . . . . . . . . . .   5
         4.2.1.  Userinfo  . . . . . . . . . . . . . . . . . . . . . .   5
         4.2.2.  Host  . . . . . . . . . . . . . . . . . . . . . . . .   6
         4.2.3.  Port  . . . . . . . . . . . . . . . . . . . . . . . .   6
       4.3.  Path (Model Identifier) . . . . . . . . . . . . . . . . .   6
       4.4.  Query Parameters  . . . . . . . . . . . . . . . . . . . .   7
       4.5.  Fragment Identifier . . . . . . . . . . . . . . . . . . .   8
     5.  Semantics and Operations  . . . . . . . . . . . . . . . . . .   8
       5.1.  Resolution  . . . . . . . . . . . . . . . . . . . . . . .   8
       5.2.  Operations  . . . . . . . . . . . . . . . . . . . . . . .   9
     6.  Well-Known Query Parameters . . . . . . . . . . . . . . . . .   9
     7.  Examples  . . . . . . . . . . . . . . . . . . . . . . . . . .  10
     8.  Encoding Considerations . . . . . . . . . . . . . . . . . . .  11
     9.  Interoperability Considerations . . . . . . . . . . . . . . .  12
     10. Security Considerations . . . . . . . . . . . . . . . . . . .  12
       10.1.  Credential Exposure  . . . . . . . . . . . . . . . . . .  12
       10.2.  Transport Security . . . . . . . . . . . . . . . . . . .  13
       10.3.  Injection Attacks  . . . . . . . . . . . . . . . . . . .  13
       10.4.  Prompt Injection via URI . . . . . . . . . . . . . . . .  13
       10.5.  Server-Side Request Forgery (SSRF) . . . . . . . . . . .  14
     11. Privacy Considerations  . . . . . . . . . . . . . . . . . . .  14
     12. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  15
       12.1.  URI Scheme Registration: llm . . . . . . . . . . . . . .  15
       12.2.  Well-Known Query Parameters Registry . . . . . . . . . .  15
     13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  15
       13.1.  Normative References . . . . . . . . . . . . . . . . . .  15
       13.2.  Informative References . . . . . . . . . . . . . . . . .  16
     Appendix A.  Comparison with Database Connection URIs . . . . . .  16
     Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  17
     Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  17





  Levy                     Expires 10 August 2026                 [Page 2]


  Internet-Draft               LLM URI Scheme                February 2026


  1.  Introduction

     Large Language Model (LLM) APIs have become a foundational component
     of modern software systems.  As of 2025, applications commonly
     integrate with multiple LLM providers including OpenAI, Anthropic,
     Google, Mistral, Cohere, and self-hosted models via frameworks such
     as Ollama and vLLM.

     Currently, there is no standardized way to represent an LLM endpoint
     configuration as a single, portable identifier.  Developers typically
     manage provider connections through a proliferation of environment
     variables (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, MODEL_NAME,
     API_BASE_URL, TEMPERATURE, MAX_TOKENS) that vary by provider,
     framework, and deployment context.

     This document defines the "llm" URI scheme to address this
     fragmentation by encoding all connection parameters into a single URI
     string, following the well-established precedent of database
     connection URIs.

  1.1.  Motivation

     The database ecosystem underwent a similar standardization journey.
     Early database clients required separate configuration for host,
     port, username, password, database name, and various driver options.
     The adoption of connection URIs (e.g.,
     postgres://user:pass@host:5432/dbname) dramatically simplified
     configuration, improved portability, and enabled tooling
     standardization.

     The LLM ecosystem is at an analogous inflection point.  A single
     connection string encoding provider, model, authentication, and
     inference parameters would provide:

     *  *Portability:* A single string that moves between environments,
        scripts, and deployment targets.

     *  *CLI ergonomics:* One argument instead of many flags.

     *  *Language agnosticism:* Every programming language includes a
        robust URI parser per RFC 3986, providing parsing, validation, and
        sanitization without additional dependencies.

     *  *Framework interoperability:* A common configuration format that
        middleware, routers, and observability tools can consume.






  Levy                     Expires 10 August 2026                 [Page 3]


  Internet-Draft               LLM URI Scheme                February 2026


  1.2.  Relationship to Other Work

     The "ai" URI scheme (draft-sogomonian-ai-uri-scheme) defines a
     general-purpose scheme for identifying AI-addressable resources such
     as agents, tools, and tasks, with its own resolution protocol (AIIP).
     The "llm" scheme defined in this document is narrower in scope: it is
     a configuration-oriented URI for identifying a specific model
     endpoint and its inference parameters, intended to be resolved by
     client libraries into standard HTTPS API calls.  The two schemes are
     complementary and non-overlapping.

  2.  Terminology

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in
     BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

     LLM:  Large Language Model.  A neural network trained on large text
        corpora that generates text completions given an input prompt.

     Provider:  An organization or service that hosts and serves an LLM
        via an HTTP API (e.g., OpenAI, Anthropic, a self-hosted Ollama
        instance).

     Model:  A specific LLM offered by a provider, identified by a model
        name or version string (e.g., "gpt-5", "claude-sonnet-4",
        "llama3").

     Inference Parameters:  Configuration values that control the model's
        output behavior for a given request, such as temperature, top-p,
        and maximum token count.

     Connection String:  A URI conforming to this specification that
        encodes a complete LLM endpoint configuration.

  3.  URI Syntax

  3.1.  The "llm" Scheme

     The "llm" URI scheme uses the generic URI syntax defined in [RFC3986]
     Section 3, with all standard components:

     llm-URI = "llm:" "//" authority path-abempty
               [ "?" query ] [ "#" fragment ]





  Levy                     Expires 10 August 2026                 [Page 4]


  Internet-Draft               LLM URI Scheme                February 2026


     The double-slash "//" is used because the scheme defines a
     hierarchical structure with a naming authority (the provider host).

     Example:

     llm://api.openai.com/gpt-5?temp=0.7&max_tokens=1500

     This identifies a GPT-5 model endpoint at api.openai.com with
     temperature 0.7 and a maximum output length of 1500 tokens.

  3.2.  ABNF

     Using the ABNF notation of [RFC5234] and the URI components defined
     in [RFC3986]:

     llm-URI       = "llm:" "//" llm-authority
                     llm-path [ "?" query ] [ "#" fragment ]

     llm-authority = [ userinfo "@" ] host [ ":" port ]

     llm-path      = "/" model-id *( "/" sub-path )

     model-id      = 1*( unreserved / pct-encoded / "." / "-" )

     sub-path      = 1*( unreserved / pct-encoded / "." / "-" )

     Where "userinfo", "host", "port", "query", "fragment", "unreserved",
     and "pct-encoded" are as defined in [RFC3986].

  4.  URI Components

  4.1.  Scheme

     The scheme component MUST be "llm" (case insensitive per [RFC3986]
     Section 3.1, but MUST be registered and SHOULD be written in
     lowercase).

  4.2.  Authority

     The authority component identifies the LLM provider and follows the
     syntax defined in Section 3.2 of [RFC3986]:

     authority = [ userinfo "@" ] host [ ":" port ]

  4.2.1.  Userinfo

     The userinfo subcomponent, if present, provides authentication
     credentials for the provider API:



  Levy                     Expires 10 August 2026                 [Page 5]


  Internet-Draft               LLM URI Scheme                February 2026


     userinfo = app-label [ ":" api-key ]

     app-label = 1*( unreserved / pct-encoded )
     api-key   = 1*( unreserved / pct-encoded )

     The "app-label" field identifies the application, project, or
     organization associated with the credential.  Client libraries SHOULD
     map this value to the provider's organization or project header where
     applicable.  For example, OpenAI-compatible APIs accept an OpenAI-
     Organization header; the app-label SHOULD be sent as the value of
     this header (or the provider's equivalent, such as Anthropic-
     Organization or X-Project-ID).  This enables providers to attribute
     usage, enforce per-project rate limits, and route billing.

     The "api-key" field contains the API secret or token.

     Example:

     llm://my-app:sk-proj-abc123@api.openai.com/gpt-5

     Implementations that produce or log LLM URIs MUST sanitize the
     userinfo component to prevent credential leakage (see Section 10).

  4.2.2.  Host

     The host identifies the provider's API endpoint.

  4.2.3.  Port

     The port subcomponent is OPTIONAL.  When omitted, clients SHOULD use
     port 443 (HTTPS) for remote providers and port 11434 for localhost
     (the conventional Ollama default).  Implementations MAY define other
     defaults.

  4.3.  Path (Model Identifier)

     The path component identifies the model.  The first path segment MUST
     be the model identifier.  Additional path segments MAY be used for
     provider-specific namespacing (e.g., organization or deployment
     identifiers).

     The model identifier SHOULD correspond to the model name string
     accepted by the provider's API.

     Examples:






  Levy                     Expires 10 August 2026                 [Page 6]


  Internet-Draft               LLM URI Scheme                February 2026


     /gpt-5                          -- Simple model name
     /gpt-5.2                        -- Versioned model name
     /anthropic/claude-sonnet-4.5    -- Namespaced model (e.g., for
                                        aggregator providers like
                                        Bedrock or Vercel AI)
     /llama3                         -- Self-hosted model

     Clients MUST NOT interpret path segments beyond mapping them to the
     provider's model selection parameter.  Path semantics are defined by
     the provider.

  4.4.  Query Parameters

     The query component carries inference parameters and provider-
     specific options as key=value pairs separated by "&", following
     standard URI query syntax.

     This document defines a set of well-known parameter names (Section 6)
     that implementations SHOULD recognize.  Providers and clients MAY
     define additional parameters.  Unrecognized parameters SHOULD be
     passed through to the provider API where possible and MUST NOT cause
     a client error.

     Parameter names are case-sensitive.  Values MUST be percent-encoded
     per [RFC3986] Section 2.1 when they contain reserved characters.

     Example:

     ?temp=0.7&max_tokens=1500&top_p=0.9&cache=true

     For simple structured parameters, implementations SHOULD use dot-
     notation to flatten nested keys:

     ?web_search.max_uses=3&web_search.enabled=true

     This avoids the brittleness of encoding full JSON objects in query
     strings.  When dot-notation is insufficient (e.g., deeply nested or
     array-valued configurations), implementations MAY percent-encode a
     JSON value:

     ?web_search=%7B%22maxUses%22%3A3%7D

     Which represents: web_search={"maxUses":3}

     Implementations that accept JSON-valued query parameters SHOULD
     impose a maximum length on individual parameter values and MUST be
     aware that many HTTP intermediaries (proxies, CDNs, load balancers)
     enforce URI length limits, commonly between 2,048 and 8,192 octets.



  Levy                     Expires 10 August 2026                 [Page 7]


  Internet-Draft               LLM URI Scheme                February 2026


     Complex configurations that would exceed practical URI length limits
     SHOULD be provided via out-of-band configuration files or environment
     variables rather than encoded in the URI.

  4.5.  Fragment Identifier

     The fragment component, if present, is client-side and is NOT sent to
     the provider.  Fragments MAY be used by client applications for
     purposes such as identifying a specific system prompt variant, a
     named configuration profile, or a UI context.

     Example:

     llm://api.openai.com/gpt-5?temp=0.7#coding-assistant

     The semantics of the fragment are not defined by this specification
     and are left to the consuming application.

  5.  Semantics and Operations

  5.1.  Resolution

     An "llm" URI does not resolve to a resource in the traditional HTTP
     sense.  Instead, it is a configuration identifier that a client
     library parses to construct API requests to the identified provider.

     The resolution process is:

     1.  Parse the URI per [RFC3986].

     2.  Extract the host to determine the provider API base URL.  The
         transport protocol MUST be HTTPS for remote hosts.  Clients MAY
         use HTTP for localhost/loopback addresses only.

     3.  Extract the path to determine the model identifier.

     4.  Extract userinfo, if present, to obtain authentication
         credentials.

     5.  Extract query parameters and map them to the provider's API
         request format.

     6.  Construct an HTTPS request to the provider's inference endpoint
         (e.g., POST /v1/chat/completions for OpenAI-compatible APIs).

     Clients MUST NOT dereference an "llm" URI by making a GET request to
     the URI itself.  The URI is a configuration descriptor, not a
     resource locator.



  Levy                     Expires 10 August 2026                 [Page 8]


  Internet-Draft               LLM URI Scheme                February 2026


  5.2.  Operations

     The primary operation associated with an LLM URI is inference:
     sending a prompt to the identified model and receiving a completion.
     This document does not define the prompt or response format, which
     varies by provider and is outside the scope of URI scheme definition.

     Implementations MAY support additional operations such as:

     *  Model metadata retrieval (listing capabilities, context window
        size, supported parameters).

     *  Streaming inference (server-sent events or chunked transfer).

     *  Batch inference.

     *  Embedding generation (when the identified model supports it).

  6.  Well-Known Query Parameters

     The following query parameter names are RECOMMENDED for
     interoperability.  Implementations SHOULD map these to the
     corresponding provider API parameters.




























  Levy                     Expires 10 August 2026                 [Page 9]


  Internet-Draft               LLM URI Scheme                February 2026


       +============+========+=========================================+
       | Parameter  | Type   | Description                             |
       +============+========+=========================================+
       | temp       | float  | Sampling temperature (0.0 to 2.0)       |
       +------------+--------+-----------------------------------------+
       | max_tokens | int    | Maximum tokens in the completion        |
       +------------+--------+-----------------------------------------+
       | top_p      | float  | Nucleus sampling threshold (0.0 to 1.0) |
       +------------+--------+-----------------------------------------+
       | top_k      | int    | Top-k sampling parameter                |
       +------------+--------+-----------------------------------------+
       | stop       | string | Stop sequence(s), comma-separated       |
       +------------+--------+-----------------------------------------+
       | seed       | int    | Random seed for reproducibility         |
       +------------+--------+-----------------------------------------+
       | stream     | bool   | Enable streaming response               |
       +------------+--------+-----------------------------------------+
       | timeout    | int    | Request timeout in milliseconds         |
       +------------+--------+-----------------------------------------+
       | retries    | int    | Maximum retry attempts                  |
       +------------+--------+-----------------------------------------+
       | cache      | bool   | Enable response caching                 |
       +------------+--------+-----------------------------------------+
       | format     | string | Response format (e.g., "json")          |
       +------------+--------+-----------------------------------------+
       | system     | string | System prompt (percent-encoded)         |
       +------------+--------+-----------------------------------------+

                                    Table 1

     Boolean values MUST be represented as "true" or "false" (case-
     insensitive).  Numeric values MUST be represented in decimal
     notation.

     Provider-specific parameters (e.g., "frequency_penalty",
     "presence_penalty", "cacheControl") SHOULD use the parameter names
     defined by the provider's API documentation and MUST be percent-
     encoded if they contain reserved characters.

  7.  Examples

     Basic usage with a cloud provider:

     llm://api.openai.com/gpt-5?temp=0.7&max_tokens=1500

     With authentication:

     llm://my-app:sk-proj-abc123@api.openai.com/gpt-5?temp=0.7



  Levy                     Expires 10 August 2026                [Page 10]


  Internet-Draft               LLM URI Scheme                February 2026


     Self-hosted model (Ollama):

     llm://localhost:11434/llama3

     Anthropic via aggregator (Bedrock):

     llm://bedrock.us-west-2.amazonaws.com/anthropic/claude-sonnet-4.5
       ?temp=0.8&cache=true

     Vercel AI SDK-style namespacing:

     llm://sdk.vercel.ai/anthropic/claude-sonnet-4.5
       ?temp=0.8&web_search=%7B%22maxUses%22%3A3%7D

     Local model with JSON output:

     llm://localhost:11434/mistral?format=json&temp=0

     With fragment for application context:

     llm://api.anthropic.com/claude-sonnet-4.5
       ?temp=0.3&max_tokens=4096#code-review

  8.  Encoding Considerations

     LLM URIs MUST conform to the encoding rules defined in [RFC3986]
     Section 2.  In particular:

     *  The model identifier in the path component MUST percent-encode any
        characters not in the "unreserved" set or "/" delimiter.

     *  Query parameter values MUST percent-encode reserved characters per
        Section 2.2 of [RFC3986].

     *  The userinfo component MUST percent-encode ":" in API keys that
        contain literal colons, and "@" in any subcomponent.

     *  Structured values (JSON objects) in query parameters MUST be fully
        percent-encoded.

     Implementations SHOULD produce URIs using UTF-8 encoding for any non-
     ASCII characters, consistent with [RFC3987] IRI-to-URI mapping.









  Levy                     Expires 10 August 2026                [Page 11]


  Internet-Draft               LLM URI Scheme                February 2026


  9.  Interoperability Considerations

     The "llm" scheme is designed to be consumed by LLM client libraries,
     CLI tools, middleware routers, and orchestration frameworks.  It is
     NOT designed to be resolved by web browsers or general-purpose HTTP
     clients.

     Known interoperability considerations include:

     *  *Provider API Variance:* LLM providers use different parameter
        names for equivalent concepts (e.g., "temperature" vs "temp",
        "max_tokens" vs "maxOutputTokens").  Client libraries SHOULD map
        the well-known parameter names defined in Section 6 to the
        provider's expected parameter names.

     *  *Model Name Formats:* Some providers use simple names ("gpt-5"),
        others use versioned names ("gpt-5-2025-01-15"), and aggregator
        services use namespaced paths ("anthropic/claude-sonnet-4.5").
        The path component accommodates all of these formats.

     *  *Authentication Methods:* While this scheme supports credential
        embedding via userinfo, some providers require additional
        authentication mechanisms (e.g., OAuth 2.0 tokens, IAM roles, mTLS
        certificates) that cannot be represented in a URI.  Clients MAY
        use query parameters or out-of-band configuration for such
        mechanisms.

  10.  Security Considerations

     The "llm" URI scheme presents several security considerations that
     implementations MUST address.

  10.1.  Credential Exposure

     LLM URIs MAY contain API keys in the userinfo component.  This
     creates a risk of credential leakage through:

     *  Application logs and monitoring systems.

     *  Browser history and bookmarks (if URIs are inadvertently used in
        web contexts).

     *  Shell history files.

     *  Error messages and stack traces.

     *  Version control systems (.env files committed to repositories).




  Levy                     Expires 10 August 2026                [Page 12]


  Internet-Draft               LLM URI Scheme                February 2026


     Implementations that produce, store, or transmit LLM URIs MUST:

     *  Sanitize the userinfo component before logging, by replacing the
        api-key portion with a redaction marker (e.g., "***").

     *  Treat LLM URIs containing credentials with the same care as any
        other bearer token or secret.

     Implementations SHOULD support credential resolution via environment
     variables or secret managers as an alternative to embedding
     credentials directly in the URI.

  10.2.  Transport Security

     Clients MUST use HTTPS (TLS 1.2 or later, TLS 1.3 RECOMMENDED) for
     all remote provider communication.  Clients MUST NOT send credentials
     or inference requests over unencrypted HTTP to non-loopback
     addresses.

     For localhost and loopback addresses (127.0.0.0/8, ::1), clients MAY
     use HTTP, as this is common for self-hosted model servers.

  10.3.  Injection Attacks

     Because query parameters are passed to provider APIs, implementations
     MUST validate and sanitize all parameter values before including them
     in API requests.  In particular:

     *  Numeric parameters MUST be validated as numeric.

     *  Boolean parameters MUST be validated as boolean.

     *  String parameters MUST be bounded in length.

     *  Structured values (JSON) MUST be parsed and validated before
        forwarding.

  10.4.  Prompt Injection via URI

     The "system" query parameter, if supported, carries a percent-encoded
     system prompt.  This creates a vector for prompt injection if URIs
     are constructed from untrusted input.  Implementations SHOULD treat
     the "system" parameter as untrusted input and apply appropriate
     content filtering.







  Levy                     Expires 10 August 2026                [Page 13]


  Internet-Draft               LLM URI Scheme                February 2026


  10.5.  Server-Side Request Forgery (SSRF)

     Applications that accept LLM URIs from untrusted sources (e.g., user-
     submitted bot configurations, webhook endpoints, or plugin manifests)
     are vulnerable to Server-Side Request Forgery.  An attacker could
     supply a URI such as llm://localhost:8080/admin or
     llm://169.254.169.254/latest/meta-data to cause the client library to
     issue HTTP requests to internal services, cloud metadata endpoints,
     or other resources that should not be externally reachable.

     Implementations that accept LLM URIs from untrusted input MUST:

     *  Maintain an allowlist of permitted provider hosts, OR maintain a
        blocklist that includes at minimum: loopback addresses
        (127.0.0.0/8, ::1), link-local addresses (169.254.0.0/16,
        fe80::/10), private network ranges (10.0.0.0/8, 172.16.0.0/12,
        192.168.0.0/16), and cloud metadata endpoints.

     *  Resolve hostnames and validate the resulting IP address against
        the allowlist/blocklist BEFORE establishing a connection, to
        prevent DNS rebinding attacks.

     *  Reject URIs with IP address literals in the host component when
        operating in production environments, unless explicitly configured
        to allow them.

     In production deployments, clients SHOULD default to blocking
     loopback and private network addresses.  Localhost access SHOULD only
     be permitted when explicitly enabled for local model serving (e.g.,
     Ollama).

  11.  Privacy Considerations

     LLM URIs may reveal information about:

     *  Which AI providers and models a user or organization employs.

     *  Inference configuration choices that may indicate use case (e.g.,
        low temperature for code generation, high temperature for creative
        writing).

     *  Application identifiers embedded in the userinfo app-label.

     When LLM URIs are shared, logged, or transmitted, this metadata
     leakage SHOULD be considered.  Implementations SHOULD provide
     mechanisms to strip or redact URI components before sharing.





  Levy                     Expires 10 August 2026                [Page 14]


  Internet-Draft               LLM URI Scheme                February 2026


  12.  IANA Considerations

  12.1.  URI Scheme Registration: llm

     In accordance with [RFC7595], this section provides the registration
     request for the "llm" URI scheme.

     Scheme name:  llm

     Status:  Provisional

     Applications/protocols that use this scheme name:  LLM client
        libraries, AI application frameworks, CLI tools, and orchestration
        systems that connect to Large Language Model inference APIs.

     Contact:  Dan Levy <dan@danlevy.net>

     Change controller:  Dan Levy <dan@danlevy.net>

     References:  This document; RFC 3986; RFC 7595.

  12.2.  Well-Known Query Parameters Registry

     This document requests IANA create a new registry titled "LLM URI
     Scheme Well-Known Query Parameters" under the "Uniform Resource
     Identifier (URI) Schemes" group.

     The initial contents of the registry are the parameters defined in
     Section 6 of this document.  New entries may be added via
     Specification Required [RFC8126] policy.

  13.  References

  13.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/rfc/rfc2119>.

     [RFC3986]  Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
                Resource Identifier (URI): Generic Syntax", STD 66,
                RFC 3986, DOI 10.17487/RFC3986, January 2005,
                <https://www.rfc-editor.org/rfc/rfc3986>.







  Levy                     Expires 10 August 2026                [Page 15]


  Internet-Draft               LLM URI Scheme                February 2026


     [RFC5234]  Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
                Specifications: ABNF", STD 68, RFC 5234,
                DOI 10.17487/RFC5234, January 2008,
                <https://www.rfc-editor.org/rfc/rfc5234>.

     [RFC7595]  Thaler, D., Ed., Hansen, T., and T. Hardie, "Guidelines
                and Registration Procedures for URI Schemes", BCP 35,
                RFC 7595, DOI 10.17487/RFC7595, June 2015,
                <https://www.rfc-editor.org/rfc/rfc7595>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

  13.2.  Informative References

     [LLM-CONN] Levy, D., "It's Time for LLM Connection Strings", January
                2026, <https://danlevy.net/llm-connection-strings/>.

     [OPENAI-API]
                OpenAI, "API Reference", n.d.,
                <https://platform.openai.com/docs/api-reference>.

     [RFC3987]  Duerst, M. and M. Suignard, "Internationalized Resource
                Identifiers (IRIs)", RFC 3987, DOI 10.17487/RFC3987,
                January 2005, <https://www.rfc-editor.org/rfc/rfc3987>.

     [RFC8126]  Cotton, M., Leiba, B., and T. Narten, "Guidelines for
                Writing an IANA Considerations Section in RFCs", BCP 26,
                RFC 8126, DOI 10.17487/RFC8126, June 2017,
                <https://www.rfc-editor.org/rfc/rfc8126>.

     [RFC8446]  Rescorla, E., "The Transport Layer Security (TLS) Protocol
                Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
                <https://www.rfc-editor.org/rfc/rfc8446>.

  Appendix A.  Comparison with Database Connection URIs

     The following table illustrates the structural analogy between the
     "llm" scheme and established database connection URI schemes:











  Levy                     Expires 10 August 2026                [Page 16]


  Internet-Draft               LLM URI Scheme                February 2026


      +===========+=========================+===========================+
      | Component | Database URI (postgres) | LLM URI (llm)             |
      +===========+=========================+===========================+
      | Scheme    | postgres://             | llm://                    |
      +-----------+-------------------------+---------------------------+
      | Auth      | user:password           | app-label:api-key         |
      +-----------+-------------------------+---------------------------+
      | Host      | db.example.com          | api.openai.com            |
      +-----------+-------------------------+---------------------------+
      | Port      | :5432                   | :443 (default)            |
      +-----------+-------------------------+---------------------------+
      | Resource  | /database_name          | /model-name               |
      +-----------+-------------------------+---------------------------+
      | Options   | ?sslmode=require        | ?temp=0.7&max_tokens=1500 |
      +-----------+-------------------------+---------------------------+

                                    Table 2

     This structural equivalence means that existing URI parsing
     libraries, URL-safe storage mechanisms, and secret management tools
     that already handle database URIs can be extended to handle LLM URIs
     with minimal effort.

  Acknowledgements

     The concept of LLM connection strings was first described in "It's
     Time for LLM Connection Strings" [LLM-CONN] published at danlevy.net.
     The author thanks Yakov Shafranovich for suggesting IANA registration
     and the commenters who provided early feedback.

  Author's Address

     Dan Levy
     DanLevy Consulting
     Denver, CO,
     United States of America
     Email: dan@danlevy.net
     URI:   https://danlevy.net













  Levy                     Expires 10 August 2026                [Page 17]


name: draft-condrey-rats-witnessd-enrollment
rev: 00
title: Trust Anchor Bootstrap Protocol for Proof of Process
time: 2026-02-07T10:31:35Z
group_uri: /api/v1/group/group/1027/
expires: 2026-08-11T10:31:34Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-07T06:55:11.565679+00:00
source: ietf
abstract:
  This document specifies a trust anchor bootstrap protocol for the
     witnessd Proof of Process framework.  The protocol defines how new
     devices enter the witnessd ecosystem, how device keys are provisioned
     and verified, and how Verifiers discover and validate device trust
     anchors.

     Three enrollment modes are defined: self-sovereign mode for
     individual users, organizational mode for enterprise deployments, and
     public registry mode for federated trust ecosystems.

full_text:
  Remote ATtestation procedureS                                 D. Condrey
  Internet-Draft                                          Writerslogic Inc
  Intended status: Standards Track                         7 February 2026
  Expires: 11 August 2026


            Trust Anchor Bootstrap Protocol for Proof of Process
                 draft-condrey-rats-witnessd-enrollment-00

  Abstract

     This document specifies a trust anchor bootstrap protocol for the
     witnessd Proof of Process framework.  The protocol defines how new
     devices enter the witnessd ecosystem, how device keys are provisioned
     and verified, and how Verifiers discover and validate device trust
     anchors.

     Three enrollment modes are defined: self-sovereign mode for
     individual users, organizational mode for enterprise deployments, and
     public registry mode for federated trust ecosystems.

  About This Document

     This note is to be removed before publishing as an RFC.

     This is a companion document to draft-condrey-rats-pop, which defines
     the core Proof of Process evidence framework.  This document
     addresses the bootstrapping problem: how do Verifiers know which
     device keys to trust?

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 11 August 2026.





  Condrey                  Expires 11 August 2026                 [Page 1]


  Internet-Draft             witnessd Enrollment             February 2026


  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
       1.1.  Requirements Notation . . . . . . . . . . . . . . . . . .   3
     2.  Enrollment Modes  . . . . . . . . . . . . . . . . . . . . . .   3
       2.1.  Self-Sovereign Mode . . . . . . . . . . . . . . . . . . .   3
       2.2.  Organizational Mode . . . . . . . . . . . . . . . . . . .   4
         2.2.1.  Well-Known URI for Device Discovery . . . . . . . . .   4
       2.3.  Public Registry Mode  . . . . . . . . . . . . . . . . . .   4
     3.  Device Enrollment Protocol  . . . . . . . . . . . . . . . . .   5
       3.1.  Enrollment Request  . . . . . . . . . . . . . . . . . . .   5
       3.2.  Challenge-Response Flow . . . . . . . . . . . . . . . . .   5
       3.3.  Certificate Issuance  . . . . . . . . . . . . . . . . . .   6
     4.  Trust Anchor Discovery  . . . . . . . . . . . . . . . . . . .   7
       4.1.  Discovery Mechanisms  . . . . . . . . . . . . . . . . . .   7
       4.2.  Caching Policy  . . . . . . . . . . . . . . . . . . . . .   7
       4.3.  Revocation Checking Integration . . . . . . . . . . . . .   8
     5.  Key Lifecycle Management  . . . . . . . . . . . . . . . . . .   8
       5.1.  Key Rotation  . . . . . . . . . . . . . . . . . . . . . .   8
       5.2.  Device Migration  . . . . . . . . . . . . . . . . . . . .   9
       5.3.  Key Compromise Response . . . . . . . . . . . . . . . . .   9
       5.4.  Certificate Renewal . . . . . . . . . . . . . . . . . . .  10
     6.  Security Considerations . . . . . . . . . . . . . . . . . . .  10
       6.1.  Threat Model  . . . . . . . . . . . . . . . . . . . . . .  10
       6.2.  Compromise Impact Analysis  . . . . . . . . . . . . . . .  11
       6.3.  Mitigation Requirements . . . . . . . . . . . . . . . . .  11
     7.  Privacy Considerations  . . . . . . . . . . . . . . . . . . .  11
       7.1.  Device Identifier Privacy . . . . . . . . . . . . . . . .  12
       7.2.  Enrollment Metadata Minimization  . . . . . . . . . . . .  12
       7.3.  Registry Privacy Modes  . . . . . . . . . . . . . . . . .  12
     8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
     9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  13
       9.1.  Normative References  . . . . . . . . . . . . . . . . . .  13
       9.2.  Informative References  . . . . . . . . . . . . . . . . .  13



  Condrey                  Expires 11 August 2026                 [Page 2]


  Internet-Draft             witnessd Enrollment             February 2026


     Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  13

  1.  Introduction

     The Proof of Process (PoP) specification [I-D.condrey-rats-pop]
     defines how Attesters generate Evidence during document authorship.
     Evidence packets are signed with device keys, but the core
     specification does not define how these keys are provisioned or how
     Verifiers establish trust in them.

     This document addresses the trust anchor bootstrap problem by
     defining:

     *  Device enrollment protocols for provisioning signing keys

     *  Trust anchor discovery mechanisms for Verifiers

     *  Multiple enrollment modes for different deployment scenarios

     *  Key lifecycle management including rotation and migration

  1.1.  Requirements Notation

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in BCP
     14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

  2.  Enrollment Modes

  2.1.  Self-Sovereign Mode

     In self-sovereign mode, users generate their own device keys and
     manually provide trust anchors to Verifiers they wish to work with.
     This mode provides maximum privacy but requires out-of-band trust
     establishment.

     Workflow:

     1.  Device generates key pair in secure hardware (if available)

     2.  Device exports public key and self-signed certificate

     3.  User manually provides public key to Verifier (via secure
         channel)

     4.  Verifier stores trust anchor associated with user identity



  Condrey                  Expires 11 August 2026                 [Page 3]


  Internet-Draft             witnessd Enrollment             February 2026


     Use cases: Individual authors, privacy-conscious users, testing
     environments.

  2.2.  Organizational Mode

     In organizational mode, device keys are signed by an organization's
     Certificate Authority (CA).  The organization publishes its CA
     certificate, and Verifiers trust devices whose keys chain to the
     organization's CA.

     Workflow:

     1.  Organization establishes CA infrastructure

     2.  Organization publishes CA certificate at well-known location

     3.  Device generates key pair and creates certificate signing request

     4.  Organization signs device certificate (may require proof of
         device identity)

     5.  Device stores signed certificate

     6.  Verifiers validate device certificates against organization CA

     Use cases: Enterprises, academic institutions, publishing houses.

  2.2.1.  Well-Known URI for Device Discovery

     Organizations SHOULD publish device trust information at:

     /.well-known/witnessd-devices

     This endpoint returns a signed JSON document listing:

     *  Organization CA certificate chain

     *  Supported enrollment methods

     *  Device certificate validation policy

     *  Contact information for enrollment requests

  2.3.  Public Registry Mode

     In public registry mode, devices register with a public discovery
     service, similar to ACME for TLS certificates.  This enables
     federated trust without requiring prior relationship establishment.



  Condrey                  Expires 11 August 2026                 [Page 4]


  Internet-Draft             witnessd Enrollment             February 2026


     Workflow:

     1.  Device generates key pair

     2.  Device proves control of user identifier (email, domain, etc.)

     3.  Registry issues device certificate bound to user identifier

     4.  Registry publishes certificate in transparency log

     5.  Verifiers query registry to validate device certificates

     Use cases: Open ecosystems, cross-organization verification, consumer
     applications.

  3.  Device Enrollment Protocol

     The device enrollment protocol establishes trust between a new device
     and the enrollment authority.  The protocol uses a challenge-response
     flow to verify device identity and key possession.

  3.1.  Enrollment Request

     Devices initiate enrollment by submitting a request containing:

     enrollment-request = {
       device-public-key: bstr,      ; Device's public key (COSE_Key)
       user-identifier: tstr,        ; Email, domain, or opaque ID
       enrollment-mode: mode-type,
       ? device-attestation: bstr,   ; TPM/Secure Enclave attestation
       ? proof-of-possession: bstr,  ; Signature over challenge
       nonce: bstr,                  ; Freshness nonce
       timestamp: time
     }

     mode-type = &(
       self-sovereign: 0,
       organizational: 1,
       public-registry: 2
     )

  3.2.  Challenge-Response Flow

     For organizational and public registry modes, the enrollment
     authority issues a challenge:

     1.  Device submits initial enrollment request




  Condrey                  Expires 11 August 2026                 [Page 5]


  Internet-Draft             witnessd Enrollment             February 2026


     2.  Authority returns challenge (random nonce + user verification
         requirement)

     3.  Device signs challenge with private key (proof of possession)

     4.  User completes verification (email link, domain TXT record, etc.)

     5.  Device submits signed challenge and verification proof

     6.  Authority validates and issues device certificate

     enrollment-challenge = {
       challenge-nonce: bstr,
       verification-method: verification-type,
       ? verification-uri: tstr,    ; For email/web verification
       expires: time
     }

     verification-type = &(
       email-link: 0,
       domain-txt: 1,
       organizational-approval: 2,
       device-attestation-only: 3
     )

     enrollment-response = {
       challenge-nonce: bstr,
       challenge-signature: bstr,   ; Signed by device private key
       verification-token: tstr,    ; From email/domain verification
       timestamp: time
     }

  3.3.  Certificate Issuance

     Upon successful verification, the authority issues a device
     certificate:















  Condrey                  Expires 11 August 2026                 [Page 6]


  Internet-Draft             witnessd Enrollment             February 2026


     device-certificate = {
       device-id: bstr,             ; Unique device identifier
       public-key: bstr,            ; COSE_Key
       user-identifier: tstr,
       issuer: tstr,                ; Authority identifier
       issued-at: time,
       expires-at: time,
       ? constraints: [* constraint],
       signature: bstr              ; Authority signature
     }

     constraint = &(
       max-evidence-per-day: uint,
       allowed-document-types: [* tstr],
       geographic-restriction: tstr
     )

     Certificates MUST be valid for no more than 1 year.  Devices MUST re-
     enroll before certificate expiration.

  4.  Trust Anchor Discovery

     Verifiers must discover and maintain trust anchors for device
     certificate validation.  This section defines discovery mechanisms
     and caching policies.

  4.1.  Discovery Mechanisms

     Verifiers discover trust anchors through:

     Well-known URI:  Query /.well-known/witnessd-devices at the
        organization's domain to retrieve CA certificates and enrollment
        policies.

     Public Registry:  Query the public registry service for device
        certificates bound to user identifiers.  The registry provides
        certificate chains and revocation status.

     Out-of-band provisioning:  For self-sovereign mode, trust anchors are
        manually configured by the Verifier operator.

  4.2.  Caching Policy

     Verifiers SHOULD cache trust anchors to reduce latency and network
     load.  Caching policies:

     *  CA certificates: Cache for up to 24 hours or until the next-update
        field indicates refresh is needed



  Condrey                  Expires 11 August 2026                 [Page 7]


  Internet-Draft             witnessd Enrollment             February 2026


     *  Device certificates: Cache for the certificate validity period
        minus a safety margin (RECOMMENDED: 1 hour)

     *  Revocation status: Cache according to revocation list TTL (see
        draft-condrey-rats-witnessd-revocation)

     Verifiers MUST refresh cached trust anchors before expiration.  Stale
     cache entries MUST NOT be used for verification.

  4.3.  Revocation Checking Integration

     Trust anchor discovery integrates with revocation checking:

     1.  Verifier receives Evidence packet

     2.  Verifier extracts device certificate from Evidence

     3.  Verifier validates certificate chain to trusted CA

     4.  Verifier checks certificate revocation status

     5.  Verifier checks Evidence revocation status

     6.  If all checks pass, Verifier proceeds with appraisal

     Certificate revocation and Evidence revocation are independent.  A
     revoked certificate invalidates all Evidence signed by that device.
     Evidence may be revoked even if the certificate remains valid.

  5.  Key Lifecycle Management

     Device keys have a lifecycle from generation through retirement.
     This section defines key management operations.

  5.1.  Key Rotation

     Devices SHOULD rotate signing keys periodically to limit exposure
     from potential compromise.  Key rotation workflow:

     1.  Device generates new key pair

     2.  Device requests certificate for new key (standard enrollment)

     3.  Device publishes key rotation notice signed by old key

     4.  Rotation notice links old key to new key

     5.  Old key enters grace period (RECOMMENDED: 30 days)



  Condrey                  Expires 11 August 2026                 [Page 8]


  Internet-Draft             witnessd Enrollment             February 2026


     6.  After grace period, old key is retired

     key-rotation-notice = {
       old-device-id: bstr,
       new-device-id: bstr,
       old-public-key: bstr,
       new-public-key: bstr,
       rotation-time: time,
       grace-period-end: time,
       old-key-signature: bstr,    ; Proves control of old key
       new-key-signature: bstr     ; Proves control of new key
     }

     Evidence signed during the grace period MAY use either key.  After
     the grace period, only the new key is valid.

  5.2.  Device Migration

     When users migrate to a new device, Evidence continuity must be
     maintained.  Migration is treated as key rotation with additional
     verification:

     *  User initiates migration from old device (if available)

     *  Old device signs migration authorization for new device

     *  New device completes enrollment with migration authorization

     *  If old device is unavailable, user completes out-of-band
        verification (email, organizational approval, etc.)

     Migration without old device access requires stronger verification to
     prevent unauthorized device takeover.

  5.3.  Key Compromise Response

     When a device key is compromised:

     1.  User reports compromise to enrollment authority

     2.  Authority revokes device certificate immediately

     3.  User enrolls new device with fresh key

     4.  User may re-endorse historical Evidence with new key (see draft-
         condrey-rats-witnessd-revocation)





  Condrey                  Expires 11 August 2026                 [Page 9]


  Internet-Draft             witnessd Enrollment             February 2026


     Hardware-backed keys (TPM, Secure Enclave) reduce compromise risk and
     are RECOMMENDED for high-security deployments.

  5.4.  Certificate Renewal

     Devices MUST renew certificates before expiration.  Renewal may use
     the existing key (if not compromised) or coincide with key rotation.
     Renewal workflow:

     1.  Device submits renewal request with current certificate

     2.  Authority validates current certificate is still valid

     3.  Authority issues new certificate (same or new key)

     4.  New certificate validity period begins from issuance

     Devices SHOULD initiate renewal at least 7 days before certificate
     expiration to allow for processing delays.

  6.  Security Considerations

  6.1.  Threat Model

     The enrollment protocol considers the following threats:

     Unauthorized enrollment:  Attacker attempts to enroll a device under
        another user's identity.  Mitigated by user identifier
        verification (email, domain, organizational approval).

     Key extraction:  Attacker attempts to extract device private key.
        Mitigated by hardware-backed key storage (TPM, Secure Enclave)
        where available.

     Enrollment authority compromise:  Attacker compromises the enrollment
        authority and issues fraudulent certificates.  Mitigated by
        certificate transparency logs and short certificate validity
        periods.

     Man-in-the-middle:  Attacker intercepts enrollment messages.
        Mitigated by TLS for all enrollment communications and challenge-
        response binding.

     Replay attacks:  Attacker replays old enrollment requests.  Mitigated
        by nonces and timestamps in all protocol messages.






  Condrey                  Expires 11 August 2026                [Page 10]


  Internet-Draft             witnessd Enrollment             February 2026


  6.2.  Compromise Impact Analysis

     The impact of various compromises:

     *  Single device key compromise: Only Evidence from that device is
        affected.  Other devices and users are unaffected.

     *  Organizational CA compromise: All devices enrolled under that CA
        are potentially affected.  Requires CA re-keying and mass device
        re-enrollment.

     *  Public registry compromise: All devices using that registry are
        potentially affected.  Requires registry recovery and certificate
        reissuance.

     Organizational deployments SHOULD use Hardware Security Modules
     (HSMs) for CA key protection.  Public registries MUST implement
     multi-party controls for certificate issuance.

  6.3.  Mitigation Requirements

     Implementations MUST:

     *  Use TLS 1.3 or later for all enrollment communications

     *  Validate certificate chains completely before trusting

     *  Check certificate revocation status before accepting Evidence

     *  Implement rate limiting to prevent enrollment abuse

     *  Log enrollment events for audit purposes

     Implementations SHOULD:

     *  Use hardware-backed key storage where available

     *  Implement certificate transparency monitoring

     *  Support multiple enrollment authorities for redundancy

     *  Provide key compromise notification mechanisms

  7.  Privacy Considerations







  Condrey                  Expires 11 August 2026                [Page 11]


  Internet-Draft             witnessd Enrollment             February 2026


  7.1.  Device Identifier Privacy

     Device identifiers enable tracking across Evidence packets.  To
     protect user privacy:

     *  Device identifiers SHOULD be opaque random values, not derived
        from hardware identifiers or user information

     *  Users MAY request new device identifiers during key rotation to
        break tracking linkage

     *  Verifiers MUST NOT share device verification patterns with third
        parties

     Self-sovereign mode provides the strongest privacy as no central
     authority learns about device enrollment.  Public registry mode has
     the weakest privacy as the registry sees all enrolled devices.

  7.2.  Enrollment Metadata Minimization

     Enrollment authorities collect metadata during device registration.
     To minimize privacy impact:

     *  Collect only the minimum information necessary for enrollment

     *  Do not retain IP addresses or location data beyond enrollment
        completion

     *  Do not share enrollment metadata with third parties

     *  Provide data deletion upon user request (subject to legal
        retention requirements)

     Enrollment authorities MUST publish a privacy policy describing data
     collection, retention, and sharing practices.

  7.3.  Registry Privacy Modes

     Public registries MAY offer privacy modes:

     Public mode:  Device certificates are publicly discoverable.  Anyone
        can verify Evidence from the device.  Suitable for public authors.

     Unlisted mode:  Device certificates are not publicly listed but can
        be retrieved by packet-id lookup.  Provides some obscurity.

     Private mode:  Device certificates are only returned to authenticated




  Condrey                  Expires 11 August 2026                [Page 12]


  Internet-Draft             witnessd Enrollment             February 2026


        Verifiers with a legitimate need.  Suitable for sensitive
        contexts.

     Users SHOULD select the privacy mode appropriate for their use case.
     The default mode is deployment-specific.

  8.  IANA Considerations

     This document requests registration of the following well-known URI:

     *  URI suffix: witnessd-devices

     *  Change controller: IETF

     *  Specification document: [this document]

  9.  References

  9.1.  Normative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, May 2017,
                <https://www.rfc-editor.org/info/rfc8174>.

  9.2.  Informative References

     [I-D.condrey-rats-pop]
                Condrey, D., "Proof of Process: An Evidence Framework for
                Digital Authorship Attestation", Work in Progress,
                Internet-Draft, draft-condrey-rats-pop,
                <https://datatracker.ietf.org/doc/html/draft-condrey-rats-
                pop>.

  Author's Address

     David Condrey
     Writerslogic Inc
     United States
     Email: david@writerslogic.com








  Condrey                  Expires 11 August 2026                [Page 13]


name: draft-willman-rtgwg-conduit-tunnels
rev: 01
title: Underlay for IPsec Transport
time: 2026-02-20T04:16:31Z
group_uri: /api/v1/group/group/1027/
expires: 2026-08-24T04:16:31Z
states: ["/api/v1/doc/state/1/", "/api/v1/doc/state/150/"]
categories: []
tags: []
fetched_at: 2026-03-07T06:55:11.565683+00:00
source: ietf
abstract:
  This document specifies CONDUIT, a tunnel fabric management protocol
     for tactical and enterprise networks operating over heterogeneous
     Wide Area Network (WAN) links.  CONDUIT automates the lifecycle
     management of IPsec tunnels, monitors tunnel health through active
     probing, and publishes quality metrics to enable Segment Routing over
     IPv6 (SRv6) based traffic engineering.

     CONDUIT follows a strict separation of concerns: it manages the
     underlay tunnel fabric while delegating all traffic engineering
     decisions to SRv6.  This architecture simplifies network operations,
     enables rapid adaptation to changing WAN conditions, and leverages
     standard SRv6 capabilities including Flexible Algorithm (Flex-Algo)
     and Topology-Independent Loop-Free Alternate (TI-LFA).

     The protocol is fully programmable via gRPC and mandates compliance
     with Commercial National Security Algorithm Suite 2.0 (CNSA 2.0)
     cryptographic requirements.

full_text:
  Routing Area Working Group                                    J. E. W. V
  Internet-Draft                               Department of the Air Force
  Updates: draft-conduit-tunnel-fabric (if                19 February 2026
           approved)                                                      
  Intended status: Standards Track                                        
  Expires: 23 August 2026


                        Underlay for IPsec Transport
                   draft-willman-rtgwg-conduit-tunnels-01

  Abstract

     This document specifies CONDUIT, a tunnel fabric management protocol
     for tactical and enterprise networks operating over heterogeneous
     Wide Area Network (WAN) links.  CONDUIT automates the lifecycle
     management of IPsec tunnels, monitors tunnel health through active
     probing, and publishes quality metrics to enable Segment Routing over
     IPv6 (SRv6) based traffic engineering.

     CONDUIT follows a strict separation of concerns: it manages the
     underlay tunnel fabric while delegating all traffic engineering
     decisions to SRv6.  This architecture simplifies network operations,
     enables rapid adaptation to changing WAN conditions, and leverages
     standard SRv6 capabilities including Flexible Algorithm (Flex-Algo)
     and Topology-Independent Loop-Free Alternate (TI-LFA).

     The protocol is fully programmable via gRPC and mandates compliance
     with Commercial National Security Algorithm Suite 2.0 (CNSA 2.0)
     cryptographic requirements.

  Status of This Memo

     This Internet-Draft is submitted in full conformance with the
     provisions of BCP 78 and BCP 79.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF).  Note that other groups may also distribute
     working documents as Internet-Drafts.  The list of current Internet-
     Drafts is at https://datatracker.ietf.org/drafts/current/.

     Internet-Drafts are draft documents valid for a maximum of six months
     and may be updated, replaced, or obsoleted by other documents at any
     time.  It is inappropriate to use Internet-Drafts as reference
     material or to cite them other than as "work in progress."

     This Internet-Draft will expire on 23 August 2026.




  V                        Expires 23 August 2026                 [Page 1]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  Copyright Notice

     Copyright (c) 2026 IETF Trust and the persons identified as the
     document authors.  All rights reserved.

     This document is subject to BCP 78 and the IETF Trust's Legal
     Provisions Relating to IETF Documents (https://trustee.ietf.org/
     license-info) in effect on the date of publication of this document.
     Please review these documents carefully, as they describe your rights
     and restrictions with respect to this document.  Code Components
     extracted from this document must include Revised BSD License text as
     described in Section 4.e of the Trust Legal Provisions and are
     provided without warranty as described in the Revised BSD License.

  Table of Contents

     1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
       1.1.  Problem Statement . . . . . . . . . . . . . . . . . . . .   3
       1.2.  Design Philosophy . . . . . . . . . . . . . . . . . . . .   4
       1.3.  Design Goals  . . . . . . . . . . . . . . . . . . . . . .   5
       1.4.  Scope . . . . . . . . . . . . . . . . . . . . . . . . . .   5
     2.  Conventions and Terminology . . . . . . . . . . . . . . . . .   6
       2.1.  Requirements Language . . . . . . . . . . . . . . . . . .   6
       2.2.  Definitions . . . . . . . . . . . . . . . . . . . . . . .   6
       2.3.  Abbreviations . . . . . . . . . . . . . . . . . . . . . .   7
     3.  Architecture  . . . . . . . . . . . . . . . . . . . . . . . .   8
       3.1.  System Overview . . . . . . . . . . . . . . . . . . . . .   8
       3.2.  Tunnel Fabric Concept . . . . . . . . . . . . . . . . . .   8
       3.3.  Separation of Concerns  . . . . . . . . . . . . . . . . .   9
     4.  Cryptographic Requirements  . . . . . . . . . . . . . . . . .  10
       4.1.  CNSA 2.0 Compliance . . . . . . . . . . . . . . . . . . .  10
       4.2.  Mandatory Algorithms  . . . . . . . . . . . . . . . . . .  10
       4.3.  Prohibited Algorithms . . . . . . . . . . . . . . . . . .  11
       4.4.  IPsec Profile . . . . . . . . . . . . . . . . . . . . . .  12
       4.5.  Certificate Requirements  . . . . . . . . . . . . . . . .  12
       4.6.  TLS Requirements  . . . . . . . . . . . . . . . . . . . .  13
     5.  Tunnel Fabric Management  . . . . . . . . . . . . . . . . . .  13
       5.1.  Tunnel Lifecycle States . . . . . . . . . . . . . . . . .  13
       5.2.  Tunnel Establishment Policy . . . . . . . . . . . . . . .  14
       5.3.  Tunnel Naming Convention  . . . . . . . . . . . . . . . .  15
       5.4.  Full Mesh Fabric  . . . . . . . . . . . . . . . . . . . .  15
     6.  Health Monitoring . . . . . . . . . . . . . . . . . . . . . .  15
       6.1.  Probe Mechanism . . . . . . . . . . . . . . . . . . . . .  16
       6.2.  Probe Packet Format . . . . . . . . . . . . . . . . . . .  16
       6.3.  Metrics Calculation . . . . . . . . . . . . . . . . . . .  18
         6.3.1.  Round-Trip Time . . . . . . . . . . . . . . . . . . .  18
         6.3.2.  Jitter  . . . . . . . . . . . . . . . . . . . . . . .  19
         6.3.3.  Packet Loss . . . . . . . . . . . . . . . . . . . . .  19



  V                        Expires 23 August 2026                 [Page 2]


  Internet-Draft        Underlay for IPsec Transport         February 2026


       6.4.  Probe Scheduling  . . . . . . . . . . . . . . . . . . . .  19
       6.5.  Tunnel State Determination  . . . . . . . . . . . . . . .  20
     7.  Metric Publishing . . . . . . . . . . . . . . . . . . . . . .  20
       7.1.  Publishing Methods  . . . . . . . . . . . . . . . . . . .  20
       7.2.  Interface Metric Calculation  . . . . . . . . . . . . . .  21
       7.3.  IGP TE Extensions . . . . . . . . . . . . . . . . . . . .  21
       7.4.  Flex-Algo Integration . . . . . . . . . . . . . . . . . .  22
       7.5.  SRLG Assignment . . . . . . . . . . . . . . . . . . . . .  23
     8.  Control Protocol  . . . . . . . . . . . . . . . . . . . . . .  23
       8.1.  Transport . . . . . . . . . . . . . . . . . . . . . . . .  23
       8.2.  Message Header  . . . . . . . . . . . . . . . . . . . . .  24
       8.3.  Message Types . . . . . . . . . . . . . . . . . . . . . .  25
       8.4.  HELLO Message . . . . . . . . . . . . . . . . . . . . . .  25
       8.5.  WAN Descriptor  . . . . . . . . . . . . . . . . . . . . .  26
       8.6.  Message Authentication  . . . . . . . . . . . . . . . . .  28
       8.7.  Anti-Replay Protection  . . . . . . . . . . . . . . . . .  29
     9.  SRv6 Integration  . . . . . . . . . . . . . . . . . . . . . .  29
       9.1.  Tunnel Interfaces as SRv6 Adjacencies . . . . . . . . . .  29
       9.2.  Flex-Algo Tunnel Assignment . . . . . . . . . . . . . . .  30
       9.3.  Traffic Steering  . . . . . . . . . . . . . . . . . . . .  31
     10. High Availability . . . . . . . . . . . . . . . . . . . . . .  31
       10.1.  HA Architecture  . . . . . . . . . . . . . . . . . . . .  31
       10.2.  Synchronized State . . . . . . . . . . . . . . . . . . .  32
       10.3.  Failover Behavior  . . . . . . . . . . . . . . . . . . .  32
     11. gRPC API  . . . . . . . . . . . . . . . . . . . . . . . . . .  32
       11.1.  Service Overview . . . . . . . . . . . . . . . . . . . .  33
       11.2.  Authentication and Authorization . . . . . . . . . . . .  33
     12. Security Considerations . . . . . . . . . . . . . . . . . . .  33
       12.1.  Cryptographic Security . . . . . . . . . . . . . . . . .  33
       12.2.  Threat Mitigations . . . . . . . . . . . . . . . . . . .  34
       12.3.  Operational Security . . . . . . . . . . . . . . . . . .  34
     13. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  35
       13.1.  UDP Port Allocation  . . . . . . . . . . . . . . . . . .  35
       13.2.  WAN Type Registry  . . . . . . . . . . . . . . . . . . .  35
     14. References  . . . . . . . . . . . . . . . . . . . . . . . . .  35
       14.1.  Normative References . . . . . . . . . . . . . . . . . .  35
       14.2.  Informative References . . . . . . . . . . . . . . . . .  36
     Appendix A.  WAN Type Characteristics . . . . . . . . . . . . . .  37
     Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  37
     Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  37

  1.  Introduction

  1.1.  Problem Statement

     Modern tactical and enterprise networks increasingly operate over
     diverse Wide Area Network (WAN) links with varying characteristics:




  V                        Expires 23 August 2026                 [Page 3]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  Satellite Communications (SATCOM): Both Geostationary (GEO) with
        approximately 600ms round-trip latency and Low Earth Orbit (LEO)
        with 20-40ms latency, limited bandwidth, and weather sensitivity.

     *  Line-of-Sight (LOS) Radio: Low latency, high mobility, but
        terrain-dependent coverage.

     *  Cellular Networks (LTE/5G): Variable quality, broad coverage, but
        may traverse untrusted infrastructure.

     *  High Frequency (HF) Radio: Very low bandwidth but extended range
        with atmospheric effects.

     *  Wired Connections: When available at fixed installations.

     Managing IPsec tunnels across these heterogeneous links presents
     significant operational challenges:

     1.  Links appear and disappear as network elements move or
         environmental conditions change.

     2.  Link quality varies dramatically and unpredictably.

     3.  Multiple paths to each destination may exist simultaneously.

     4.  Failover must complete within sub-second timeframes to maintain
         voice and command-and-control (C2) applications.

     5.  All traffic must be encrypted using approved cryptographic
         algorithms.

     Existing approaches typically couple tunnel management with traffic
     engineering decisions, creating unnecessary complexity and limiting
     interoperability with standard routing protocols.

  1.2.  Design Philosophy

     CONDUIT adheres to a strict separation of concerns:

     *  SRv6 Traffic Engineering Layer: Responsible for all path selection
        decisions, including Flexible Algorithm computation, fast reroute
        via TI-LFA, traffic class steering, and load balancing across
        equal-cost paths.

     *  CONDUIT Tunnel Fabric Layer: Responsible for tunnel lifecycle
        management (creation, maintenance, deletion), health monitoring
        through active probing, metric publication to enable SRv6-based
        decisions, and IPsec security association management.



  V                        Expires 23 August 2026                 [Page 4]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     This separation ensures that CONDUIT does not duplicate functionality
     available in standard SRv6 implementations while providing essential
     tunnel management capabilities that SRv6 requires but does not
     natively provide.

  1.3.  Design Goals

     The following quantitative goals guide CONDUIT's design:

      +=====================+===========================================+
      | Parameter           | Requirement                               |
      +=====================+===========================================+
      | Tunnel Scale        | Support for 1000+ tunnels per node        |
      +---------------------+-------------------------------------------+
      | Failover Detection  | Less than 500ms to detect tunnel failure  |
      +---------------------+-------------------------------------------+
      | Metric Publish Rate | 1 Hz default, 10 Hz burst capability      |
      +---------------------+-------------------------------------------+
      | Cryptographic Suite | Full CNSA 2.0 compliance                  |
      +---------------------+-------------------------------------------+
      | API Coverage        | 100% of functionality accessible via gRPC |
      +---------------------+-------------------------------------------+
      | Probe Overhead      | Less than 1% of available bandwidth       |
      +---------------------+-------------------------------------------+

                                    Table 1

  1.4.  Scope

     This document specifies:

     *  Tunnel lifecycle management procedures

     *  Health monitoring through active probing

     *  Metric calculation and publishing mechanisms

     *  Control protocol message formats

     *  Integration requirements for SRv6-based traffic engineering

     *  Cryptographic requirements for CNSA 2.0 compliance

     This document does not specify:

     *  SRv6 data plane operations (covered by [RFC8986])





  V                        Expires 23 August 2026                 [Page 5]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  Flexible Algorithm computation (covered by draft-ietf-lsr-flex-
        algo)

     *  IPsec key exchange procedures (covered by [RFC7296])

     *  IGP extensions for traffic engineering (covered by [RFC8570])

  2.  Conventions and Terminology

  2.1.  Requirements Language

     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
     "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
     "OPTIONAL" in this document are to be interpreted as described in BCP
     14 [RFC2119] [RFC8174] when, and only when, they appear in all
     capitals, as shown here.

  2.2.  Definitions

     Node:  A device implementing the CONDUIT protocol.


     Peer:  A remote node with which tunnels are established.


     WAN:  A Wide Area Network interface, either physical or virtual,

      capable of carrying IP traffic to remote peers.

     Tunnel:  An IPsec-protected communication path between two WAN

      interfaces on different nodes.

     Fabric:  The complete set of tunnels managed by a CONDUIT node.


     Probe:  A measurement packet transmitted through a tunnel to assess

      its quality characteristics.

     Metric:  A quantitative measure of tunnel quality, such as latency,

      jitter, or packet loss.

     Locator:  The SRv6 locator block assigned to a node, typically a /48

      IPv6 prefix.




  V                        Expires 23 August 2026                 [Page 6]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  2.3.  Abbreviations

     CNSA:  Commercial National Security Algorithm Suite


     ECMP:  Equal-Cost Multi-Path


     ESP:  Encapsulating Security Payload


     Flex-Algo:  Flexible Algorithm


     GEO:  Geostationary Earth Orbit


     gRPC:  gRPC Remote Procedure Call


     HMAC:  Hash-based Message Authentication Code


     IGP:  Interior Gateway Protocol


     IKE:  Internet Key Exchange


     LEO:  Low Earth Orbit


     LOS:  Line-of-Sight


     mTLS:  Mutual Transport Layer Security


     RTT:  Round-Trip Time


     SA:  Security Association


     SPI:  Security Parameter Index


     SRLG:  Shared Risk Link Group



  V                        Expires 23 August 2026                 [Page 7]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     SRv6:  Segment Routing over IPv6


     TE:  Traffic Engineering


     TI-LFA: Topology-Independent Loop-Free Alternate

  3.  Architecture

  3.1.  System Overview

     A CONDUIT deployment consists of the following components:

     *  SDN Controller (Optional): Provides centralized policy management,
        SRv6 policy configuration, and network-wide optimization.
        Communicates with CONDUIT nodes via gRPC.

     *  SRv6 Data Plane: Executes traffic engineering decisions using
        standard SRv6 mechanisms including IS-IS or OSPFv3 with SRv6
        extensions, Flexible Algorithm for constraint-based path
        computation, and TI-LFA for sub-50ms fast reroute.

     *  CONDUIT Daemon: Manages the tunnel fabric, comprising the Tunnel
        Lifecycle Manager for creating and deleting tunnels, Health
        Monitor for probing tunnel quality, Metric Publisher for feeding
        metrics to SRv6/IGP, and the gRPC API Server for external control.

     *  IKEv2/IPsec Subsystem: Handles cryptographic operations and
        security association management.  CONDUIT interfaces with this
        subsystem but does not implement cryptographic operations
        directly.

     *  WAN Interfaces: Physical or virtual interfaces connecting to
        various transport networks.

  3.2.  Tunnel Fabric Concept

     CONDUIT creates tunnels between WAN interfaces according to the
     configured tunnel policy.  In a full mesh configuration with N local
     WANs and M remote WANs to a given peer, CONDUIT establishes up to N x
     M tunnels.

     For example, consider a node with three WAN interfaces (SATCOM, LOS
     Radio, and LTE) connecting to a peer also having three WAN
     interfaces.  A full mesh fabric would comprise nine tunnels:

     *  SATCOM to SATCOM



  V                        Expires 23 August 2026                 [Page 8]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  SATCOM to LOS

     *  SATCOM to LTE

     *  LOS to SATCOM

     *  LOS to LOS

     *  LOS to LTE

     *  LTE to SATCOM

     *  LTE to LOS

     *  LTE to LTE

     Each tunnel becomes a distinct interface visible to the SRv6 data
     plane and IGP.  The IGP sees multiple adjacencies to the same peer,
     each with potentially different metrics based on tunnel quality.

  3.3.  Separation of Concerns

     CONDUIT explicitly delegates the following functions to SRv6:

       +========================+======================================+
       | Function               | Mechanism                            |
       +========================+======================================+
       | Path Selection         | SRv6 Flex-Algo, IGP SPF computation  |
       +------------------------+--------------------------------------+
       | Traffic Classification | SRv6 Policy color matching           |
       +------------------------+--------------------------------------+
       | Load Balancing         | IGP ECMP across equal-metric tunnels |
       +------------------------+--------------------------------------+
       | Fast Reroute           | TI-LFA pre-computed backup paths     |
       +------------------------+--------------------------------------+
       | QoS Marking            | SRv6 Traffic Class field             |
       +------------------------+--------------------------------------+

                                    Table 2

     CONDUIT is responsible for:










  V                        Expires 23 August 2026                 [Page 9]


  Internet-Draft        Underlay for IPsec Transport         February 2026


       +=====================+========================================+
       | Function            | Mechanism                              |
       +=====================+========================================+
       | Tunnel Creation     | Automatic based on WAN availability    |
       +---------------------+----------------------------------------+
       | Tunnel Deletion     | Automatic when WAN becomes unavailable |
       +---------------------+----------------------------------------+
       | Health Monitoring   | Active probing with configurable rates |
       +---------------------+----------------------------------------+
       | Metric Calculation  | RTT, jitter, loss measurement          |
       +---------------------+----------------------------------------+
       | Metric Publishing   | Interface metrics, IGP TE extensions   |
       +---------------------+----------------------------------------+
       | IPsec SA Management | Via IKEv2 with CNSA 2.0 parameters     |
       +---------------------+----------------------------------------+

                                   Table 3

  4.  Cryptographic Requirements

  4.1.  CNSA 2.0 Compliance

     CONDUIT implementations MUST comply with the Commercial National
     Security Algorithm Suite 2.0 (CNSA 2.0) as specified by the National
     Security Agency for protection of classified information.

     Implementations intended for use in environments not requiring CNSA
     2.0 compliance MAY support additional algorithms but MUST default to
     CNSA 2.0 algorithms and MUST provide configuration options to enforce
     CNSA 2.0-only operation.

  4.2.  Mandatory Algorithms

     The following algorithms are REQUIRED:

















  V                        Expires 23 August 2026                [Page 10]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     +======================+==============+============================+
     | Function             | Algorithm    | Parameters                 |
     +======================+==============+============================+
     | Symmetric Encryption | AES-256-GCM  | 256-bit key, 96-bit IV,    |
     |                      |              | 128-bit authentication tag |
     +----------------------+--------------+----------------------------+
     | Key Exchange         | ECDH         | P-384 curve (secp384r1)    |
     +----------------------+--------------+----------------------------+
     | Digital Signature    | ECDSA        | P-384 curve (secp384r1)    |
     +----------------------+--------------+----------------------------+
     | Hash Function        | SHA-384      | 384-bit output             |
     +----------------------+--------------+----------------------------+
     | Message              | HMAC-SHA-384 | 384-bit output             |
     | Authentication       | [RFC4868]    |                            |
     +----------------------+--------------+----------------------------+
     | Pseudo-Random        | HMAC-SHA-384 | Per [RFC4868]              |
     | Function (IKEv2)     |              |                            |
     +----------------------+--------------+----------------------------+
     | Key Derivation       | HKDF-SHA-384 | Per [RFC5869]              |
     +----------------------+--------------+----------------------------+

                                   Table 4

  4.3.  Prohibited Algorithms

     CONDUIT implementations MUST reject configuration of the following
     algorithms and MUST NOT negotiate their use:

     *  AES with key sizes less than 256 bits (AES-128, AES-192)

     *  Triple DES (3DES) and DES

     *  SHA-1 and SHA-256 for authentication or integrity

     *  MD5

     *  RSA with key sizes less than 3072 bits

     *  Diffie-Hellman groups with less than 3072-bit modulus

     *  ECDH or ECDSA with curves smaller than P-384

     *  ChaCha20-Poly1305 (not approved for CNSA)

     Implementations MUST generate a configuration error and refuse to
     operate if prohibited algorithms are specified.





  V                        Expires 23 August 2026                [Page 11]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  4.4.  IPsec Profile

     CONDUIT implementations MUST configure IPsec with the following
     parameters:

     IKEv2 Parameters:

     *  Version: IKEv2 only (version 1 MUST NOT be used)

     *  Encryption: AES-256-GCM with 16-octet ICV (aes256gcm16)

     *  PRF: HMAC-SHA-384 (prf-hmac-sha384)

     *  Diffie-Hellman Group: 20 (384-bit random ECP, P-384)

     *  Authentication: ECDSA with SHA-384 using P-384 certificates

     ESP (Child SA) Parameters:

     *  Encryption: AES-256-GCM with 16-octet ICV

     *  Extended Sequence Numbers: Enabled

     *  Traffic Selectors: As required for overlay traffic

     Lifetime Parameters:

     *  IKE SA Lifetime: 86400 seconds (24 hours) maximum

     *  Child SA Lifetime: 28800 seconds (8 hours) maximum

     *  Child SA Volume Limit: 1 TiB maximum

  4.5.  Certificate Requirements

     Node certificates MUST meet the following requirements:

     *  Format: X.509 version 3

     *  Key Algorithm: ECDSA with P-384 curve

     *  Signature Algorithm: ecdsa-with-SHA384

     *  Key Usage: digitalSignature, keyAgreement

     *  Extended Key Usage: serverAuth, clientAuth, IPsecIKE





  V                        Expires 23 August 2026                [Page 12]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     Certificate Authority certificates MUST use ECDSA with P-384 and
     SHA-384 signatures.

     Certificate validation MUST include revocation checking via CRL or
     OCSP when network connectivity permits.

  4.6.  TLS Requirements

     The gRPC API MUST be protected using TLS with the following
     configuration:

     *  Minimum Version: TLS 1.3 (TLS 1.2 and earlier MUST NOT be used)

     *  Cipher Suite: TLS_AES_256_GCM_SHA384 only

     *  Certificate Type: ECDSA with P-384 curve

     *  Client Authentication: Required (mutual TLS)

     Implementations MUST reject TLS connections that do not meet these
     requirements.

  5.  Tunnel Fabric Management

  5.1.  Tunnel Lifecycle States

     Each tunnel exists in one of the following states:

     DISCOVERED:  A peer has been discovered but tunnel establishment has

        not begun.  Entry: peer discovery.  Exit: initiate IKEv2.

     INITIATING:  IKEv2 negotiation is in progress.  Entry: IKEv2

        initiation.  Exit: IKE_SA and CHILD_SA established, or timeout/
        failure.

     ESTABLISHED:  The tunnel is operational and probes are succeeding

        within acceptable thresholds.  Entry: successful IKEv2 completion
        or recovery from DEGRADED.  Exit: metrics degrade, probe failures
        accumulate, or rekey initiated.

     DEGRADED:  The tunnel is operational but metrics exceed preferred

        thresholds.  Entry: metrics exceed degradation thresholds.  Exit:
        metrics recover or probe failures accumulate.




  V                        Expires 23 August 2026                [Page 13]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     REKEYING:  Security association rekeying is in progress.  Entry:

        rekey initiated (time or volume based).  Exit: rekey success or
        failure.

     DOWN:  The tunnel has failed and is not currently usable.  Entry:

        consecutive probe failures exceed threshold or IKE failure.  Exit:
        probes succeed or WAN becomes unavailable.

     DELETED:  The tunnel is marked for removal.  Entry: WAN unavailable

        or administrative deletion.  Exit: cleanup complete.

     State transitions MUST be reported via the gRPC API event stream and
     SHOULD be logged for operational visibility.

  5.2.  Tunnel Establishment Policy

     CONDUIT supports three fabric modes:

     Full Mesh:  Create tunnels for all combinations of local and remote

        WAN interfaces.  This mode maximizes path diversity but increases
        resource consumption.

     Primary Only:  Create tunnels only between designated primary WAN

        interfaces on each node.  This mode minimizes resource consumption
        but limits path diversity.

     Custom:  Apply a rule-based policy to determine which tunnel

        combinations to establish.  Rules may specify:

        *  Local WAN type (e.g., SATCOM, LOS, cellular)

        *  Remote WAN type

        *  Action (create or skip)

        *  Priority (affects establishment order)

     The tunnel policy SHOULD be configurable via the gRPC API and SHOULD
     support runtime modification without service interruption.






  V                        Expires 23 August 2026                [Page 14]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  5.3.  Tunnel Naming Convention

     Tunnel interfaces SHOULD be named according to the following
     convention to facilitate operational identification:

        tun-<peer>-<local_wan>-<remote_wan>

     Where:

     *  <peer> is a short identifier for the remote node

     *  <local_wan> is a short identifier for the local WAN interface

     *  <remote_wan> is a short identifier for the remote WAN interface

     Example names:

     *  tun-hq-sat-sat (to headquarters, SATCOM to SATCOM)

     *  tun-hq-los-lte (to headquarters, LOS radio to LTE)

     *  tun-fwd1-hf-hf (to forward unit 1, HF to HF)

  5.4.  Full Mesh Fabric

     When operating in full mesh mode, the number of tunnels T to a single
     peer is:

        T = L x R

     Where L is the number of local WAN interfaces and R is the number of
     remote WAN interfaces on the peer.

     The total tunnels across all peers is:

        Total = sum(L x R_i) for each peer i

     Implementations MUST support configurable limits on:

     *  Maximum tunnels per peer

     *  Maximum total tunnels

     When limits would be exceeded, implementations SHOULD prioritize
     tunnel creation based on configured policy and WAN type preferences.

  6.  Health Monitoring




  V                        Expires 23 August 2026                [Page 15]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  6.1.  Probe Mechanism

     CONDUIT monitors tunnel health through active probing.  Each tunnel
     is independently probed to measure:

           +=================+==============+=====================+
           | Metric          | Unit         | Collection Method   |
           +=================+==============+=====================+
           | Round-Trip Time | Microseconds | Timestamp echo      |
           +-----------------+--------------+---------------------+
           | Jitter          | Microseconds | [RFC3550] algorithm |
           +-----------------+--------------+---------------------+
           | Packet Loss     | Percentage   | Sequence tracking   |
           +-----------------+--------------+---------------------+
           | Availability    | Percentage   | Probe success rate  |
           +-----------------+--------------+---------------------+

                                   Table 5

  6.2.  Probe Packet Format

     CONDUIT probe packets use the following format:





























  V                        Expires 23 August 2026                [Page 16]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                     Magic (0x434E4454)                        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |    Version    |     Type      |            Flags              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                      Sequence Number                          |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                    TX Timestamp (64-bit)                      +
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                    RX Timestamp (64-bit)                      +
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                                                               +
     |                                                               |
     +                      HMAC-SHA-384                             +
     |                        (384 bits)                             |
     +                                                               +
     |                                                               |
     +                                                               +
     |                                                               |
     +                                                               +
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                     Padding (variable)                        +
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Field descriptions:

     Magic (32 bits):  Fixed value 0x434E4454 ("CNDT" in ASCII).  Used to

        identify CONDUIT probe packets.

     Version (8 bits):  Protocol version.  This specification defines

        version 1.

     Type (8 bits):  Probe type:

        *  0x01: Echo Request




  V                        Expires 23 August 2026                [Page 17]


  Internet-Draft        Underlay for IPsec Transport         February 2026


        *  0x02: Echo Reply

        *  0x03: Timestamp Request

        *  0x04: Timestamp Reply

     Flags (16 bits):  Reserved for future use.  MUST be set to zero on

        transmission and ignored on receipt.

     Sequence Number (32 bits):  Monotonically increasing sequence number

        used for loss detection and RTT matching.

     TX Timestamp (64 bits):  Transmission timestamp in microseconds since

        the Unix epoch.  Set by the sender.

     RX Timestamp (64 bits):  Reception timestamp in microseconds since

        the Unix epoch.  Set by the responder in reply packets; zero in
        request packets.

     HMAC-SHA-384 (384 bits):  Message authentication code computed over

        all preceding fields.  Provides probe authenticity and integrity.

     Padding (variable):  Optional padding to achieve desired probe size

        for bandwidth estimation or MTU testing.

  6.3.  Metrics Calculation

  6.3.1.  Round-Trip Time

     RTT is calculated as:

        RTT = T_local_rx - T_local_tx

     Where T_local_rx is the local time when the reply was received and
     T_local_tx is the local time when the request was sent.

     For more accurate one-way delay measurement when nodes have
     synchronized clocks:

        OWD_forward = T_remote_rx - T_local_tx
        OWD_reverse = T_local_rx - T_remote_tx




  V                        Expires 23 August 2026                [Page 18]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  6.3.2.  Jitter

     Jitter is calculated using the algorithm specified in [RFC3550]:

        D(i) = (R(i) - R(i-1)) - (S(i) - S(i-1))
        J(i) = J(i-1) + (|D(i)| - J(i-1)) / 16

     Where:

     *  R(i) is the receive timestamp of probe i

     *  S(i) is the send timestamp of probe i

     *  J(i) is the jitter estimate after probe i

  6.3.3.  Packet Loss

     Packet loss is calculated over a sliding window:
        Loss% = ((Probes_sent - Probes_received) / Probes_sent) x 100

     The default window size is 100 probes.  Implementations SHOULD make
     this configurable.

  6.4.  Probe Scheduling

     The base probe interval is configurable, with a default of 100ms.

     The effective probe interval SHOULD be adjusted based on tunnel
     state:

              +=============+============+======================+
              | State       | Multiplier | Rationale            |
              +=============+============+======================+
              | Established | 1.0        | Normal monitoring    |
              +-------------+------------+----------------------+
              | Degraded    | 0.5        | Increased monitoring |
              |             |            | of troubled tunnel   |
              +-------------+------------+----------------------+
              | Recovering  | 0.25       | Rapid assessment     |
              |             |            | during recovery      |
              +-------------+------------+----------------------+
              | Down        | 2.0        | Reduced load on      |
              |             |            | failed path          |
              +-------------+------------+----------------------+

                                    Table 6





  V                        Expires 23 August 2026                [Page 19]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     To prevent probe synchronization across tunnels, implementations
     SHOULD add random jitter of +/- 10% to each probe interval.

  6.5.  Tunnel State Determination

     Tunnel state transitions are triggered by threshold crossings:

     Degraded Thresholds:

     *  RTT increases by more than 50% above baseline

     *  Packet loss exceeds 1%

     *  Jitter increases by more than 100% above baseline

     Down Thresholds:

     *  Five or more consecutive probe failures

     *  Packet loss exceeds 25%

     Thresholds SHOULD be configurable via the gRPC API.

  7.  Metric Publishing

  7.1.  Publishing Methods

     CONDUIT publishes tunnel metrics through multiple mechanisms to
     enable SRv6-based traffic engineering:

        +====================+==================+=====================+
        | Method             | Target           | Use Case            |
        +====================+==================+=====================+
        | Interface Metrics  | Operating system | Simple deployments  |
        +--------------------+------------------+---------------------+
        | IGP TE Extensions  | IS-IS / OSPF     | Distributed TE      |
        +--------------------+------------------+---------------------+
        | gRPC Streaming     | SDN Controller   | Centralized TE      |
        +--------------------+------------------+---------------------+
        | Flex-Algo Affinity | IGP Flex-Algo    | Constraint-based TE |
        +--------------------+------------------+---------------------+

                                    Table 7

     Implementations MUST support interface metric publishing.
     Implementations SHOULD support IGP TE extensions and gRPC streaming.





  V                        Expires 23 August 2026                [Page 20]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  7.2.  Interface Metric Calculation

     CONDUIT calculates an interface metric from tunnel quality
     measurements using the following formula:

        Metric = Latency_component + Loss_component + Jitter_component

     Where:

     *  Latency_component = RTT_ms (1 ms = 1 metric unit)

     *  Loss_component = Loss% x 100 (1% loss = 100 metric units)

     *  Jitter_component = Jitter_ms x 10 (1 ms jitter = 10 metric units)

     The resulting metric MUST be clamped to the range 1-65535.

     Example calculations:

              +==============+========+======+========+========+
              | Tunnel Type  | RTT    | Loss | Jitter | Metric |
              +==============+========+======+========+========+
              | GEO SATCOM   | 600 ms | 0.1% | 10 ms  | 710    |
              +--------------+--------+------+--------+--------+
              | LOS Radio    | 5 ms   | 0%   | 0.5 ms | 10     |
              +--------------+--------+------+--------+--------+
              | LTE Cellular | 50 ms  | 0.5% | 5 ms   | 150    |
              +--------------+--------+------+--------+--------+

                                   Table 8

     The SRv6 data plane, seeing these metrics, will naturally prefer the
     LOS tunnel (metric 10) over LTE (metric 150) over SATCOM (metric 710)
     when computing shortest paths.

  7.3.  IGP TE Extensions

     When IGP TE metric publishing is enabled, CONDUIT advertises the
     following per-tunnel attributes:

     *  TE Metric: Calculated as described in Section 7.2

     *  Maximum Link Bandwidth: Configured WAN bandwidth

     *  Available Bandwidth: Estimated available capacity

     *  Unidirectional Link Delay: Measured one-way delay per [RFC8570]




  V                        Expires 23 August 2026                [Page 21]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  Unidirectional Delay Variation: Measured jitter per [RFC8570]

     *  Unidirectional Packet Loss: Measured loss per [RFC8570]

     *  Shared Risk Link Groups: Assigned SRLG values

     *  Administrative Group: For Flex-Algo affinity matching

     For IS-IS, these attributes are advertised using the extensions
     defined in [RFC8570].  For OSPF, the equivalent extensions from
     [RFC7471] apply.

  7.4.  Flex-Algo Integration

     CONDUIT assigns tunnels to Flexible Algorithm groups based on
     measured characteristics and configured constraints.

     A Flex-Algo definition specifies:

     *  Algorithm ID: An integer in the range 128-255

     *  Metric Type: IGP metric, minimum delay, or TE metric

     *  Constraints: Maximum delay, minimum bandwidth, SRLG exclusions

     CONDUIT evaluates each tunnel against each Flex-Algo definition and
     sets the appropriate affinity bits in IGP advertisements.

     Example Flex-Algo definitions for tactical networks:

     Flex-Algo 128 (Low Latency):

     *  Metric Type: Minimum Delay

     *  Constraint: Maximum delay 100 ms

     *  Use: Voice, real-time C2

     Flex-Algo 129 (High Bandwidth):

     *  Metric Type: IGP

     *  Constraint: Minimum bandwidth 1 Mbps

     *  Use: Video, bulk transfer

     Flex-Algo 130 (Resilient):




  V                        Expires 23 August 2026                [Page 22]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  Metric Type: IGP

     *  Constraint: Exclude SRLG "satcom"

     *  Use: Critical C2 requiring terrestrial diversity

     With these definitions, the SRv6 data plane automatically steers
     voice traffic over low-latency tunnels (LOS radio), video over high-
     bandwidth tunnels (possibly ECMP across multiple), and critical C2
     over tunnels not sharing SATCOM failure modes.

  7.5.  SRLG Assignment

     Shared Risk Link Groups identify tunnels that share common failure
     modes.  CONDUIT assigns SRLG values based on:

     *  WAN Type: All tunnels using SATCOM share a "satcom" SRLG

     *  Physical Path: Tunnels using the same physical link share an SRLG

     *  Provider: Tunnels using the same carrier share an SRLG

     *  Geographic: Tunnels transiting the same geographic area may share
        an SRLG

     SRLG assignment enables SRv6 TI-LFA to compute backup paths that
     avoid shared failure modes and Flex-Algo to exclude entire failure
     domains.

  8.  Control Protocol

  8.1.  Transport

     CONDUIT control messages are transported via UDP.  Two ports are
     used:

     *  Port 4794: Control messages (HELLO, WAN_UPDATE, etc.)

     *  Port 4795: Probe packets

     Control messages are transmitted through established IPsec tunnels
     when available.  For initial peer discovery before tunnel
     establishment, messages may be sent via the underlying WAN with HMAC
     authentication.







  V                        Expires 23 August 2026                [Page 23]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  8.2.  Message Header

     All CONDUIT control messages share a common header:

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |    Version    |     Type      |            Flags              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |            Length             |           Reserved            |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                         Node ID                               +
     |                         (64 bits)                             |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                      Sequence Number                          |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                      Timestamp (64 bits)                      +
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                                                               +
     |                                                               |
     +                      HMAC-SHA-384                             +
     |                        (384 bits)                             |
     +                                                               +
     |                                                               |
     +                                                               +
     |                                                               |
     +                                                               +
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Field descriptions:

     Version (8 bits):  Protocol version.  This specification defines

        version 1.

     Type (8 bits):  Message type as defined in Section 8.3.


     Flags (16 bits):  Message-specific flags.


     Length (16 bits):  Total message length in octets, including header.




  V                        Expires 23 August 2026                [Page 24]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     Reserved (16 bits): Reserved for future use.  MUST be zero.

     Node ID (64 bits):  Unique identifier of the sending node.


     Sequence Number (32 bits):  Per-peer monotonically increasing

      counter for anti-replay protection.

     Timestamp (64 bits):  Message creation time in microseconds since

      Unix epoch.

     HMAC-SHA-384 (384 bits):  Message authentication code computed over

      the entire message with the HMAC field set to zero.

     Total header size: 72 octets.

  8.3.  Message Types

       +=======+===============+======================================+
       | Value | Name          | Description                          |
       +=======+===============+======================================+
       | 0x01  | HELLO         | Peer discovery and WAN advertisement |
       +-------+---------------+--------------------------------------+
       | 0x02  | HELLO_ACK     | Response to HELLO                    |
       +-------+---------------+--------------------------------------+
       | 0x03  | WAN_UPDATE    | WAN availability change notification |
       +-------+---------------+--------------------------------------+
       | 0x04  | TUNNEL_STATE  | Tunnel state change notification     |
       +-------+---------------+--------------------------------------+
       | 0x05  | KEEPALIVE     | Peer liveness check                  |
       +-------+---------------+--------------------------------------+
       | 0x06  | KEEPALIVE_ACK | Response to KEEPALIVE                |
       +-------+---------------+--------------------------------------+
       | 0x0F  | ERROR         | Error notification                   |
       +-------+---------------+--------------------------------------+

                                   Table 9

  8.4.  HELLO Message

     The HELLO message is used for peer discovery and WAN capability
     advertisement:






  V                        Expires 23 August 2026                [Page 25]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |   Num WANs    |   Reserved    |         Capabilities          |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |        Hold Time (sec)        |        Max Tunnels            |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                    SRv6 Locator (48 bits)                     +
     |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                               |           Reserved            |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     ~                      WAN Descriptors                          ~
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Field descriptions:

     Num WANs (8 bits):  Number of WAN Descriptors following.


     Capabilities (16 bits):  Bitmask of supported capabilities.


     Hold Time (16 bits):  Time in seconds the receiver should consider

      the sender reachable without further messages.

     Max Tunnels (16 bits):  Maximum tunnels this node can establish to

      a single peer.

     SRv6 Locator (48 bits):  The node's SRv6 locator prefix.


  8.5.  WAN Descriptor

     Each WAN interface is described by a WAN Descriptor:












  V                        Expires 23 August 2026                [Page 26]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |    WAN ID     |   WAN Type    |     State     | Addr Flags    |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                      Bandwidth (Kbps)                         |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |              MTU              |     Typical Latency (ms)      |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     +                  IPv6 Address (if flag set)                   +
     |                        (128 bits)                             |
     +                                                               +
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                  IPv4 Address (if flag set)                   |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |            SRLG ID            |           Reserved            |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     WAN Type values:






























  V                        Expires 23 August 2026                [Page 27]


  Internet-Draft        Underlay for IPsec Transport         February 2026


              +=======+===============+=========================+
              | Value | Name          | Typical Characteristics |
              +=======+===============+=========================+
              | 0x01  | SATCOM_GEO    | 600ms RTT, 2-10 Mbps    |
              +-------+---------------+-------------------------+
              | 0x02  | SATCOM_LEO    | 30ms RTT, 50-200 Mbps   |
              +-------+---------------+-------------------------+
              | 0x03  | LOS_RADIO     | 5ms RTT, 10-100 Mbps    |
              +-------+---------------+-------------------------+
              | 0x04  | TROPOSCATTER  | 10ms RTT, 5-20 Mbps     |
              +-------+---------------+-------------------------+
              | 0x05  | HF_RADIO      | 50ms RTT, 9.6-64 Kbps   |
              +-------+---------------+-------------------------+
              | 0x06  | CELLULAR_LTE  | 30ms RTT, 10-100 Mbps   |
              +-------+---------------+-------------------------+
              | 0x07  | CELLULAR_5G   | 10ms RTT, 100-1000 Mbps |
              +-------+---------------+-------------------------+
              | 0x08  | WIRE_ETHERNET | 1ms RTT, 1-100 Gbps     |
              +-------+---------------+-------------------------+
              | 0x09  | WIRE_FIBER    | 1ms RTT, 1-100 Gbps     |
              +-------+---------------+-------------------------+
              | 0x0A  | WIFI          | 5ms RTT, 50-500 Mbps    |
              +-------+---------------+-------------------------+

                                    Table 10

     Address Flags:

     *  Bit 0: IPv4 address present

     *  Bit 1: IPv6 address present

     *  Bit 2: Behind NAT (IPv4)

     *  Bit 3: Behind NAT (IPv6)

  8.6.  Message Authentication

     All CONDUIT control messages MUST be authenticated using HMAC- SHA-
     384.

     The authentication key is derived from the IKEv2 SA when
     available:








  V                        Expires 23 August 2026                [Page 28]


  Internet-Draft        Underlay for IPsec Transport         February 2026


        AUTH_KEY = HKDF-SHA-384(
           IKM = SK_d,
           salt = "CONDUIT-v1-auth",
           info = Node_ID_local || Node_ID_remote,
           length = 48
        )

     For initial HELLO messages before IKEv2 SA establishment, a pre-
     shared key MAY be used.  Unauthenticated HELLO messages MUST NOT be
     accepted in production deployments.

  8.7.  Anti-Replay Protection

     Each peer maintains anti-replay state consisting of:

     *  Expected sequence number (next expected value)

     *  Window bitmap (received sequences within window)

     *  Window size (default 64)

     *  Maximum timestamp skew (default 60 seconds)

     Message acceptance rules:

     1.  If sequence >= expected: Accept and update expected

     2.  If sequence < expected - window_size: Reject (too old)

     3.  If sequence in window and bit set: Reject (replay)

     4.  If sequence in window and bit clear: Accept and set bit

     5.  If timestamp differs from local time by more than maximum skew:
         Reject

  9.  SRv6 Integration

  9.1.  Tunnel Interfaces as SRv6 Adjacencies

     Each CONDUIT tunnel is presented to the SRv6 data plane as a distinct
     interface.  From the perspective of IS-IS or OSPFv3, each tunnel
     represents a separate adjacency to the peer node.

     This design enables:

     *  Independent metrics per tunnel based on measured quality




  V                        Expires 23 August 2026                [Page 29]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  ECMP load balancing across multiple tunnels to the same peer

     *  Flex-Algo differentiation based on tunnel characteristics

     *  TI-LFA backup path computation considering tunnel diversity

     For example, a node with three tunnels to a peer (via SATCOM, LOS,
     and LTE) would advertise three adjacencies in IS-IS, each with its
     own metric.  The IGP SPF computation naturally prefers lower-metric
     paths.

  9.2.  Flex-Algo Tunnel Assignment

     CONDUIT evaluates each tunnel against configured Flex-Algo
     definitions and assigns appropriate affinities.

     A tunnel qualifies for a Flex-Algo if:

     *  Measured delay is less than the maximum delay constraint (if
        specified)

     *  Configured bandwidth meets the minimum bandwidth constraint (if
        specified)

     *  Tunnel SRLG membership does not intersect with excluded SRLGs (if
        specified)

     Assignment is dynamic: as tunnel metrics change, Flex-Algo membership
     is re-evaluated and IGP advertisements are updated accordingly.

     Example assignment for a deployment with three Flex-Algos:

     +===========+====================+====================+=============+
     | Tunnel    | Algo 128 (Low Lat) | Algo 129           | Algo 130    |
     |           |                    | (High BW)          | (Resilient) |
     +===========+====================+====================+=============+
     | SATCOM    | Excluded           | Included           | Excluded    |
     | (600ms)   |                    |                    |             |
     +-----------+--------------------+--------------------+-------------+
     | LOS Radio | Included           | Included           | Included    |
     | (5ms)     |                    |                    |             |
     +-----------+--------------------+--------------------+-------------+
     | LTE       | Included           | Included           | Included    |
     | (50ms)    |                    |                    |             |
     +-----------+--------------------+--------------------+-------------+

                                    Table 11




  V                        Expires 23 August 2026                [Page 30]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     The SATCOM tunnel is excluded from Flex-Algo 128 (delay exceeds 100ms
     threshold) and Flex-Algo 130 (SRLG "satcom" is excluded).

  9.3.  Traffic Steering

     With CONDUIT providing tunnel metrics and Flex-Algo assignments,
     traffic steering is accomplished through standard SRv6 mechanisms:

     *  SRv6 Policy Color: Traffic matching a policy with a specific color
        is steered to paths computed using the associated Flex-Algo

     *  DSCP Mapping: DSCP values can be mapped to SRv6 policy colors

     *  BGP Communities: BGP communities can signal desired Flex-Algo

     Example mapping for tactical traffic:

          +==============+===========+==========+===================+
          | Traffic Type | DSCP      | SR Color | Flex-Algo         |
          +==============+===========+==========+===================+
          | Voice        | EF (46)   | 100      | 128 (Low Latency) |
          +--------------+-----------+----------+-------------------+
          | Video        | AF41 (34) | 200      | 129 (High BW)     |
          +--------------+-----------+----------+-------------------+
          | Critical C2  | CS6 (48)  | 300      | 130 (Resilient)   |
          +--------------+-----------+----------+-------------------+
          | Best Effort  | BE (0)    | 0        | Default (SPF)     |
          +--------------+-----------+----------+-------------------+

                                    Table 12

  10.  High Availability

  10.1.  HA Architecture

     CONDUIT supports high availability through active-standby or active-
     active node pairs sharing a virtual IP address.

     In active-standby mode:

     *  Primary node handles all tunnel management and peer communication

     *  Standby node maintains synchronized state and pre-established
        tunnels

     *  Failover occurs when primary failure is detected

     In active-active mode:



  V                        Expires 23 August 2026                [Page 31]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  Both nodes actively manage tunnels

     *  Anycast addressing distributes peer connections

     *  State synchronization ensures consistency

  10.2.  Synchronized State

     The following state is synchronized between HA peers:

       +===============+==================+===========================+
       | Data          | Sync Method      | Purpose                   |
       +===============+==================+===========================+
       | Peer List     | Periodic + Event | Both nodes know all peers |
       +---------------+------------------+---------------------------+
       | Tunnel State  | Event-driven     | Enable fast failover      |
       +---------------+------------------+---------------------------+
       | Metrics       | Periodic (1 Hz)  | Consistent path costs     |
       +---------------+------------------+---------------------------+
       | Configuration | Event-driven     | Policy consistency        |
       +---------------+------------------+---------------------------+

                                   Table 13

     IKEv2 Security Associations are NOT synchronized.  Each node
     maintains independent SAs with peers.  This simplifies implementation
     and avoids complexities of SA state transfer.

  10.3.  Failover Behavior

     Upon detecting primary node failure:

     1.  Standby assumes the virtual IP address

     2.  Standby's pre-established tunnels become active

     3.  SRv6 reconverges via TI-LFA (sub-50ms for pre-computed backups)

     4.  Traffic flows through standby's tunnels

     Target failover time is less than 3 seconds for complete recovery,
     with TI-LFA providing sub-50ms forwarding continuity for traffic with
     pre-computed backup paths.

  11.  gRPC API






  V                        Expires 23 August 2026                [Page 32]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  11.1.  Service Overview

     CONDUIT exposes all functionality through a gRPC API comprising three
     services:

     TunnelFabric Service:  Primary interface for tunnel management,

        including node configuration, WAN interface management, peer
        discovery and management, tunnel lifecycle operations, tunnel
        policy configuration, and metric publishing configuration.

     Telemetry Service:  Provides access to operational metrics, including

        current tunnel metrics, real-time metric streaming, historical
        metric queries, and fabric statistics.

     HighAvailability Service:  Manages HA operations, including HA status

        and configuration, synchronization status, and manual failover
        triggers.

     The complete Protocol Buffer definitions for these services are
     available in a companion document.

  11.2.  Authentication and Authorization

     All gRPC connections MUST use mutual TLS (mTLS) with certificates
     meeting the requirements of Section 4.5.

     Implementations SHOULD support role-based access control (RBAC) for
     API authorization.  Recommended roles include:

     *  Administrator: Full access to all operations

     *  Operator: Read access plus tunnel state changes

     *  Monitor: Read-only access to status and metrics

  12.  Security Considerations

  12.1.  Cryptographic Security

     CONDUIT's security relies on the strength of its cryptographic
     foundations.  The mandatory use of CNSA 2.0 algorithms provides
     protection appropriate for classified information.

     Key security properties:




  V                        Expires 23 August 2026                [Page 33]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     *  All tunnel traffic is protected by AES-256-GCM, providing
        confidentiality and integrity

     *  All control messages are authenticated using HMAC-SHA-384

     *  Key exchange uses ECDH with P-384, providing forward secrecy

     *  Node authentication uses ECDSA with P-384 certificates

  12.2.  Threat Mitigations

     CONDUIT addresses the following threats:

      +=====================+===========================================+
      | Threat              | Mitigation                                |
      +=====================+===========================================+
      | Probe Spoofing      | HMAC-SHA-384 authentication on all probes |
      +---------------------+-------------------------------------------+
      | Replay Attacks      | Sequence numbers and timestamp validation |
      +---------------------+-------------------------------------------+
      | Man-in-the-Middle   | IPsec with certificate-based auth         |
      +---------------------+-------------------------------------------+
      | Unauthorized API    | Mutual TLS with certificate validation    |
      +---------------------+-------------------------------------------+
      | Tunnel Manipulation | Authenticated control protocol messages   |
      +---------------------+-------------------------------------------+
      | Denial of Service   | Rate limiting on control plane            |
      +---------------------+-------------------------------------------+

                                    Table 14

  12.3.  Operational Security

     Deployments SHOULD implement the following operational security
     measures:

     *  Certificates should be rotated at least annually

     *  Private keys should be stored in hardware security modules (HSM)
        where available

     *  All tunnel state changes should be logged for audit purposes

     *  Rate limiting should be applied to control plane message
        processing

     *  Network segmentation should isolate CONDUIT management traffic




  V                        Expires 23 August 2026                [Page 34]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  13.  IANA Considerations

  13.1.  UDP Port Allocation

     This document requests allocation of two UDP ports:

             +======+=================+==========================+
             | Port | Name            | Description              |
             +======+=================+==========================+
             | 4794 | conduit-control | CONDUIT control messages |
             +------+-----------------+--------------------------+
             | 4795 | conduit-probe   | CONDUIT probe packets    |
             +------+-----------------+--------------------------+

                                    Table 15

  13.2.  WAN Type Registry

     This document requests creation of a "CONDUIT WAN Types" registry
     with the following initial values:

     +-------+----------------+-------------------------+
     | Value | Name           | Reference               |
     +-------+----------------+-------------------------+
     |  0x00 | Reserved       | This document           |
     |  0x01 | SATCOM_GEO     | This document           |
     |  0x02 | SATCOM_LEO     | This document           |
     |  0x03 | LOS_RADIO      | This document           |
     |  0x04 | TROPOSCATTER   | This document           |
     |  0x05 | HF_RADIO       | This document           |
     |  0x06 | CELLULAR_LTE   | This document           |
     |  0x07 | CELLULAR_5G    | This document           |
     |  0x08 | WIRE_ETHERNET  | This document           |
     |  0x09 | WIRE_FIBER     | This document           |
     |  0x0A | WIFI           | This document           |
     | 0x0B-0xEF | Unassigned |                         |
     | 0xF0-0xFF | Private Use | This document          |
     +-------+----------------+-------------------------+

     New values in the range 0x0B-0xEF require Standards Action.

  14.  References

  14.1.  Normative References







  V                        Expires 23 August 2026                [Page 35]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     [RFC7296]  Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., and T.
                Kivinen, "Internet Key Exchange Protocol Version 2
                (IKEv2)", STD 79, RFC 7296, DOI 10.17487/RFC7296, October
                2014, <https://www.rfc-editor.org/info/rfc7296>.

     [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
                2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
                May 2017, <https://www.rfc-editor.org/info/rfc8174>.

     [RFC8570]  Ginsberg, L., Ed., Previdi, S., Ed., Giacalone, S., Ward,
                D., Drake, J., and Q. Wu, "IS-IS Traffic Engineering (TE)
                Metric Extensions", RFC 8570, DOI 10.17487/RFC8570, March
                2019, <https://www.rfc-editor.org/info/rfc8570>.

     [RFC8986]  Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer,
                D., Matsushima, S., and Z. Li, "Segment Routing over IPv6
                (SRv6) Network Programming", RFC 8986,
                DOI 10.17487/RFC8986, February 2021,
                <https://www.rfc-editor.org/info/rfc8986>.

  14.2.  Informative References

     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
                Requirement Levels", BCP 14, RFC 2119,
                DOI 10.17487/RFC2119, March 1997,
                <https://www.rfc-editor.org/info/rfc2119>.

     [RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
                Jacobson, "RTP: A Transport Protocol for Real-Time
                Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
                July 2003, <https://www.rfc-editor.org/info/rfc3550>.

     [RFC4868]  Kelly, S. and S. Frankel, "Using HMAC-SHA-256, HMAC-SHA-
                384, and HMAC-SHA-512 with IPsec", RFC 4868,
                DOI 10.17487/RFC4868, May 2007,
                <https://www.rfc-editor.org/info/rfc4868>.

     [RFC5869]  Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand
                Key Derivation Function (HKDF)", RFC 5869,
                DOI 10.17487/RFC5869, May 2010,
                <https://www.rfc-editor.org/info/rfc5869>.

     [RFC7471]  Giacalone, S., Ward, D., Drake, J., Atlas, A., and S.
                Previdi, "OSPF Traffic Engineering (TE) Metric
                Extensions", RFC 7471, DOI 10.17487/RFC7471, March 2015,
                <https://www.rfc-editor.org/info/rfc7471>.





  V                        Expires 23 August 2026                [Page 36]


  Internet-Draft        Underlay for IPsec Transport         February 2026


  Appendix A.  WAN Type Characteristics

     The following table provides typical characteristics for each WAN
     type.  Actual values vary based on specific equipment, configuration,
     and environmental conditions.

     +===============+=========+===============+=========================+
     | WAN Type      | Latency | Bandwidth     | Notes                   |
     +===============+=========+===============+=========================+
     | SATCOM_GEO    | 600ms   | 2-10 Mbps     | Weather sensitive       |
     +---------------+---------+---------------+-------------------------+
     | SATCOM_LEO    | 20-40ms | 50-200 Mbps   | Handover during orbit   |
     +---------------+---------+---------------+-------------------------+
     | LOS_RADIO     | 5ms     | 10-100 Mbps   | Terrain dependent       |
     +---------------+---------+---------------+-------------------------+
     | TROPOSCATTER  | 10ms    | 5-20 Mbps     | Over-horizon capability |
     +---------------+---------+---------------+-------------------------+
     | HF_RADIO      | 50ms    | 9.6-64 Kbps   | Atmospheric effects     |
     +---------------+---------+---------------+-------------------------+
     | CELLULAR_LTE  | 30ms    | 10-100 Mbps   | Coverage dependent      |
     +---------------+---------+---------------+-------------------------+
     | CELLULAR_5G   | 10ms    | 100-1000      | Limited coverage        |
     |               |         | Mbps          |                         |
     +---------------+---------+---------------+-------------------------+
     | WIRE_ETHERNET | <1ms    | 1-100 Gbps    | Fixed installations     |
     +---------------+---------+---------------+-------------------------+
     | WIRE_FIBER    | <1ms    | 1-100 Gbps    | Fixed installations     |
     +---------------+---------+---------------+-------------------------+
     | WIFI          | 5ms     | 50-500 Mbps   | Short range             |
     +---------------+---------+---------------+-------------------------+

                                    Table 16

  Acknowledgements

     The authors thank the members of the tactical networking community
     for their input on operational requirements and deployment
     considerations.

  Author's Address

     John Edward Willman V
     Department of the Air Force
     1800 Air Force Pentagon
     Washington, DC 20330
     United States of America
     Phone: +1 786 994 3023
     Email: john.willman.1@us.af.mil



  V                        Expires 23 August 2026                [Page 37]


  Internet-Draft        Underlay for IPsec Transport         February 2026


     URI:   https://www.linkedin.com/in/johnewillmanv


















































  V                        Expires 23 August 2026                [Page 38]


