diff --git a/data/drafts.db b/data/drafts.db index cd5a2b9..3965e61 100644 Binary files a/data/drafts.db and b/data/drafts.db differ diff --git a/data/reports/gaps.md b/data/reports/gaps.md index 63e895e..03a9d86 100644 --- a/data/reports/gaps.md +++ b/data/reports/gaps.md @@ -1,125 +1,133 @@ # Gap Analysis: IETF AI/Agent Draft Landscape -*Generated 2026-03-08 15:15 UTC — analyzing 474 drafts, 498 technical ideas* +*Generated 2026-03-08 17:16 UTC — analyzing 756 drafts, 501 technical ideas* ## Overview -This report identifies **12 gaps** — areas, problems, or technical challenges not adequately addressed by the current 474 IETF AI/agent drafts. Each gap is cross-referenced with related drafts and extracted technical ideas to show partial coverage. +This report identifies **11 gaps** — areas, problems, or technical challenges not adequately addressed by the current 756 IETF AI/agent drafts. Each gap is cross-referenced with related drafts and extracted technical ideas to show partial coverage. | Severity | Count | |----------|------:| | **CRITICAL** | 3 | -| **HIGH** | 6 | +| **HIGH** | 5 | | **MEDIUM** | 3 | ### Safety Deficit -Only **46** of 474 drafts address AI safety/alignment, while **150** focus on A2A protocols and **110** on autonomous operations. The ratio of capability-building to safety is roughly **5:1**. +Only **112** of 756 drafts address AI safety/alignment, while **157** focus on A2A protocols and **124** on autonomous operations. The ratio of capability-building to safety is roughly **2:1**. --- -## 1. Real-time Agent Behavior Verification +## 1. Agent Legal Liability Framework | | | |---|---| | **Severity** | CRITICAL | -| **Category** | AI safety/alignment | -| **Drafts in category** | 46 | +| **Category** | policy/governance | +| **Drafts in category** | 214 | -Current AI safety drafts focus on governance but lack technical protocols for real-time verification that agents are behaving according to their declared policies. There's no standard way to cryptographically prove agent actions match stated intentions. +No standard addresses who is legally responsible when autonomous agents cause harm or make binding commitments. Current frameworks focus on technical risk but ignore liability assignment between agent creators, operators, and users. -**Evidence:** Only 46 safety drafts versus 474 total, with governance focus rather than technical verification +**Evidence:** Real-world AI agent deployments are stalled due to unclear liability chains, especially in financial and healthcare domains ### Related Drafts **Keyword matches** (drafts mentioning gap topic): -- [draft-an-nmrg-i2icf-cits](https://datatracker.ietf.org/doc/draft-an-nmrg-i2icf-cits/) (score 3.7) — Interface to In-Network Computing Functions for Cooperative Intelligent Transpor -- [draft-zhao-detnet-enhanced-use-cases](https://datatracker.ietf.org/doc/draft-zhao-detnet-enhanced-use-cases/) (score 3.2) — Enhanced Use Cases for Scaling Deterministic Networks -- [draft-zhang-rvp-problem-statement](https://datatracker.ietf.org/doc/draft-zhang-rvp-problem-statement/) (score 3.5) — Problem Statements and Requirements of Real-Virtual Agent Protocol (RVP): Commun -- [draft-yuan-rtgwg-traffic-agent-usecase](https://datatracker.ietf.org/doc/draft-yuan-rtgwg-traffic-agent-usecase/) (score 3.7) — Use cases of the AI Network Traffic Optimization Agent -- [draft-altanai-aipref-realtime-protocol-bindings](https://datatracker.ietf.org/doc/draft-altanai-aipref-realtime-protocol-bindings/) (score 3.6) — AI Preferences for Real-Time Protocol Bindings - [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment +- [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment +- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite +- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps +- [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks +- [draft-madhavan-aipref-displaybasedpref](https://datatracker.ietf.org/doc/draft-madhavan-aipref-displaybasedpref/) (score 2.5) — A Vocabulary for Controlling Usage of Content Collected by Search and AI Crawler -**Top-rated in AI safety/alignment** (46 drafts): +**Top-rated in policy/governance** (214 drafts): + +- [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra +- [draft-aylward-daap-v2](https://datatracker.ietf.org/doc/draft-aylward-daap-v2/) (4.8) — Defines comprehensive protocol for AI agent accountability including authentication, monitoring, and +- [itu-t-y-3172](https://datatracker.ietf.org/doc/itu-t-y-3172/) (4.7) — Establishes comprehensive architectural framework for machine learning deployment in future networks +- [iso-iec-22989-2022](https://datatracker.ietf.org/doc/iso-iec-22989-2022/) (4.7) — ISO/IEC standard defining core AI concepts and establishing standardized terminology across the fiel +- [iso-iec-42001-2023](https://datatracker.ietf.org/doc/iso-iec-42001-2023/) (4.6) — ISO/IEC standard establishing comprehensive AI management system requirements covering governance, r + +### Partially Addressing Ideas + +51 extracted ideas touch on this gap: + +| Idea | Draft | Type | +|------|-------|------| +| Structured Responsibility and Traceability Architecture | draft-takagi-srta-trinity | architecture | +| Intelligent Agent Communication Gateway Architecture | draft-agent-gw | architecture | +| Tiered Risk-Based Governance for Autonomous AI Agents | draft-aylward-aiga-1 | architecture | +| Tiered Risk-Based Governance for Autonomous AI Agents | draft-aylward-aiga-2 | architecture | +| Distributed AI Accountability Protocol | draft-aylward-daap-v2 | protocol | +| Zero Trust Runtime Agent Architecture | draft-berlinai-vera | architecture | +| Agentic Hypercall Protocol | draft-campbell-agentic-http | pattern | +| Context Distribution Optimization Framework | draft-chang-agent-context-interaction | mechanism | + +*...and 43 more* + +--- + +## 2. Agent Capability Degradation Detection + +| | | +|---|---| +| **Severity** | CRITICAL | +| **Category** | AI safety/alignment | +| **Drafts in category** | 112 | + +No standard defines how to detect when an agent's capabilities are degrading due to concept drift, adversarial inputs, or model corruption. Current monitoring focuses on system metrics not capability assessment. + +**Evidence:** Production AI systems show gradual performance degradation that goes undetected until major failures occur + +### Related Drafts + +**Keyword matches** (drafts mentioning gap topic): + +- [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment +- [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment +- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite +- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps +- [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks +- [draft-li-dmsc-inf-architecture](https://datatracker.ietf.org/doc/draft-li-dmsc-inf-architecture/) (score 3.1) — Dynamic Multi-agent Secured Collaboration Infrastructure Architecture + +**Top-rated in AI safety/alignment** (112 drafts): - [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra - [draft-aylward-daap-v2](https://datatracker.ietf.org/doc/draft-aylward-daap-v2/) (4.8) — Defines comprehensive protocol for AI agent accountability including authentication, monitoring, and - [draft-guy-bary-stamp-protocol](https://datatracker.ietf.org/doc/draft-guy-bary-stamp-protocol/) (4.6) — Defines STAMP protocol for cryptographic delegation and proof in AI agent systems. Provides task-bou - [draft-drake-email-tpm-attestation](https://datatracker.ietf.org/doc/draft-drake-email-tpm-attestation/) (4.6) — Defines hardware attestation for email using TPM verification chains to prevent spam and provide Syb -- [draft-goswami-agentic-jwt](https://datatracker.ietf.org/doc/draft-goswami-agentic-jwt/) (4.5) — Extends OAuth 2.0 with Agentic JWT to address authorization challenges in autonomous AI systems. Int +- [iso-iec-42001-2023](https://datatracker.ietf.org/doc/iso-iec-42001-2023/) (4.6) — ISO/IEC standard establishing comprehensive AI management system requirements covering governance, r ### Partially Addressing Ideas -18 extracted ideas touch on this gap: +22 extracted ideas touch on this gap: | Idea | Draft | Type | |------|-------|------| -| Distributed AI Accountability Protocol | draft-aylward-daap-v2 | protocol | -| AGENTS.TXT Policy File | draft-srijal-agents-policy | protocol | -| AI Network Security Agent | draft-yuan-rtgwg-security-agent-usecase | architecture | -| A2A Protocol Transport over MOQT | draft-a2a-moqt-transport | protocol | -| Post-Discovery Authorization Handshake | draft-barney-caam | protocol | -| Evidence-based Autonomy Maturity Model | draft-berlinai-vera | mechanism | -| Verifiable Agent Conversation Format | draft-birkholz-verifiable-agent-conversations | protocol | -| Intent-Based Just-in-Time Authorization | draft-chen-agent-decoupled-authorization-model | architecture | +| Intelligent Agent Communication Gateway Architecture | draft-agent-gw | architecture | +| AI-Native Network Protocol (AINP) | draft-ainp-protocol | protocol | +| Agentic Data Optimization Layer (ADOL) | draft-chang-agent-token-efficient | protocol | +| Structured OAuth Scope Syntax for Agent Permissions | draft-chen-oauth-scope-agent-extensions | extension | +| Capability-based Agent Discovery Mechanism | draft-cui-ai-agent-discovery-invocation | mechanism | +| Intent-based Agent Selection | draft-cui-ai-agent-discovery-invocation | extension | +| Agent Attachment Protocol | draft-dunbar-agent-attachment | protocol | +| EAT Extensions for Agent Capability Attestation | draft-huang-rats-agentic-eat-cap-attest | extension | -*...and 10 more* +*...and 14 more* --- -## 2. Multi-Agent Consensus Under Byzantine Conditions - -| | | -|---|---| -| **Severity** | CRITICAL | -| **Category** | A2A protocols | -| **Drafts in category** | 150 | - -While agent discovery and A2A protocols exist, there's no framework for handling consensus when some agents may be compromised or malicious. Critical for autonomous systems making collective decisions. - -**Evidence:** Complex autonomous systems require Byzantine fault tolerance but it's absent from protocol designs - -### Related Drafts - -**Keyword matches** (drafts mentioning gap topic): - -- [draft-li-dmsc-mcps-agw](https://datatracker.ietf.org/doc/draft-li-dmsc-mcps-agw/) (score 3.5) — Multi-agent Collaboration Protocol Suite based on Agent Gateway -- [draft-li-dmsc-inf-architecture](https://datatracker.ietf.org/doc/draft-li-dmsc-inf-architecture/) (score 3.1) — Dynamic Multi-agent Secured Collaboration Infrastructure Architecture -- [draft-yue-anima-agent-recovery-networks](https://datatracker.ietf.org/doc/draft-yue-anima-agent-recovery-networks/) (score 4.1) — Task-Oriented Multi-Agent Recovery Framework for High-Reliability in Converged M -- [draft-chang-agent-context-interaction](https://datatracker.ietf.org/doc/draft-chang-agent-context-interaction/) (score 2.9) — Agent Context Interaction Optimizations -- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps -- [draft-ramakrishna-satp-views-addresses](https://datatracker.ietf.org/doc/draft-ramakrishna-satp-views-addresses/) (score 3.4) — Views and View Addresses for Secure Asset Transfer - -**Top-rated in A2A protocols** (150 drafts): - -- [draft-guy-bary-stamp-protocol](https://datatracker.ietf.org/doc/draft-guy-bary-stamp-protocol/) (4.6) — Defines STAMP protocol for cryptographic delegation and proof in AI agent systems. Provides task-bou -- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (4.6) — Defines YANG data model for hierarchical language model coordination across tiny, small, and large L -- [draft-ietf-lake-edhoc](https://datatracker.ietf.org/doc/draft-ietf-lake-edhoc/) (4.6) — Specifies EDHOC, a compact authenticated Diffie-Hellman key exchange protocol for constrained enviro -- [draft-chang-agent-token-efficient](https://datatracker.ietf.org/doc/draft-chang-agent-token-efficient/) (4.5) — Defines ADOL (Agentic Data Optimization Layer) to address token bloat in agent communication protoco -- [draft-chen-oauth-rar-agent-extensions](https://datatracker.ietf.org/doc/draft-chen-oauth-rar-agent-extensions/) (4.2) — Extends OAuth RAR with policy_context and lifecycle_binding members for AI agent environments. Enabl - -### Partially Addressing Ideas - -2 extracted ideas touch on this gap: - -| Idea | Draft | Type | -|------|-------|------| -| ASRank Structural Vulnerability Analysis | draft-xu-sidrops-asrank-vulnerabilities | requirement | -| MCP and A2A Complementary Solutions for Network Management | draft-zeng-opsawg-applicability-mcp-a2a | architecture | - ---- - -## 3. Emergency Agent Shutdown Coordination +## 3. Emergency Agent Override Protocols | | | |---|---| | **Severity** | CRITICAL | | **Category** | AI safety/alignment | -| **Drafts in category** | 46 | +| **Drafts in category** | 112 | -Missing protocols for coordinated emergency shutdown of autonomous agent networks when safety issues are detected. Individual agent controls exist but not network-wide coordination mechanisms. +No standard defines how to safely emergency-stop or override autonomous agents across distributed systems when they exhibit dangerous behavior. Current approaches assume centralized control that may not exist. -**Evidence:** Human-in-the-loop drafts exist but no emergency coordination protocols for autonomous systems +**Evidence:** Incidents with autonomous trading systems and industrial controls show need for fail-safe override mechanisms ### Related Drafts @@ -128,177 +136,48 @@ Missing protocols for coordinated emergency shutdown of autonomous agent network - [draft-aylward-daap-v2](https://datatracker.ietf.org/doc/draft-aylward-daap-v2/) (score 4.8) — Distributed AI Accountability Protocol (DAAP) Version 2.0 - [draft-khatri-sipcore-call-transfer-fail-response](https://datatracker.ietf.org/doc/draft-khatri-sipcore-call-transfer-fail-response/) (score 3.3) — A SIP Response Code (497) for Call Transfer Failure - [draft-cui-dmsc-agent-cdi](https://datatracker.ietf.org/doc/draft-cui-dmsc-agent-cdi/) (score 3.0) — Cross-Domain Interoperability Framework for AI Agent Collaboration -- [draft-yu-ai-agent-use-cases-in-6g](https://datatracker.ietf.org/doc/draft-yu-ai-agent-use-cases-in-6g/) (score 2.5) — AI Agent Use Cases and Requirements in 6G Network -- [draft-zhang-rvp-problem-statement](https://datatracker.ietf.org/doc/draft-zhang-rvp-problem-statement/) (score 3.5) — Problem Statements and Requirements of Real-Virtual Agent Protocol (RVP): Commun +- [iso-22733-1-2021](https://datatracker.ietf.org/doc/iso-22733-1-2021/) (score 4.3) — ISO 22733-1:2021: Road vehicles — Test method to evaluate the performance of aut +- [iso-22733-1-2022](https://datatracker.ietf.org/doc/iso-22733-1-2022/) (score 2.5) — ISO 22733-1:2022: Road vehicles — Test method to evaluate the performance of aut - [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment -**Top-rated in AI safety/alignment** (46 drafts): +**Top-rated in AI safety/alignment** (112 drafts): - [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra - [draft-aylward-daap-v2](https://datatracker.ietf.org/doc/draft-aylward-daap-v2/) (4.8) — Defines comprehensive protocol for AI agent accountability including authentication, monitoring, and - [draft-guy-bary-stamp-protocol](https://datatracker.ietf.org/doc/draft-guy-bary-stamp-protocol/) (4.6) — Defines STAMP protocol for cryptographic delegation and proof in AI agent systems. Provides task-bou - [draft-drake-email-tpm-attestation](https://datatracker.ietf.org/doc/draft-drake-email-tpm-attestation/) (4.6) — Defines hardware attestation for email using TPM verification chains to prevent spam and provide Syb -- [draft-goswami-agentic-jwt](https://datatracker.ietf.org/doc/draft-goswami-agentic-jwt/) (4.5) — Extends OAuth 2.0 with Agentic JWT to address authorization challenges in autonomous AI systems. Int +- [iso-iec-42001-2023](https://datatracker.ietf.org/doc/iso-iec-42001-2023/) (4.6) — ISO/IEC standard establishing comprehensive AI management system requirements covering governance, r ### Partially Addressing Ideas -9 extracted ideas touch on this gap: +15 extracted ideas touch on this gap: | Idea | Draft | Type | |------|-------|------| -| Distributed AI Accountability Protocol | draft-aylward-daap-v2 | protocol | -| Agentic network architecture for multi-agent coordination | draft-chuyi-nmrg-agentic-network-inference | architecture | -| Dynamic Task Coordination Requirements for AI Agents | draft-cui-ai-agent-task | requirement | -| Multi-Agent Communication Framework for AIOps | draft-fu-nmop-agent-communication-framework | architecture | -| Meta-Layer Coordination Substrate | draft-meta-layer-overview | architecture | -| Trinity Configuration for Agent Coordination | draft-takagi-srta-trinity | pattern | -| Internet of Agents Task Protocol for heterogeneous collaboration | draft-yang-dmsc-ioa-task-protocol | protocol | -| Task-Oriented Multi-Agent Recovery Framework | draft-yue-anima-agent-recovery-networks | architecture | +| LLM-Enhanced Autonomic Service Agent Architecture | draft-han-anima-ai-asa | architecture | +| Multipath Traffic Engineering Capabilities Advertisement | draft-kompella-lsr-mptecap | mechanism | +| Agent Collaboration Protocols Architecture | draft-liu-dmsc-acps-arc | architecture | +| Agent Lifecycle Support | draft-liu-dmsc-acps-arc | protocol | +| Zero Trust Interoperability Framework | draft-liu-saag-zt-problem-statement | requirement | +| Cross-device Communication Protocol Gap Analysis for Network AI Agents | draft-mao-rtgwg-agent-comm-protocol-gap-analysis | requirement | +| Comparative analysis of messaging protocols for agentic AI | draft-mpsb-agntcy-messaging | pattern | +| Cross-Device Communication Framework for Network AI Agents | draft-mzsg-rtgwg-agent-cross-device-comm-framework | architecture | -*...and 1 more* +*...and 7 more* --- -## 4. Cross-Protocol Agent Migration +## 4. Cross-Domain Agent Identity Portability | | | |---|---| | **Severity** | HIGH | -| **Category** | A2A protocols | -| **Drafts in category** | 150 | +| **Category** | agent identity/auth | +| **Drafts in category** | 160 | -While A2A protocols exist, there's no standardized mechanism for agents to migrate between different protocol frameworks or service providers while maintaining state and identity. This creates vendor lock-in and limits agent portability across heterogeneous systems. +Agents cannot maintain consistent identity across different organizational domains or standards bodies' protocols. IETF defines authentication within network boundaries while ISO focuses on domain-specific identity, creating fragmentation. -**Evidence:** 150 A2A protocol drafts with high overlap suggest fragmentation without migration solutions - -### Related Drafts - -**Keyword matches** (drafts mentioning gap topic): - -- [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (score 4.8) — Verifiable Operations Ledger and Trace (VOLT) Protocol -- [draft-han-ai-agent-impact-infra](https://datatracker.ietf.org/doc/draft-han-ai-agent-impact-infra/) (score 2.3) — The Impact of AI Agent to Network Infrastructure -- [draft-narajala-ans](https://datatracker.ietf.org/doc/draft-narajala-ans/) (score 4.2) — Agent Name Service (ANS): A Universal Directory for Secure AI Agent Discovery an -- [draft-ietf-emu-eap-edhoc](https://datatracker.ietf.org/doc/draft-ietf-emu-eap-edhoc/) (score 3.2) — Using the Extensible Authentication Protocol (EAP) with Ephemeral Diffie-Hellman -- [draft-howe-sipcore-mcp-extension](https://datatracker.ietf.org/doc/draft-howe-sipcore-mcp-extension/) (score 3.7) — SIP Extension for Model Context Protocol (MCP) -- [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment - -**Top-rated in A2A protocols** (150 drafts): - -- [draft-guy-bary-stamp-protocol](https://datatracker.ietf.org/doc/draft-guy-bary-stamp-protocol/) (4.6) — Defines STAMP protocol for cryptographic delegation and proof in AI agent systems. Provides task-bou -- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (4.6) — Defines YANG data model for hierarchical language model coordination across tiny, small, and large L -- [draft-ietf-lake-edhoc](https://datatracker.ietf.org/doc/draft-ietf-lake-edhoc/) (4.6) — Specifies EDHOC, a compact authenticated Diffie-Hellman key exchange protocol for constrained enviro -- [draft-chang-agent-token-efficient](https://datatracker.ietf.org/doc/draft-chang-agent-token-efficient/) (4.5) — Defines ADOL (Agentic Data Optimization Layer) to address token bloat in agent communication protoco -- [draft-chen-oauth-rar-agent-extensions](https://datatracker.ietf.org/doc/draft-chen-oauth-rar-agent-extensions/) (4.2) — Extends OAuth RAR with policy_context and lifecycle_binding members for AI agent environments. Enabl - -### Partially Addressing Ideas - -No directly related technical ideas found in current drafts — this gap is entirely unaddressed. - ---- - -## 5. Agent Resource Accounting and Billing - -| | | -|---|---| -| **Severity** | HIGH | -| **Category** | new | -| **Drafts in category** | 0 | - -No standardized protocols exist for tracking and billing computational resources consumed by autonomous agents across distributed systems. This is essential for commercial deployment but completely unaddressed. - -**Evidence:** High focus on protocols and deployment but zero drafts addressing economic models - -### Related Drafts - -**Keyword matches** (drafts mentioning gap topic): - -- [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment -- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite -- [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment -- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps -- [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks -- [draft-jia-oauth-scope-aggregation](https://datatracker.ietf.org/doc/draft-jia-oauth-scope-aggregation/) (score 3.5) — OAuth 2.0 Scope Aggregation for Multi-Step AI Agent Workflows - -### Partially Addressing Ideas - -10 extracted ideas touch on this gap: - -| Idea | Draft | Type | -|------|-------|------| -| SCIM 2.0 Extension for Agents and Agentic Applications | draft-abbey-scim-agent-extension | extension | -| Events Query Protocol | draft-gupta-httpapi-events-query | protocol | -| Micro Agent Communication Protocol (µACP) | draft-mallick-muacp | protocol | -| MOQT Binding for A2A and MCP Protocols | draft-nandakumar-ai-agent-moq-transport | extension | -| AI Agent Protocol Requirements | draft-rosenberg-ai-protocols | requirement | -| SCIM 2.0 Agent Extension | draft-scim-agent-extension | extension | -| Authorized Connection Policy Framework | draft-steckbeck-ua-conn-sec | mechanism | -| Agent Workflow Protocol Well-Known Resource | draft-vinaysingh-awp-wellknown | extension | - -*...and 2 more* - ---- - -## 6. Agent Capability Advertisement Verification - -| | | -|---|---| -| **Severity** | HIGH | -| **Category** | Agent discovery/reg | -| **Drafts in category** | 87 | - -While agent discovery protocols exist, there's no way to cryptographically verify that advertised agent capabilities are accurate. Agents could falsely claim capabilities leading to system failures. - -**Evidence:** 87 discovery drafts but no mention of capability verification mechanisms - -### Related Drafts - -**Keyword matches** (drafts mentioning gap topic): - -- [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment -- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite -- [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment -- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps -- [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks -- [draft-li-dmsc-inf-architecture](https://datatracker.ietf.org/doc/draft-li-dmsc-inf-architecture/) (score 3.1) — Dynamic Multi-agent Secured Collaboration Infrastructure Architecture - -**Top-rated in Agent discovery/reg** (87 drafts): - -- [draft-narajala-ans](https://datatracker.ietf.org/doc/draft-narajala-ans/) (4.2) — Introduces Agent Name Service (ANS) as a DNS-based universal directory for AI agent discovery and ve -- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (4.2) — Specifies a comprehensive multi-agent collaboration protocol suite using Agent Gateways for registra -- [draft-cui-dns-native-agent-naming-resolution](https://datatracker.ietf.org/doc/draft-cui-dns-native-agent-naming-resolution/) (4.1) — Specifies DNS-native naming and resolution for AI agents using FQDNs and SVCB records. Emphasizes DN -- [draft-nederveld-adl](https://datatracker.ietf.org/doc/draft-nederveld-adl/) (4.1) — Defines ADL, a JSON-based standard for describing AI agents including their capabilities, tools, per -- [draft-rosenberg-ai-protocols](https://datatracker.ietf.org/doc/draft-rosenberg-ai-protocols/) (4.1) — Establishes framework for AI agent communications on the Internet, surveying existing protocols like - -### Partially Addressing Ideas - -27 extracted ideas touch on this gap: - -| Idea | Draft | Type | -|------|-------|------| -| DNS-based AI Agent Discovery | draft-mozleywilliams-dnsop-bandaid | mechanism | -| DNS namespace for AI agent discovery | draft-mozleywilliams-dnsop-dnsaid | mechanism | -| Agent Registration and Discovery Protocol | draft-pioli-agent-discovery | protocol | -| Intent-based Agent Interconnection Protocol | draft-sun-zhang-iaip | protocol | -| Capability Advertisement and Intent Resolution | draft-sz-dmsc-iaip | mechanism | -| Intelligent Agent Communication Gateway Architecture | draft-agent-gw | architecture | -| AI-Native Network Protocol (AINP) | draft-ainp-protocol | protocol | -| Distributed AI Accountability Protocol | draft-aylward-daap-v2 | protocol | - -*...and 19 more* - ---- - -## 7. Cross-Domain Agent Communication Security - -| | | -|---|---| -| **Severity** | HIGH | -| **Category** | Agent identity/auth | -| **Drafts in category** | 145 | - -Current identity/auth solutions don't address secure communication between agents operating in different security domains or trust boundaries. Critical for enterprise and government deployments. - -**Evidence:** 145 identity drafts show awareness but cross-domain scenarios appear unaddressed +**Evidence:** Enterprise deployments require agents to work across cloud providers, on-premises systems, and partner networks with different identity systems ### Related Drafts @@ -311,7 +190,7 @@ Current identity/auth solutions don't address secure communication between agent - [draft-intellinode-ai-semantic-contract](https://datatracker.ietf.org/doc/draft-intellinode-ai-semantic-contract/) (score 3.2) — Semantic-Driven Traffic Shaping Contract for AI Networks - [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment -**Top-rated in Agent identity/auth** (145 drafts): +**Top-rated in agent identity/auth** (160 drafts): - [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra - [draft-aylward-daap-v2](https://datatracker.ietf.org/doc/draft-aylward-daap-v2/) (4.8) — Defines comprehensive protocol for AI agent accountability including authentication, monitoring, and @@ -321,90 +200,103 @@ Current identity/auth solutions don't address secure communication between agent ### Partially Addressing Ideas -54 extracted ideas touch on this gap: +33 extracted ideas touch on this gap: | Idea | Draft | Type | |------|-------|------| -| Agent Gateway Intercommunication Framework | draft-han-rtgwg-agent-gateway-intercomm-framework | architecture | -| Agent Gateway Requirements | draft-liu-rtgwg-agent-gateway-requirements | requirement | -| AI Agent Security Requirements Framework | draft-ni-a2a-ai-agent-security-requirements | requirement | -| Centralized Gateway for Multi-Agent Communication | draft-song-dmsc-problem-statement | architecture | -| Multi-Tenant Policy Enforcement Infrastructure | draft-song-dmsc-problem-statement | architecture | -| Intelligent Agent Communication Gateway Architecture | draft-agent-gw | architecture | -| AI-Native Network Protocol (AINP) | draft-ainp-protocol | protocol | -| Agent-to-Agent Communication in Transportation Networks | draft-an-nmrg-i2icf-cits | pattern | +| Cross-Domain Agent Interoperability Framework | draft-cui-dmsc-agent-cdi | architecture | +| Cross-Domain Authorization Information Sharing for Multi-Agent Systems | draft-diaconu-agents-authz-info-sharing | mechanism | +| Agent Authorization Profile for OAuth 2.0 | draft-aap-oauth-profile | extension | +| SCIM 2.0 Extension for Agents and Agentic Applications | draft-abbey-scim-agent-extension | extension | +| Distributed AI Accountability Protocol | draft-aylward-daap-v2 | protocol | +| Intent-Based Just-in-Time Authorization | draft-chen-agent-decoupled-authorization-model | architecture | +| Dynamic Behavior-Based Authentication and Authorization Requirements | draft-chen-ai-agent-auth-new-requirements | requirement | +| Agentic network architecture for multi-agent coordination | draft-chuyi-nmrg-agentic-network-inference | architecture | -*...and 46 more* +*...and 25 more* --- -## 8. Agent Performance Degradation Detection +## 5. Real-Time Agent Behavior Explanation | | | |---|---| | **Severity** | HIGH | -| **Category** | new | -| **Drafts in category** | 0 | +| **Category** | human-agent interaction | +| **Drafts in category** | 57 | -No standardized protocols exist for detecting when AI agents are experiencing model drift, adversarial attacks, or performance degradation. Essential for maintaining autonomous system reliability. +No standard defines how autonomous agents should provide real-time explanations of their decision-making to humans during operation. Current explainable AI frameworks are post-hoc rather than interactive. -**Evidence:** ML traffic management exists but not agent health monitoring standards +**Evidence:** Regulatory requirements emerging in EU AI Act and similar legislation demand real-time explainability for high-risk AI systems ### Related Drafts **Keyword matches** (drafts mentioning gap topic): +- [draft-an-nmrg-i2icf-cits](https://datatracker.ietf.org/doc/draft-an-nmrg-i2icf-cits/) (score 3.7) — Interface to In-Network Computing Functions for Cooperative Intelligent Transpor +- [draft-zhao-detnet-enhanced-use-cases](https://datatracker.ietf.org/doc/draft-zhao-detnet-enhanced-use-cases/) (score 3.2) — Enhanced Use Cases for Scaling Deterministic Networks +- [draft-zhang-rvp-problem-statement](https://datatracker.ietf.org/doc/draft-zhang-rvp-problem-statement/) (score 3.5) — Problem Statements and Requirements of Real-Virtual Agent Protocol (RVP): Commun +- [draft-yuan-rtgwg-traffic-agent-usecase](https://datatracker.ietf.org/doc/draft-yuan-rtgwg-traffic-agent-usecase/) (score 3.7) — Use cases of the AI Network Traffic Optimization Agent +- [draft-altanai-aipref-realtime-protocol-bindings](https://datatracker.ietf.org/doc/draft-altanai-aipref-realtime-protocol-bindings/) (score 3.6) — AI Preferences for Real-Time Protocol Bindings - [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment -- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite -- [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment -- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps -- [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks -- [draft-xiong-rtgwg-use-cases-hp-wan](https://datatracker.ietf.org/doc/draft-xiong-rtgwg-use-cases-hp-wan/) (score 2.6) — Use Cases for High-performance Wide Area Network + +**Top-rated in human-agent interaction** (57 drafts): + +- [draft-drake-email-tpm-attestation](https://datatracker.ietf.org/doc/draft-drake-email-tpm-attestation/) (4.6) — Defines hardware attestation for email using TPM verification chains to prevent spam and provide Syb +- [iso-37181-2022](https://datatracker.ietf.org/doc/iso-37181-2022/) (4.5) — Establishes guidelines for introducing and organizing autonomous vehicles on public roads. Addresses +- [iso-pas-8800-2024](https://datatracker.ietf.org/doc/iso-pas-8800-2024/) (4.5) — Addresses safety-related E/E systems using AI technology in series-production road vehicles, coverin +- [draft-ietf-aipref-vocab](https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/) (4.4) — Defines a standardized vocabulary for expressing preferences about how digital assets should be used +- [iso-iec-ts-6254-2025](https://datatracker.ietf.org/doc/iso-iec-ts-6254-2025/) (4.4) — Provides approaches for achieving explainability and interpretability of ML and AI systems across li ### Partially Addressing Ideas -5 extracted ideas touch on this gap: +13 extracted ideas touch on this gap: | Idea | Draft | Type | |------|-------|------| -| Virtual In-Cloud Router as IPv6 Enhancement Agent | draft-he-yi-srv6ops-ipv6-enhancemnet-in-cloud-uc | architecture | -| 6G Agent Protocol Requirements and Enabling Technologies | draft-hw-ai-agent-6g | requirement | -| Comparative analysis of messaging protocols for agentic AI | draft-mpsb-agntcy-messaging | pattern | | AI Network Security Agent | draft-yuan-rtgwg-security-agent-usecase | architecture | -| Task-Oriented Multi-Agent Recovery Framework | draft-yue-anima-agent-recovery-networks | architecture | +| A2A Protocol Transport over MOQT | draft-a2a-moqt-transport | protocol | +| Distributed AI Accountability Protocol | draft-aylward-daap-v2 | protocol | +| Post-Discovery Authorization Handshake | draft-barney-caam | protocol | +| Evidence-based Autonomy Maturity Model | draft-berlinai-vera | mechanism | +| Intent-Based Just-in-Time Authorization | draft-chen-agent-decoupled-authorization-model | architecture | +| Dynamic Behavior-Based Authentication and Authorization Requirements | draft-chen-ai-agent-auth-new-requirements | requirement | +| Dynamic Task Coordination Requirements for AI Agents | draft-cui-ai-agent-task | requirement | + +*...and 5 more* --- -## 9. Legal Liability Attribution Protocols +## 6. Multi-Agent Conflict Resolution | | | |---|---| | **Severity** | HIGH | -| **Category** | Policy/governance | -| **Drafts in category** | 115 | +| **Category** | A2A protocols | +| **Drafts in category** | 157 | -Missing technical protocols for creating audit trails that can determine legal liability when autonomous agents cause harm. Governance drafts exist but not technical accountability mechanisms. +No protocol exists for resolving conflicts when multiple autonomous agents have competing objectives or try to access the same resources simultaneously. Current A2A protocols assume cooperative scenarios. -**Evidence:** 115 governance drafts but legal technology gap for liability attribution +**Evidence:** Multi-agent systems in production environments frequently deadlock or exhibit emergent adversarial behavior ### Related Drafts **Keyword matches** (drafts mentioning gap topic): -- [draft-madhavan-aipref-displaybasedpref](https://datatracker.ietf.org/doc/draft-madhavan-aipref-displaybasedpref/) (score 2.5) — A Vocabulary for Controlling Usage of Content Collected by Search and AI Crawler -- [draft-farzdusa-aipref-enduser](https://datatracker.ietf.org/doc/draft-farzdusa-aipref-enduser/) (score 3.8) — AI Preferences Signaling: End User Impact -- [draft-kotecha-agentic-dispute-protocol](https://datatracker.ietf.org/doc/draft-kotecha-agentic-dispute-protocol/) (score 3.6) — Agentic Dispute Protocol -- [draft-cui-dmsc-agent-cdi](https://datatracker.ietf.org/doc/draft-cui-dmsc-agent-cdi/) (score 3.0) — Cross-Domain Interoperability Framework for AI Agent Collaboration -- [draft-ietf-aipref-vocab](https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/) (score 4.4) — A Vocabulary For Expressing AI Usage Preferences -- [draft-aylward-aiga-1](https://datatracker.ietf.org/doc/draft-aylward-aiga-1/) (score 4.2) — AI Governance and Accountability Protocol (AIGA) +- [draft-li-dmsc-mcps-agw](https://datatracker.ietf.org/doc/draft-li-dmsc-mcps-agw/) (score 3.5) — Multi-agent Collaboration Protocol Suite based on Agent Gateway +- [draft-li-dmsc-inf-architecture](https://datatracker.ietf.org/doc/draft-li-dmsc-inf-architecture/) (score 3.1) — Dynamic Multi-agent Secured Collaboration Infrastructure Architecture +- [draft-yue-anima-agent-recovery-networks](https://datatracker.ietf.org/doc/draft-yue-anima-agent-recovery-networks/) (score 4.1) — Task-Oriented Multi-Agent Recovery Framework for High-Reliability in Converged M +- [draft-chang-agent-context-interaction](https://datatracker.ietf.org/doc/draft-chang-agent-context-interaction/) (score 2.9) — Agent Context Interaction Optimizations +- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps +- [draft-jurkovikj-httpapi-agentic-state](https://datatracker.ietf.org/doc/draft-jurkovikj-httpapi-agentic-state/) (score 3.9) — HTTP Profile for Synchronized Resource State (Agentic State Transfer) -**Top-rated in Policy/governance** (115 drafts): +**Top-rated in A2A protocols** (157 drafts): -- [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra -- [draft-aylward-daap-v2](https://datatracker.ietf.org/doc/draft-aylward-daap-v2/) (4.8) — Defines comprehensive protocol for AI agent accountability including authentication, monitoring, and -- [draft-goswami-agentic-jwt](https://datatracker.ietf.org/doc/draft-goswami-agentic-jwt/) (4.5) — Extends OAuth 2.0 with Agentic JWT to address authorization challenges in autonomous AI systems. Int -- [draft-wang-cats-odsi](https://datatracker.ietf.org/doc/draft-wang-cats-odsi/) (4.5) — Specifies framework for decentralized LLM inference across untrusted participants with layer-aware e -- [draft-birkholz-verifiable-agent-conversations](https://datatracker.ietf.org/doc/draft-birkholz-verifiable-agent-conversations/) (4.5) — Defines CDDL-based data format for verifiable agent conversation records using COSE signing. Support +- [draft-guy-bary-stamp-protocol](https://datatracker.ietf.org/doc/draft-guy-bary-stamp-protocol/) (4.6) — Defines STAMP protocol for cryptographic delegation and proof in AI agent systems. Provides task-bou +- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (4.6) — Defines YANG data model for hierarchical language model coordination across tiny, small, and large L +- [draft-ietf-lake-edhoc](https://datatracker.ietf.org/doc/draft-ietf-lake-edhoc/) (4.6) — Specifies EDHOC, a compact authenticated Diffie-Hellman key exchange protocol for constrained enviro +- [draft-chang-agent-token-efficient](https://datatracker.ietf.org/doc/draft-chang-agent-token-efficient/) (4.5) — Defines ADOL (Agentic Data Optimization Layer) to address token bloat in agent communication protoco +- [iso-23725-2024](https://datatracker.ietf.org/doc/iso-23725-2024/) (4.4) — Defines interoperability interfaces between fleet management and autonomous haulage systems in surfa ### Partially Addressing Ideas @@ -412,85 +304,223 @@ No directly related technical ideas found in current drafts — this gap is enti --- -## 10. Agent Memory and State Persistence Standards +## 7. Inter-Standards-Body Protocol Bridging | | | |---|---| -| **Severity** | MEDIUM | -| **Category** | Data formats/interop | -| **Drafts in category** | 165 | +| **Severity** | HIGH | +| **Category** | data formats/interop | +| **Drafts in category** | 214 | -No standardized formats or protocols exist for how agents should persist long-term memory, experience, and learned behaviors across system restarts or migrations. Each implementation creates proprietary solutions. +Protocols developed by different standards bodies (IETF, ITU-T, ISO) cannot interoperate, creating silos where agents using ITU-T frameworks cannot communicate with those following IETF protocols. -**Evidence:** 165 data format drafts focus on communication but not persistent state management +**Evidence:** Enterprise environments need single agents to work across telecom networks (ITU-T), internet protocols (IETF), and industrial systems (ISO) + +### Related Drafts + +**Keyword matches** (drafts mentioning gap topic): + +- [draft-cui-nmrg-auto-test](https://datatracker.ietf.org/doc/draft-cui-nmrg-auto-test/) (score 3.6) — Framework and Automation Levels for AI-Assisted Network Protocol Testing +- [draft-cosmos-protocol-specification](https://datatracker.ietf.org/doc/draft-cosmos-protocol-specification/) (score 3.3) — The Cosmos Protocol Specification (Trust-Native Semantic Protocol) +- [draft-tu-nmrg-blockchain-trusted-protocol](https://datatracker.ietf.org/doc/draft-tu-nmrg-blockchain-trusted-protocol/) (score 1.8) — A Blockchain Trusted Protocol for Intelligent Communication Network +- [draft-yang-ioa-protocol](https://datatracker.ietf.org/doc/draft-yang-ioa-protocol/) (score 3.6) — Internet of Agents Protocol (IoA Protocol) for Heterogeneous Agent Collaboration +- [draft-yang-dmsc-ioa-task-protocol](https://datatracker.ietf.org/doc/draft-yang-dmsc-ioa-task-protocol/) (score 3.0) — Internet of Agents Task Protocol (IoA Task Protocol) for Heterogeneous Agent Col +- [draft-song-dmsc-problem-statement](https://datatracker.ietf.org/doc/draft-song-dmsc-problem-statement/) (score 3.0) — Problem Statement and Requirements for Dynamic Multi-agent Secured Collaboration + +**Top-rated in data formats/interop** (214 drafts): + +- [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra +- [itu-t-y-3172](https://datatracker.ietf.org/doc/itu-t-y-3172/) (4.7) — Establishes comprehensive architectural framework for machine learning deployment in future networks +- [iso-iec-22989-2022](https://datatracker.ietf.org/doc/iso-iec-22989-2022/) (4.7) — ISO/IEC standard defining core AI concepts and establishing standardized terminology across the fiel +- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (4.6) — Defines YANG data model for hierarchical language model coordination across tiny, small, and large L +- [draft-ietf-lake-app-profiles](https://datatracker.ietf.org/doc/draft-ietf-lake-app-profiles/) (4.6) — Defines canonical CBOR representation for EDHOC application profiles and coordination mechanisms for + +### Partially Addressing Ideas + +No directly related technical ideas found in current drafts — this gap is entirely unaddressed. + +--- + +## 8. Agent Behavioral Audit Trails + +| | | +|---|---| +| **Severity** | HIGH | +| **Category** | policy/governance | +| **Drafts in category** | 214 | + +Missing standards for maintaining immutable logs of agent decisions and actions that can support forensic analysis and regulatory compliance. Current logging focuses on system events not decision rationale. + +**Evidence:** Financial and healthcare regulations require detailed audit trails, but AI systems cannot provide decision-level accountability ### Related Drafts **Keyword matches** (drafts mentioning gap topic): - [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment -- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite - [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment +- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite - [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps - [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks -- [draft-gaikwad-llm-benchmarking-terminology](https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-terminology/) (score 2.7) — Benchmarking Terminology for Large Language Model Serving +- [draft-kamimura-rats-behavioral-evidence](https://datatracker.ietf.org/doc/draft-kamimura-rats-behavioral-evidence/) (score 3.0) — On the Relationship Between Remote Attestation and Behavioral Evidence Recording -**Top-rated in Data formats/interop** (165 drafts): +**Top-rated in policy/governance** (214 drafts): - [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra -- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (4.6) — Defines YANG data model for hierarchical language model coordination across tiny, small, and large L -- [draft-ietf-lake-app-profiles](https://datatracker.ietf.org/doc/draft-ietf-lake-app-profiles/) (4.6) — Defines canonical CBOR representation for EDHOC application profiles and coordination mechanisms for -- [draft-chang-agent-token-efficient](https://datatracker.ietf.org/doc/draft-chang-agent-token-efficient/) (4.5) — Defines ADOL (Agentic Data Optimization Layer) to address token bloat in agent communication protoco -- [draft-birkholz-verifiable-agent-conversations](https://datatracker.ietf.org/doc/draft-birkholz-verifiable-agent-conversations/) (4.5) — Defines CDDL-based data format for verifiable agent conversation records using COSE signing. Support +- [draft-aylward-daap-v2](https://datatracker.ietf.org/doc/draft-aylward-daap-v2/) (4.8) — Defines comprehensive protocol for AI agent accountability including authentication, monitoring, and +- [itu-t-y-3172](https://datatracker.ietf.org/doc/itu-t-y-3172/) (4.7) — Establishes comprehensive architectural framework for machine learning deployment in future networks +- [iso-iec-22989-2022](https://datatracker.ietf.org/doc/iso-iec-22989-2022/) (4.7) — ISO/IEC standard defining core AI concepts and establishing standardized terminology across the fiel +- [iso-iec-42001-2023](https://datatracker.ietf.org/doc/iso-iec-42001-2023/) (4.6) — ISO/IEC standard establishing comprehensive AI management system requirements covering governance, r ### Partially Addressing Ideas -18 extracted ideas touch on this gap: +9 extracted ideas touch on this gap: | Idea | Draft | Type | |------|-------|------| | Compliance-oriented agent memory model | draft-gaikwad-aps-profile | pattern | -| Zero Trust Interoperability Framework | draft-liu-saag-zt-problem-statement | requirement | -| Intelligent Agent Communication Gateway Architecture | draft-agent-gw | architecture | -| Zero Trust Runtime Agent Architecture | draft-berlinai-vera | architecture | -| Agentic Hypercall Protocol | draft-campbell-agentic-http | pattern | +| Delegated Agent Authorization Protocol | draft-mishra-oauth-agent-grants | protocol | +| Distributed AI Accountability Protocol | draft-aylward-daap-v2 | protocol | +| Verifiable Agent Conversation Format | draft-birkholz-verifiable-agent-conversations | protocol | +| Intent-Based Just-in-Time Authorization | draft-chen-agent-decoupled-authorization-model | architecture | +| Dynamic Behavior-Based Authentication and Authorization Requirements | draft-chen-ai-agent-auth-new-requirements | requirement | | Agent Persistent State Profile | draft-gaikwad-aps-profile | architecture | -| Agentic AI for Autonomous Network Management | draft-hong-nmrg-agenticai-ps | requirement | -| LISP-based geospatial intelligence network | draft-ietf-lisp-nexagon | protocol | +| Agent Interaction & Delegation Protocol | draft-vandoulas-aidp | protocol | -*...and 10 more* +*...and 1 more* --- -## 11. Agent-to-Human Escalation Standards +## 9. Agent Resource Consumption Limits | | | |---|---| | **Severity** | MEDIUM | -| **Category** | Human-agent interaction | -| **Drafts in category** | 41 | +| **Category** | autonomous netops | +| **Drafts in category** | 124 | -While human-in-the-loop protocols exist, there's no standardized framework for when and how agents should escalate decisions to humans based on uncertainty, risk, or ethical considerations. +Missing standards for how agents should self-regulate computational, network, and energy resource usage to prevent runaway consumption. Current traffic management focuses on traditional workloads, not autonomous agent behavior patterns. -**Evidence:** Only 41 human-agent interaction drafts versus complex autonomous systems requiring escalation +**Evidence:** Early agent deployments show unpredictable resource usage spikes that can destabilize infrastructure ### Related Drafts **Keyword matches** (drafts mentioning gap topic): -- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (score 4.6) — Hierarchical Topology for Language Model Coordination -- [draft-ietf-websec-mime-sniff](https://datatracker.ietf.org/doc/draft-ietf-websec-mime-sniff/) (score 3.7) — Media Type Sniffing -- [draft-scrm-aiproto-usecases](https://datatracker.ietf.org/doc/draft-scrm-aiproto-usecases/) (score 4.1) — Agentic AI Use Cases -- [draft-zeng-opsawg-llm-netconf-gap](https://datatracker.ietf.org/doc/draft-zeng-opsawg-llm-netconf-gap/) (score 3.9) — Gap Analysis of Network Configuration Protocols in LLM-Driven Intent-Based Netwo -- [draft-jadoon-nmrg-agentic-ai-autonomous-networks](https://datatracker.ietf.org/doc/draft-jadoon-nmrg-agentic-ai-autonomous-networks/) (score 4.1) — Agentic AI Architectural Principles for Autonomous Computer Networks +- [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment +- [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment +- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite +- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps +- [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks +- [draft-jia-oauth-scope-aggregation](https://datatracker.ietf.org/doc/draft-jia-oauth-scope-aggregation/) (score 3.5) — OAuth 2.0 Scope Aggregation for Multi-Step AI Agent Workflows -**Top-rated in Human-agent interaction** (41 drafts): +**Top-rated in autonomous netops** (124 drafts): -- [draft-drake-email-tpm-attestation](https://datatracker.ietf.org/doc/draft-drake-email-tpm-attestation/) (4.6) — Defines hardware attestation for email using TPM verification chains to prevent spam and provide Syb -- [draft-ietf-aipref-vocab](https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/) (4.4) — Defines a standardized vocabulary for expressing preferences about how digital assets should be used -- [draft-dhir-http-agent-profile](https://datatracker.ietf.org/doc/draft-dhir-http-agent-profile/) (4.2) — Defines HTTP Agent Profile for authenticating agent traffic, separating human from agent traffic, an -- [draft-song-tsvwg-camp](https://datatracker.ietf.org/doc/draft-song-tsvwg-camp/) (4.2) — Proposes CAMP, a multipath transport protocol for interactive multimodal LLM systems that maintains -- [draft-liu-agent-operation-authorization](https://datatracker.ietf.org/doc/draft-liu-agent-operation-authorization/) (4.1) — Specifies framework for verifiable delegation of actions from humans to AI agents using JWT tokens. +- [itu-t-y-3172](https://datatracker.ietf.org/doc/itu-t-y-3172/) (4.7) — Establishes comprehensive architectural framework for machine learning deployment in future networks +- [iso-pas-8800-2024](https://datatracker.ietf.org/doc/iso-pas-8800-2024/) (4.5) — Addresses safety-related E/E systems using AI technology in series-production road vehicles, coverin +- [draft-cui-nmrg-llm-benchmark](https://datatracker.ietf.org/doc/draft-cui-nmrg-llm-benchmark/) (4.3) — Provides comprehensive evaluation framework for LLM-based network configuration agents. Includes emu +- [iso-22733-1-2021](https://datatracker.ietf.org/doc/iso-22733-1-2021/) (4.3) — Specifies test methodology for evaluating autonomous emergency braking system performance in car-to- +- [draft-wmz-nmrg-agent-ndt-arch](https://datatracker.ietf.org/doc/draft-wmz-nmrg-agent-ndt-arch/) (4.2) — Comprehensive architecture combining Network Digital Twin with Agentic AI for intent-based network o + +### Partially Addressing Ideas + +12 extracted ideas touch on this gap: + +| Idea | Draft | Type | +|------|-------|------| +| SCIM 2.0 Extension for Agents and Agentic Applications | draft-abbey-scim-agent-extension | extension | +| Context Distribution Optimization Framework | draft-chang-agent-context-interaction | mechanism | +| Events Query Protocol | draft-gupta-httpapi-events-query | protocol | +| Micro Agent Communication Protocol (µACP) | draft-mallick-muacp | protocol | +| MOQT Binding for A2A and MCP Protocols | draft-nandakumar-ai-agent-moq-transport | extension | +| AI Agent Protocol Requirements | draft-rosenberg-ai-protocols | requirement | +| SCIM 2.0 Agent Extension | draft-scim-agent-extension | extension | +| Authorized Connection Policy Framework | draft-steckbeck-ua-conn-sec | mechanism | + +*...and 4 more* + +--- + +## 10. Agent Training Data Provenance Tracking + +| | | +|---|---| +| **Severity** | MEDIUM | +| **Category** | data formats/interop | +| **Drafts in category** | 214 | + +Missing standards for tracking the lineage and provenance of training data as it flows between agents and gets updated through federated learning or agent interactions. + +**Evidence:** Data protection regulations require knowing data sources, but current AI systems cannot trace training data origins + +### Related Drafts + +**Keyword matches** (drafts mentioning gap topic): + +- [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment +- [draft-zheng-dispatch-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-dispatch-agent-identity-management/) (score 3.3) — Agent Identity Managenment +- [draft-li-dmsc-macp](https://datatracker.ietf.org/doc/draft-li-dmsc-macp/) (score 4.2) — Multi-agent Collaboration Protocol Suite +- [draft-fu-nmop-agent-communication-framework](https://datatracker.ietf.org/doc/draft-fu-nmop-agent-communication-framework/) (score 3.0) — Agent Communication Framework for Network AIOps +- [draft-zyyhl-agent-networks-framework](https://datatracker.ietf.org/doc/draft-zyyhl-agent-networks-framework/) (score 3.6) — Framework for AI Agent Networks +- [draft-akhavain-moussa-ai-network](https://datatracker.ietf.org/doc/draft-akhavain-moussa-ai-network/) (score 3.9) — AI Network for Training, Inference, and Agentic Interactions + +**Top-rated in data formats/interop** (214 drafts): + +- [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra +- [itu-t-y-3172](https://datatracker.ietf.org/doc/itu-t-y-3172/) (4.7) — Establishes comprehensive architectural framework for machine learning deployment in future networks +- [iso-iec-22989-2022](https://datatracker.ietf.org/doc/iso-iec-22989-2022/) (4.7) — ISO/IEC standard defining core AI concepts and establishing standardized terminology across the fiel +- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (4.6) — Defines YANG data model for hierarchical language model coordination across tiny, small, and large L +- [draft-ietf-lake-app-profiles](https://datatracker.ietf.org/doc/draft-ietf-lake-app-profiles/) (4.6) — Defines canonical CBOR representation for EDHOC application profiles and coordination mechanisms for + +### Partially Addressing Ideas + +30 extracted ideas touch on this gap: + +| Idea | Draft | Type | +|------|-------|------| +| EAT Profile for AI Agent Attestation | draft-messous-eat-ai | extension | +| Warrant Certificate Authority (WCA) | draft-bondar-wca | architecture | +| Blockchain-Anchored Integrity for AI Provenance | draft-reilly-sentinel-protocol | mechanism | +| AI Traffic Characterization Framework | draft-aft-ai-traffic | requirement | +| AI Traffic Characterization Framework | draft-ai-traffic | requirement | +| Network Architecture for AI Training and Inference | draft-akhavain-moussa-ai-network | architecture | +| Verifiable Agent Conversation Format | draft-birkholz-verifiable-agent-conversations | protocol | +| Agentic Data Optimization Layer (ADOL) | draft-chang-agent-token-efficient | protocol | + +*...and 22 more* + +--- + +## 11. Agent-Generated Content Attribution + +| | | +|---|---| +| **Severity** | MEDIUM | +| **Category** | data formats/interop | +| **Drafts in category** | 214 | + +Missing technical standards for embedding cryptographic attribution in content created by agents, enabling detection of AI-generated text, code, or decisions. Current synthetic content guidance lacks implementation details. + +**Evidence:** Need to distinguish agent-generated content from human-generated for legal, security, and quality assurance purposes + +### Related Drafts + +**Keyword matches** (drafts mentioning gap topic): + +- [draft-romanchuk-normative-admissibility](https://datatracker.ietf.org/doc/draft-romanchuk-normative-admissibility/) (score 3.4) — Normative Admissibility Framework for Agent Speech Acts +- [draft-li-semantic-routing-architecture](https://datatracker.ietf.org/doc/draft-li-semantic-routing-architecture/) (score 3.6) — Semantic Routing Architecture for AI Agents Communication +- [draft-cui-nmrg-llm-nm](https://datatracker.ietf.org/doc/draft-cui-nmrg-llm-nm/) (score 4.1) — A Framework for LLM Agent-Assisted Network Management with Human-in-the-Loop +- [draft-mpsb-agntcy-messaging](https://datatracker.ietf.org/doc/draft-mpsb-agntcy-messaging/) (score 2.6) — An Overview of Messaging Systems and Their Applicability to Agentic AI +- [draft-bondar-wca](https://datatracker.ietf.org/doc/draft-bondar-wca/) (score 3.9) — Warrant Certificate Authorities (WCA): Auditable Data Provenance for AI-Agent To +- [draft-it-aipref-attachment](https://datatracker.ietf.org/doc/draft-it-aipref-attachment/) (score 3.2) — Indicating Preferences Regarding Content Usage + +**Top-rated in data formats/interop** (214 drafts): + +- [draft-cowles-volt](https://datatracker.ietf.org/doc/draft-cowles-volt/) (4.8) — Defines tamper-evident execution trace format for AI agent workflows using hash chains and cryptogra +- [itu-t-y-3172](https://datatracker.ietf.org/doc/itu-t-y-3172/) (4.7) — Establishes comprehensive architectural framework for machine learning deployment in future networks +- [iso-iec-22989-2022](https://datatracker.ietf.org/doc/iso-iec-22989-2022/) (4.7) — ISO/IEC standard defining core AI concepts and establishing standardized terminology across the fiel +- [draft-williams-netmod-lm-hierarchy-topology](https://datatracker.ietf.org/doc/draft-williams-netmod-lm-hierarchy-topology/) (4.6) — Defines YANG data model for hierarchical language model coordination across tiny, small, and large L +- [draft-ietf-lake-app-profiles](https://datatracker.ietf.org/doc/draft-ietf-lake-app-profiles/) (4.6) — Defines canonical CBOR representation for EDHOC application profiles and coordination mechanisms for ### Partially Addressing Ideas @@ -498,57 +528,19 @@ No directly related technical ideas found in current drafts — this gap is enti --- -## 12. Federated Agent Learning Privacy - -| | | -|---|---| -| **Severity** | MEDIUM | -| **Category** | new | -| **Drafts in category** | 0 | - -Federated AI operations models exist but lack privacy-preserving protocols for agents learning from shared experiences without exposing sensitive data from individual deployments. - -**Evidence:** Federated models mentioned but privacy-preserving learning protocols absent - -### Related Drafts - -**Keyword matches** (drafts mentioning gap topic): - -- [draft-kale-agntcy-federated-privacy](https://datatracker.ietf.org/doc/draft-kale-agntcy-federated-privacy/) (score 3.2) — Privacy-Preserving Federated Learning Architecture for Multi-Tenant AI Agent Sys -- [draft-cui-dmsc-agent-cdi](https://datatracker.ietf.org/doc/draft-cui-dmsc-agent-cdi/) (score 3.0) — Cross-Domain Interoperability Framework for AI Agent Collaboration -- [draft-ai-traffic](https://datatracker.ietf.org/doc/draft-ai-traffic/) (score 2.5) — Handling inter-DC/Edge AI-related network traffic: Problem statement -- [draft-aft-ai-traffic](https://datatracker.ietf.org/doc/draft-aft-ai-traffic/) (score 3.1) — Handling inter-DC/Edge AI-related network traffic: Problem statement -- [draft-aylward-aiga-1](https://datatracker.ietf.org/doc/draft-aylward-aiga-1/) (score 4.2) — AI Governance and Accountability Protocol (AIGA) -- [draft-zheng-agent-identity-management](https://datatracker.ietf.org/doc/draft-zheng-agent-identity-management/) (score 3.7) — Agent Identity Managenment - -### Partially Addressing Ideas - -5 extracted ideas touch on this gap: - -| Idea | Draft | Type | -|------|-------|------| -| Privacy-Preserving Federated Learning for Multi-Tenant AI Agents | draft-kale-agntcy-federated-privacy | architecture | -| Cross-Domain Agent Interoperability Framework | draft-cui-dmsc-agent-cdi | architecture | -| HTTP Agent Profile (HAP) | draft-dhir-http-agent-profile | protocol | -| AI Network Security Agent | draft-yuan-rtgwg-security-agent-usecase | architecture | -| AI Network Traffic Optimization Agent | draft-yuan-rtgwg-traffic-agent-usecase | architecture | - ---- - ## Cross-Cutting Analysis ### Gaps by Category | Category | Drafts | Gaps | Gap Topics | |----------|-------:|-----:|------------| -| a2a protocols | 150 | 2 | Multi-Agent Consensus Under Byzantine Conditions; Cross-Protocol Agent Migration | -| agent discovery/reg | 87 | 1 | Agent Capability Advertisement Verification | -| agent identity/auth | 145 | 1 | Cross-Domain Agent Communication Security | -| ai safety/alignment | 46 | 2 | Real-time Agent Behavior Verification; Emergency Agent Shutdown Coordination | -| data formats/interop | 165 | 1 | Agent Memory and State Persistence Standards | -| human-agent interaction | 41 | 1 | Agent-to-Human Escalation Standards | -| new | 0 | 3 | Agent Resource Accounting and Billing; Agent Performance Degradation Detection; Federated Agent Learning Privacy | -| policy/governance | 115 | 1 | Legal Liability Attribution Protocols | +| a2a protocols | 157 | 1 | Multi-Agent Conflict Resolution | +| agent identity/auth | 160 | 1 | Cross-Domain Agent Identity Portability | +| ai safety/alignment | 112 | 2 | Agent Capability Degradation Detection; Emergency Agent Override Protocols | +| autonomous netops | 124 | 1 | Agent Resource Consumption Limits | +| data formats/interop | 214 | 3 | Inter-Standards-Body Protocol Bridging; Agent Training Data Provenance Tracking; Agent-Generated Content Attribution | +| human-agent interaction | 57 | 1 | Real-Time Agent Behavior Explanation | +| policy/governance | 214 | 2 | Agent Legal Liability Framework; Agent Behavioral Audit Trails | ## Recommendations diff --git a/scripts/fetch-all-sources.py b/scripts/fetch-all-sources.py index dacd298..9e42609 100644 --- a/scripts/fetch-all-sources.py +++ b/scripts/fetch-all-sources.py @@ -15,7 +15,7 @@ cfg = Config.load() db = Database(cfg) # Only fetch from new sources (IETF and W3C already done recently) -sources_to_fetch = ["etsi", "itu", "iso"] +sources_to_fetch = ["etsi", "itu", "iso", "nist"] total_new = 0 for source_name in sources_to_fetch: diff --git a/src/ietf_analyzer/analyzer.py b/src/ietf_analyzer/analyzer.py index 9a20305..e74eca4 100644 --- a/src/ietf_analyzer/analyzer.py +++ b/src/ietf_analyzer/analyzer.py @@ -160,29 +160,34 @@ Return ONLY a JSON object like {{"draft-name":[...], ...}}, no fences.""" # independent gap analyses and intersect results, have domain experts validate. # ============================================================================ GAP_ANALYSIS_PROMPT = """\ -You are analyzing the landscape of {total} IETF Internet-Drafts related to AI agents and autonomous systems. +You are analyzing the landscape of {total} documents related to AI agents and autonomous systems from multiple standards organizations. -## Categories and Draft Counts +## IETF Drafts — Categories and Draft Counts {category_summary} -## Most Common Technical Ideas +## Most Common Technical Ideas (from IETF drafts) {top_ideas} ## Known Overlap Clusters (groups of highly similar drafts) {overlap_summary} -Identify 8-15 GAPS — areas, problems, or technical challenges NOT adequately addressed by existing drafts. +## Other Standards Bodies +{other_sources_summary} + +Identify 8-15 GAPS — areas, problems, or technical challenges NOT adequately addressed by existing drafts and standards. Return a JSON array: -[{{"topic":"short topic name","description":"2-3 sentence description","category":"closest category or new","severity":"critical|high|medium|low","evidence":"what suggests this gap matters"}}] +[{{"topic":"short topic name","description":"2-3 sentence description","category":"closest category or new","severity":"critical|high|medium|low","evidence":"what suggests this gap matters","addressed_by":"which existing standards (if any) partially address this, from any source"}}] Focus on: -1. Problems mentioned but not solved -2. Missing infrastructure pieces +1. Problems mentioned but not solved — even across organizations +2. Missing infrastructure pieces (no standard from ANY body covers it) 3. Security/privacy/safety issues not addressed -4. Interoperability gaps between competing proposals +4. Interoperability gaps between competing proposals or between standards bodies 5. Real-world deployment concerns ignored +6. Cross-organization coordination gaps (e.g., IETF protocol needs ISO governance framework) +Consider what NIST, ISO, ETSI, ITU-T, and W3C already cover vs what remains missing. JSON array only, no fences.""" SCORE_NOVELTY_PROMPT = """\ @@ -638,11 +643,31 @@ class Analyzer: for c, n in sorted(cat_counts.items(), key=lambda x: x[1], reverse=True)[:5]: overlap_summary += f"- {c} ({n} drafts, high internal overlap)\n" + # Build summary of non-IETF sources + other_rows = self.db.conn.execute( + "SELECT source, name, title, abstract FROM drafts WHERE source != 'ietf' ORDER BY source, name" + ).fetchall() + source_groups: dict[str, list] = {} + for r in other_rows: + src = r["source"].upper() + source_groups.setdefault(src, []).append(r) + + other_lines = [] + for src, docs in sorted(source_groups.items()): + other_lines.append(f"\n### {src} ({len(docs)} documents)") + for d in docs[:30]: # cap per source to fit context + abstract = (d["abstract"] or "")[:150] + other_lines.append(f"- **{d['title'][:100]}**: {abstract}") + if len(docs) > 30: + other_lines.append(f" ... and {len(docs) - 30} more") + other_sources_summary = "\n".join(other_lines) if other_lines else "(No other sources available)" + prompt = GAP_ANALYSIS_PROMPT.format( total=total, category_summary=category_summary, top_ideas=top_ideas, overlap_summary=overlap_summary, + other_sources_summary=other_sources_summary, ) phash = _prompt_hash(prompt) diff --git a/src/ietf_analyzer/cli.py b/src/ietf_analyzer/cli.py index 986ee76..98633b1 100644 --- a/src/ietf_analyzer/cli.py +++ b/src/ietf_analyzer/cli.py @@ -3081,3 +3081,255 @@ def export(export_type: str, fmt: str, output_file: str | None): finally: db.close() + + +# ── auto ───────────────────────────────────────────────────────────────────── + + +@main.command("auto") +@click.option("--cost-limit", default=2.0, help="Auto-approve operations under this USD amount (default: $2)") +@click.option("--yes", "-y", is_flag=True, help="Skip all confirmation prompts") +@click.option("--dry-run", is_flag=True, help="Show what would be done without doing it") +@click.option("--source", "-s", default=None, help="Limit to specific source (ietf,w3c,etsi,iso,itu)") +def auto(cost_limit: float, yes: bool, dry_run: bool, source: str | None): + """Auto-heal: fetch, analyze, embed, extract ideas, and update gaps. + + Automatically processes all unrated, unembedded, and idea-less drafts + across all sources. Uses cheap models (Haiku) for bulk operations. + Operations estimated above --cost-limit require confirmation. + + Examples: + + ietf auto # run full pipeline, auto-approve under $2 + + ietf auto --dry-run # show plan without executing + + ietf auto -s iso # only process ISO drafts + + ietf auto --cost-limit 5 # raise approval threshold to $5 + + ietf auto -y # skip all prompts (for cron) + """ + cfg = Config.load() + db = Database(cfg) + + try: + _auto_heal(cfg, db, cost_limit=cost_limit, yes=yes, dry_run=dry_run, source_filter=source) + finally: + db.close() + + +def _estimate_cost(n_drafts: int, operation: str) -> float: + """Estimate USD cost for an operation. Conservative estimates.""" + # Haiku: ~$0.25/M input, ~$1.25/M output + # Sonnet: ~$3/M input, ~$15/M output + # Average draft abstract: ~500 tokens input, ~200 tokens output + costs = { + "analyze_cheap": n_drafts * 0.0005, # ~$0.50 per 1000 drafts (Haiku) + "analyze_quality": n_drafts * 0.005, # ~$5.00 per 1000 drafts (Sonnet) + "ideas_cheap": n_drafts * 0.001, # ~$1.00 per 1000 drafts (Haiku batch) + "ideas_quality": n_drafts * 0.008, # ~$8.00 per 1000 drafts (Sonnet) + "gaps": 0.05, # single Claude call + "embed": 0.0, # Ollama is free/local + "authors": 0.0, # Datatracker API is free + "fetch": 0.0, # Datatracker API is free + } + return costs.get(operation, 0.0) + + +def _auto_heal(cfg, db, cost_limit: float, yes: bool, dry_run: bool, source_filter: str | None): + """Run the full auto-heal pipeline.""" + import time as _time + + from rich.panel import Panel + + steps: list[dict] = [] + total_cost = 0.0 + + # ── Step 1: Fetch new drafts from all sources ── + sources = [source_filter] if source_filter else cfg.observatory_sources + steps.append({ + "name": f"Fetch new drafts from {', '.join(sources)}", + "sources": sources, + "cost": 0.0, + "action": "fetch", + }) + + # ── Step 2: Analyze unrated drafts ── + unrated = db.unrated_drafts(limit=10000) + if source_filter: + unrated = [d for d in unrated if (d.source or "ietf") == source_filter] + n_unrated = len(unrated) + analyze_cost = _estimate_cost(n_unrated, "analyze_cheap") + steps.append({ + "name": f"Analyze {n_unrated} unrated drafts (Haiku)", + "count": n_unrated, + "cost": analyze_cost, + "action": "analyze", + }) + total_cost += analyze_cost + + # ── Step 3: Fetch authors ── + missing_authors = db.conn.execute( + "SELECT COUNT(*) FROM drafts WHERE name NOT IN (SELECT DISTINCT draft_name FROM draft_authors)" + ).fetchone()[0] + steps.append({ + "name": f"Fetch authors for {missing_authors} drafts", + "count": missing_authors, + "cost": 0.0, + "action": "authors", + }) + + # ── Step 4: Embed missing drafts ── + missing_embed = db.drafts_without_embeddings(limit=10000) + if source_filter: + source_names = {row[0] for row in db.conn.execute( + "SELECT name FROM drafts WHERE source = ?", (source_filter,) + ).fetchall()} + missing_embed = [n for n in missing_embed if n in source_names] + n_embed = len(missing_embed) + steps.append({ + "name": f"Embed {n_embed} drafts (Ollama, free)", + "count": n_embed, + "cost": 0.0, + "action": "embed", + }) + + # ── Step 5: Extract ideas ── + missing_ideas = db.drafts_without_ideas(limit=10000) + if source_filter: + if not source_names: + source_names = {row[0] for row in db.conn.execute( + "SELECT name FROM drafts WHERE source = ?", (source_filter,) + ).fetchall()} + missing_ideas = [n for n in missing_ideas if n in source_names] + n_ideas = len(missing_ideas) + ideas_cost = _estimate_cost(n_ideas, "ideas_cheap") + steps.append({ + "name": f"Extract ideas from {n_ideas} drafts (Haiku)", + "count": n_ideas, + "cost": ideas_cost, + "action": "ideas", + }) + total_cost += ideas_cost + + # ── Step 6: Refresh gaps ── + gap_cost = _estimate_cost(0, "gaps") + steps.append({ + "name": "Refresh gap analysis", + "cost": gap_cost, + "action": "gaps", + }) + total_cost += gap_cost + + # ── Show plan ── + plan_lines = [] + for s in steps: + count = s.get("count", 1) + if count == 0: + plan_lines.append(f" [dim]SKIP[/] {s['name']}") + else: + cost_str = f" [yellow]~${s['cost']:.2f}[/]" if s["cost"] > 0 else "" + plan_lines.append(f" [green]RUN[/] {s['name']}{cost_str}") + + auto_approved = total_cost <= cost_limit + plan_lines.append(f"\n [bold]Estimated total cost: ${total_cost:.2f}[/]") + if auto_approved: + plan_lines.append(f" [green]Auto-approved (under ${cost_limit:.2f} limit)[/]") + else: + plan_lines.append(f" [yellow]Requires approval (over ${cost_limit:.2f} limit)[/]") + + console.print(Panel("\n".join(plan_lines), title="Auto-Heal Plan")) + + if dry_run: + console.print("[bold yellow]DRY RUN[/] — no changes made.") + return + + # ── Approval ── + if not auto_approved and not yes: + if not click.confirm(f"Estimated cost ${total_cost:.2f} exceeds ${cost_limit:.2f} limit. Proceed?"): + console.print("[yellow]Aborted.[/]") + return + + # ── Execute ── + start = _time.time() + + for step in steps: + action = step["action"] + count = step.get("count", 0) + + if action == "fetch": + console.print(f"\n[bold cyan]>>> Fetching from {step['sources']}...[/]") + from .sources import get_fetcher + from .observatory import _doc_to_draft + for src_name in step["sources"]: + try: + fetcher = get_fetcher(src_name, cfg) + before = db.count_drafts() + results = fetcher.search(keywords=cfg.search_keywords) + for doc in results: + db.upsert_draft(_doc_to_draft(doc)) + after = db.count_drafts() + new = after - before + console.print(f" [{src_name}] +{new} new drafts") + fetcher.close() + except Exception as e: + console.print(f" [{src_name}] [red]Error: {e}[/]") + + elif action == "analyze" and count > 0: + console.print(f"\n[bold cyan]>>> Analyzing {count} drafts (Haiku)...[/]") + from .analyzer import Analyzer + analyzer = Analyzer(cfg, db) + orig_model = cfg.claude_model + cfg.claude_model = cfg.claude_model_cheap + try: + done = analyzer.rate_all_unrated(limit=count) + console.print(f" Analyzed [bold green]{done}[/] drafts") + finally: + cfg.claude_model = orig_model + + elif action == "authors" and count > 0: + console.print(f"\n[bold cyan]>>> Fetching authors for {count} drafts...[/]") + from .authors import AuthorNetwork + author_net = AuthorNetwork(cfg, db) + done = author_net.fetch_all_authors() + console.print(f" Fetched authors for [bold green]{done}[/] drafts") + + elif action == "embed" and count > 0: + console.print(f"\n[bold cyan]>>> Embedding {count} drafts (Ollama)...[/]") + from .embeddings import Embedder + with Embedder(cfg, db) as embedder: + done = embedder.embed_all_missing() + console.print(f" Embedded [bold green]{done}[/] drafts") + + elif action == "ideas" and count > 0: + console.print(f"\n[bold cyan]>>> Extracting ideas from {count} drafts (Haiku)...[/]") + from .analyzer import Analyzer + analyzer = Analyzer(cfg, db) + done = analyzer.extract_all_ideas(limit=count, batch_size=5, cheap=True) + console.print(f" Extracted ideas from [bold green]{done}[/] drafts") + + elif action == "gaps": + console.print(f"\n[bold cyan]>>> Refreshing gap analysis...[/]") + from .analyzer import Analyzer + analyzer = Analyzer(cfg, db) + gaps = analyzer.gap_analysis() + if gaps: + console.print(f" Found [bold green]{len(gaps)}[/] gaps") + + elapsed = _time.time() - start + console.print(f"\n[bold green]Auto-heal complete![/] ({elapsed:.1f}s, ~${total_cost:.2f})") + + # Show final counts + total = db.count_drafts() + rated = db.conn.execute("SELECT COUNT(*) FROM ratings").fetchone()[0] + embedded = db.conn.execute("SELECT COUNT(*) FROM embeddings").fetchone()[0] + idea_count = db.conn.execute("SELECT COUNT(*) FROM ideas").fetchone()[0] + gap_count = db.conn.execute("SELECT COUNT(*) FROM gaps").fetchone()[0] + console.print(f" Drafts: {total} | Rated: {rated} | Embedded: {embedded} | Ideas: {idea_count} | Gaps: {gap_count}") + + by_source = db.conn.execute( + "SELECT source, COUNT(*) FROM drafts GROUP BY source ORDER BY COUNT(*) DESC" + ).fetchall() + source_str = " | ".join(f"{s}: {c}" for s, c in by_source) + console.print(f" Sources: {source_str}") diff --git a/src/ietf_analyzer/config.py b/src/ietf_analyzer/config.py index 5447f4a..03cfe60 100644 --- a/src/ietf_analyzer/config.py +++ b/src/ietf_analyzer/config.py @@ -52,7 +52,7 @@ class Config: # Observatory — add "w3c" to enable W3C spec tracking: # ietf observatory update --source w3c (one-off) # or set observatory_sources to ["ietf", "w3c"] in config.json - observatory_sources: list[str] = field(default_factory=lambda: ["ietf", "w3c", "etsi", "itu", "iso"]) + observatory_sources: list[str] = field(default_factory=lambda: ["ietf", "w3c", "etsi", "itu", "iso", "nist"]) dashboard_dir: str = str(DEFAULT_DATA_DIR.parent / "docs") w3c_groups: list[str] = field(default_factory=lambda: [ "webmachinelearning", "wot", "credentials", "did", "vc" diff --git a/src/ietf_analyzer/sources/__init__.py b/src/ietf_analyzer/sources/__init__.py index 81c6121..26aecbf 100644 --- a/src/ietf_analyzer/sources/__init__.py +++ b/src/ietf_analyzer/sources/__init__.py @@ -5,6 +5,7 @@ from .etsi import ETSIFetcher from .ietf import IETFFetcher from .iso import ISOFetcher from .itu import ITUFetcher +from .nist import NISTFetcher from .w3c import W3CFetcher FETCHERS = { @@ -13,6 +14,7 @@ FETCHERS = { "etsi": ETSIFetcher, "itu": ITUFetcher, "iso": ISOFetcher, + "nist": NISTFetcher, } diff --git a/src/ietf_analyzer/sources/iso.py b/src/ietf_analyzer/sources/iso.py index 6218c8e..513936f 100644 --- a/src/ietf_analyzer/sources/iso.py +++ b/src/ietf_analyzer/sources/iso.py @@ -151,10 +151,11 @@ class ISOFetcher: continue name = _iso_id_to_name(ref) + abstract = scope[:2000] if scope else f"ISO/IEC standard: {title}. Committee: {committee}." docs.append(SourceDocument( name=name, title=f"{ref}: {title}", - abstract=f"ISO/IEC standard: {title}. Committee: {committee}. Status: {status}.", + abstract=abstract, source="iso", source_id=ref, source_url=f"https://www.iso.org/standard/{ref.split(':')[0].replace('/', '%2F').replace(' ', '%20')}.html", diff --git a/src/webui/data.py b/src/webui/data.py index c0cf915..c764d16 100644 --- a/src/webui/data.py +++ b/src/webui/data.py @@ -892,11 +892,13 @@ def _compute_author_network_full(db: Database) -> AuthorNetwork: if len(component) >= 2: org_mix: dict[str, int] = Counter() + member_orgs: dict[str, str] = {} cluster_drafts: dict[str, str] = {} # name -> title for m in component: org = author_info.get(m, {}).get("org", "") if org: org_mix[org] += 1 + member_orgs[m] = org for dn in author_info.get(m, {}).get("drafts", []): if dn not in cluster_drafts: d = _all_drafts_map.get(dn) @@ -904,9 +906,10 @@ def _compute_author_network_full(db: Database) -> AuthorNetwork: clusters.append({ "id": len(clusters), "members": component, + "member_orgs": member_orgs, "org_mix": dict(org_mix.most_common()), "size": len(component), - "drafts": [{"name": n, "title": t} for n, t in list(cluster_drafts.items())[:15]], + "drafts": [{"name": n, "title": t} for n, t in list(cluster_drafts.items())], "draft_count": len(cluster_drafts), }) @@ -1062,11 +1065,78 @@ def _compute_idea_clusters(db: Database) -> dict: except Exception: pass + # --- Cross-cluster links --- + # Find pairs of clusters whose ideas are semantically related + # Use centroid similarity + best idea-pair links + links = [] + if len(clusters) >= 2: + # Build cluster centroids from normalized embeddings + cluster_centroids = {} + cluster_member_indices: dict[int, list[int]] = defaultdict(list) + for idx, iid in enumerate(idea_ids): + cid = iid_to_new.get(iid, int(labels[idx])) + cluster_member_indices[cid].append(idx) + + for cid, indices in cluster_member_indices.items(): + if indices: + centroid = matrix_norm[indices].mean(axis=0) + norm = np.linalg.norm(centroid) + if norm > 0: + cluster_centroids[cid] = centroid / norm + + # Compute pairwise centroid similarity for all cluster pairs + cids_sorted = sorted(cluster_centroids.keys()) + for ci_idx, ci in enumerate(cids_sorted): + for cj in cids_sorted[ci_idx + 1:]: + sim = float(np.dot(cluster_centroids[ci], cluster_centroids[cj])) + if sim < 0.45: + continue + + # Find the best idea pair across these two clusters + best_sim = 0.0 + best_pair = (None, None) + # Sample up to 20 ideas per cluster to keep it fast + ci_members = cluster_member_indices[ci][:20] + cj_members = cluster_member_indices[cj][:20] + for mi in ci_members: + for mj in cj_members: + pair_sim = float(np.dot(matrix_norm[mi], matrix_norm[mj])) + if pair_sim > best_sim: + best_sim = pair_sim + best_pair = (idea_ids[mi], idea_ids[mj]) + + if best_sim < 0.5: + continue + + # Get theme names + ci_theme = next((c["theme"] for c in clusters if c["id"] == ci), f"Cluster {ci}") + cj_theme = next((c["theme"] for c in clusters if c["id"] == cj), f"Cluster {cj}") + + idea_a = idea_map.get(best_pair[0], {}) + idea_b = idea_map.get(best_pair[1], {}) + + links.append({ + "source": ci, + "target": cj, + "source_theme": ci_theme, + "target_theme": cj_theme, + "similarity": round(sim, 3), + "best_pair_sim": round(best_sim, 3), + "idea_a": idea_a.get("title", ""), + "idea_a_draft": idea_a.get("draft_name", ""), + "idea_b": idea_b.get("title", ""), + "idea_b_draft": idea_b.get("draft_name", ""), + }) + + links.sort(key=lambda l: l["best_pair_sim"], reverse=True) + links = links[:50] # cap at top 50 links + total = len(idea_ids) clustered = sum(c["size"] for c in clusters) return { "clusters": clusters, "scatter": scatter, + "links": links, "stats": {"total": total, "clustered": clustered, "num_clusters": len(clusters)}, "empty": False, } diff --git a/src/webui/templates/authors.html b/src/webui/templates/authors.html index 5485b0e..31cc1de 100644 --- a/src/webui/templates/authors.html +++ b/src/webui/templates/authors.html @@ -116,34 +116,72 @@

Clusters are formed by connected-component analysis of the co-authorship graph: authors who share 2+ drafts are linked, and all authors reachable through such links form a cluster. This reveals research teams and institutional collaboration patterns — a cluster of 20 authors from 3 organizations means those groups actively co-author across org boundaries. Click a cluster to highlight it in the graph above.

{% for c in network.clusters[:12] %} -
-
+
+ +
Cluster #{{ c.id + 1 }} -
+
{{ c.size }} authors {{ c.draft_count }} drafts +
+ +
{% for org, count in c.org_mix.items() %} {{ org }} ({{ count }}) {% endfor %}
+ +
- {{ c.members[:5] | join(', ') }}{% if c.members | length > 5 %} +{{ c.members | length - 5 }} more{% endif %} + {{ c.members[:3] | join(', ') }}{% if c.members | length > 3 %} +{{ c.members | length - 3 }} more{% endif %}
+ + {% if c.drafts %}
- {% for d in c.drafts[:5] %} + {% for d in c.drafts[:3] %}
{{ d.title }}
{% endfor %} - {% if c.draft_count > 5 %} -
+{{ c.draft_count - 5 }} more drafts
+ {% if c.draft_count > 3 %} +
+{{ c.draft_count - 3 }} more drafts
{% endif %}
{% endif %} + + +
{% endfor %}
@@ -478,6 +516,20 @@ const network = {{ network | tojson }}; }); }); + // Toggle expand/collapse on cluster card chevron click + document.querySelectorAll('.cluster-card').forEach(card => { + card.addEventListener('click', function(e) { + // Don't toggle if clicking a link or the highlight header + if (e.target.closest('a') || e.target.closest('[onclick*="highlightCluster"]')) return; + const detail = card.querySelector('.cluster-detail'); + const chevron = card.querySelector('.cluster-chevron'); + if (detail) { + detail.classList.toggle('hidden'); + chevron.style.transform = detail.classList.contains('hidden') ? '' : 'rotate(180deg)'; + } + }); + }); + // Expose cluster highlighting globally window.highlightCluster = function(clusterId) { const cluster = (network.clusters || []).find(c => c.id === clusterId); diff --git a/src/webui/templates/comparison.html b/src/webui/templates/comparison.html index 6eda29a..35e8f92 100644 --- a/src/webui/templates/comparison.html +++ b/src/webui/templates/comparison.html @@ -71,7 +71,7 @@ {{ draft.title }}
{{ draft.name }}
-
{{ draft.abstract[:200] }}
+
{{ (draft.abstract | striptags)[:200] }}
{% if draft.rating %} @@ -79,9 +79,15 @@ {% for dim, label in [('novelty', 'Nov'), ('maturity', 'Mat'), ('relevance', 'Rel'), ('momentum', 'Mom'), ('overlap', 'Ovl')] %}
{{ label }}
+ {% if dim == 'overlap' %} +
+ {{ draft.rating[dim] }} +
+ {% else %}
{{ draft.rating[dim] }}
+ {% endif %}
{% endfor %}
diff --git a/src/webui/templates/draft_detail.html b/src/webui/templates/draft_detail.html index 2d70c5e..71173ba 100644 --- a/src/webui/templates/draft_detail.html +++ b/src/webui/templates/draft_detail.html @@ -91,7 +91,7 @@ Abstract -

{{ draft.abstract or "No abstract available." }}

+

{{ (draft.abstract | striptags) or "No abstract available." }}

@@ -120,10 +120,18 @@ {{ label }}
+ {% if dim == "overlap" %} + {{ val }}/5 + {% else %} {{ val }}/5 + {% endif %}
+ {% if dim == "overlap" %} +
+ {% else %}
+ {% endif %}
{% if draft.rating[dim + '_note'] %}

{{ draft.rating[dim + '_note'] }}

@@ -231,7 +239,11 @@ {% for dim, abbr in [("novelty","N"), ("maturity","M"), ("overlap","O"), ("momentum","Mo"), ("relevance","R")] %} {% set v = draft.rating[dim] %}
+ {% if dim == "overlap" %} +
{{ v }}
+ {% else %}
{{ v }}
+ {% endif %}
{{ abbr }}
{% endfor %} diff --git a/src/webui/templates/drafts.html b/src/webui/templates/drafts.html index 8a267cf..0eb4e0a 100644 --- a/src/webui/templates/drafts.html +++ b/src/webui/templates/drafts.html @@ -68,6 +68,11 @@ color: #c084fc; border: 1px solid rgba(168, 85, 247, 0.3); } + .source-nist { + background: rgba(6, 182, 212, 0.15); + color: #22d3ee; + border: 1px solid rgba(6, 182, 212, 0.3); + } .source-generated { background: rgba(148, 163, 184, 0.15); color: #94a3b8; @@ -180,6 +185,7 @@ + @@ -426,7 +432,7 @@
- {{ d.overlap }} diff --git a/src/webui/templates/idea_clusters.html b/src/webui/templates/idea_clusters.html index dce5609..2ab0ed6 100644 --- a/src/webui/templates/idea_clusters.html +++ b/src/webui/templates/idea_clusters.html @@ -63,6 +63,20 @@
+ + +

Cluster Details

@@ -180,6 +194,135 @@ if (data.empty) { }, CFG); } + // --- Cluster Relationship Network --- + const links = data.links || []; + if (links.length > 0) { + document.getElementById('networkSection').classList.remove('hidden'); + + // Build node set from clusters that have links + const linkedIds = new Set(); + links.forEach(l => { linkedIds.add(l.source); linkedIds.add(l.target); }); + + const nodes = data.clusters.filter(c => linkedIds.has(c.id)); + const nodeMap = {}; + nodes.forEach((n, i) => { nodeMap[n.id] = i; }); + + // Force-directed layout using Plotly scatter + annotations for edges + // Position nodes in a circle, then use link structure + const n = nodes.length; + const nodeX = nodes.map((_, i) => Math.cos(2 * Math.PI * i / n) * 4); + const nodeY = nodes.map((_, i) => Math.sin(2 * Math.PI * i / n) * 4); + + // Simple force-directed: pull linked nodes closer + for (let iter = 0; iter < 80; iter++) { + for (const link of links) { + const si = nodeMap[link.source]; + const ti = nodeMap[link.target]; + if (si === undefined || ti === undefined) continue; + const dx = nodeX[ti] - nodeX[si]; + const dy = nodeY[ti] - nodeY[si]; + const dist = Math.sqrt(dx*dx + dy*dy) || 1; + const force = (link.best_pair_sim - 0.5) * 0.15; + nodeX[si] += dx/dist * force; + nodeY[si] += dy/dist * force; + nodeX[ti] -= dx/dist * force; + nodeY[ti] -= dy/dist * force; + } + // Repulsion between all nodes + for (let i = 0; i < n; i++) { + for (let j = i+1; j < n; j++) { + const dx = nodeX[j] - nodeX[i]; + const dy = nodeY[j] - nodeY[i]; + const dist = Math.sqrt(dx*dx + dy*dy) || 0.1; + if (dist < 1.5) { + const repel = 0.3 / (dist * dist); + nodeX[i] -= dx/dist * repel; + nodeY[i] -= dy/dist * repel; + nodeX[j] += dx/dist * repel; + nodeY[j] += dy/dist * repel; + } + } + } + } + + // Edge traces (one per link for click handling) + const edgeTraces = links.map((link, li) => { + const si = nodeMap[link.source]; + const ti = nodeMap[link.target]; + if (si === undefined || ti === undefined) return null; + const width = 1 + (link.best_pair_sim - 0.5) * 8; + const opacity = 0.3 + (link.best_pair_sim - 0.5) * 1.2; + return { + x: [nodeX[si], nodeX[ti], null], + y: [nodeY[si], nodeY[ti], null], + mode: 'lines', + line: { width: width, color: `rgba(100,116,139,${opacity})` }, + hoverinfo: 'text', + text: `${link.source_theme} ↔ ${link.target_theme}
Similarity: ${(link.best_pair_sim * 100).toFixed(0)}%`, + customdata: [li, li, null], + showlegend: false, + }; + }).filter(Boolean); + + // Node trace + const nodeTrace = { + x: nodeX, y: nodeY, + mode: 'markers+text', + type: 'scatter', + marker: { + size: nodes.map(n => 12 + Math.sqrt(n.size) * 3), + color: nodes.map((_, i) => PALETTE[nodes[i].id % PALETTE.length]), + line: { width: 2, color: 'rgba(15,23,42,0.8)' }, + }, + text: nodes.map(n => n.theme.length > 25 ? n.theme.substring(0, 22) + '...' : n.theme), + textposition: 'top center', + textfont: { size: 10, color: '#cbd5e1' }, + hovertext: nodes.map(n => + `${n.theme}
${n.size} ideas, ${n.drafts.length} drafts` + ), + hoverinfo: 'text', + showlegend: false, + }; + + Plotly.newPlot('networkPlot', [...edgeTraces, nodeTrace], { + ...PLOTLY_LAYOUT, + xaxis: { visible: false, showgrid: false, zeroline: false }, + yaxis: { visible: false, showgrid: false, zeroline: false }, + hovermode: 'closest', + margin: { t: 10, r: 20, b: 10, l: 20 }, + }, CFG); + + // Click handler for edges — show link detail + document.getElementById('networkPlot').on('plotly_click', function(ev) { + const pt = ev.points[0]; + if (pt.data.customdata && pt.data.customdata[pt.pointNumber] !== null) { + const link = links[pt.data.customdata[pt.pointNumber]]; + if (!link) return; + const detail = document.getElementById('linkDetail'); + const simPct = (link.best_pair_sim * 100).toFixed(0); + document.getElementById('linkTitle').innerHTML = + `${link.source_theme}` + + ` ↔ ` + + `${link.target_theme}` + + ` ${simPct}% similar`; + document.getElementById('linkContent').innerHTML = ` +
+
+
${link.idea_a}
+ ${link.idea_a_draft} +
+
+
${link.idea_b}
+ ${link.idea_b_draft} +
+
+

These two ideas from different clusters have the strongest cross-cluster similarity.

+ `; + detail.classList.remove('hidden'); + } + }); + } + // --- Cluster Cards --- const grid = document.getElementById('clusterGrid'); @@ -190,15 +333,42 @@ if (data.empty) { if (filter === 'large' && cluster.size < 10) return; const color = PALETTE[i % PALETTE.length]; - const topIdeas = cluster.ideas.slice(0, 5); - const ideaListHtml = topIdeas.map(idea => - `
  • - ${idea.title} -
    • ` - ).join(''); - const extraCount = cluster.size - topIdeas.length; - const extraHtml = extraCount > 0 - ? `
  • +${extraCount} more
    • ` : ''; + const cardId = `cluster-${i}`; + const topIdeas = cluster.ideas.slice(0, 3); + + // Deduplicate ideas by title, track which drafts have each + const ideaByTitle = {}; + cluster.ideas.forEach(idea => { + if (!ideaByTitle[idea.title]) { + ideaByTitle[idea.title] = { ...idea, drafts: [] }; + } + ideaByTitle[idea.title].drafts.push(idea.draft_name); + }); + const uniqueIdeas = Object.values(ideaByTitle); + + // Preview: first 3 unique ideas + const previewHtml = uniqueIdeas.slice(0, 3).map(idea => { + const draftTag = idea.drafts.length > 1 + ? `(${idea.drafts.length} drafts)` + : `${idea.drafts[0].replace('draft-', '').substring(0, 20)}`; + return `
  • + ${idea.title} ${draftTag} +
    • `; + }).join(''); + const previewExtra = uniqueIdeas.length > 3 + ? `
  • +${uniqueIdeas.length - 3} more unique ideas
    • ` : ''; + + // Full idea list (shown on expand) + const fullIdeasHtml = uniqueIdeas.map(idea => { + const draftLinks = idea.drafts.map(d => + `${d.replace('draft-', '').substring(0, 28)}` + ).join(', '); + return `
    +
    ${idea.title}
    + ${idea.description ? `
    ${idea.description.substring(0, 200)}
    ` : ''} +
    ${draftLinks}
    +
    `; + }).join(''); // WG badges const wgBadges = (cluster.wgs || []).filter(w => w.wg !== 'none').map(w => @@ -224,22 +394,39 @@ if (data.empty) { ? `cross-WG` : ''; const card = document.createElement('div'); - card.className = 'bg-slate-900 rounded-xl border p-5 ' + + card.className = 'bg-slate-900 rounded-xl border p-5 cursor-pointer hover:border-slate-600 transition ' + (cluster.cross_wg ? 'border-amber-800/40' : 'border-slate-800'); + card.onclick = () => { + const detail = document.getElementById(cardId); + const chevron = document.getElementById(`chevron-${i}`); + if (detail.classList.contains('hidden')) { + detail.classList.remove('hidden'); + chevron.style.transform = 'rotate(180deg)'; + } else { + detail.classList.add('hidden'); + chevron.style.transform = ''; + } + }; card.innerHTML = `

    ${cluster.theme}

    ${crossBadge} ${cluster.size} ideas +
    - + ${(wgBadges || noneHtml) ? `

    Working Groups

    ${wgBadges} ${noneHtml}
    ` : ''} ${catBadges ? `

    Categories

    ${catBadges}
    ` : ''}

    ${cluster.drafts.length} source draft${cluster.drafts.length !== 1 ? 's' : ''}

    ${draftBadges}${extraDrafts}
    + + `; grid.appendChild(card); }); diff --git a/src/webui/templates/ratings.html b/src/webui/templates/ratings.html index c666462..59d9df6 100644 --- a/src/webui/templates/ratings.html +++ b/src/webui/templates/ratings.html @@ -185,8 +185,13 @@ document.getElementById('scatter').on('plotly_click', function(data) { return 'score-low'; } - function dimBadge(val) { - const cls = val >= 4 ? 'text-green-400' : val >= 3 ? 'text-yellow-400' : 'text-slate-500'; + function dimBadge(val, inverted = false) { + let cls; + if (inverted) { + cls = val <= 2 ? 'text-green-400' : val <= 3 ? 'text-yellow-400' : 'text-red-400'; + } else { + cls = val >= 4 ? 'text-green-400' : val >= 3 ? 'text-yellow-400' : 'text-slate-500'; + } return `${val}`; } @@ -207,7 +212,7 @@ document.getElementById('scatter').on('plotly_click', function(data) { ${dimBadge(d.maturity)} ${dimBadge(d.relevance)} ${dimBadge(d.momentum)} - ${dimBadge(d.overlap)} + ${dimBadge(d.overlap, true)} ${d.category}