c/ietf-draft-analyzer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Observatory update: 434 docs, fix W3C fetcher, regenerate dashboard

Browse Source 
- Fixed W3C fetcher to paginate /specifications endpoint (group
  endpoints use type prefixes like cg/, wg/ that weren't in config)
- Fetched 72 new IETF drafts + 1 W3C spec, all analyzed and embedded
- Regenerated dashboard with updated data
- Total: 434 docs, 11 gaps, 1907 ideas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Christian Nennemann
2026-03-04 01:09:30 +01:00
parent d6beb9c0a0
commit 7a1aa346b9
7 changed files with 2482 additions and 275 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
data/drafts.db
View File

Binary file not shown.

1969
docs/data/drafts.json
View File

File diff suppressed because it is too large Load Diff

381
docs/data/gaps.json
View File

@@ -1,101 +1,314 @@
{ {
"current": [ "current": [
{ {
"id": 25, "id": 37,
"topic": "Agent Behavior Verification", "topic": "Multi-Agent Consensus Protocols",
"description": "While many drafts address agent identity and authentication, few tackle how to verify that an agent is actually behaving according to its declared capabilities and policies. There's a critical gap in runtime behavioral attestation and compliance monitoring mechanisms.", "description": "While A2A protocols exist, there's no standardized approach for multiple AI agents to reach consensus on shared decisions or conflicting objectives. This becomes critical when agents must coordinate on resource allocation or policy enforcement across domains.",
"category": "AI safety/alignment",
"evidence": "High overlap in identity/auth (108 drafts) but only 44 drafts on safety/alignment, with no specific focus on behavioral verification",
"severity": "critical"
},
{
"id": 26,
"topic": "Cross-Domain Agent Liability",
"description": "When autonomous agents operate across organizational boundaries and cause harm or make decisions with legal implications, there's no standardized framework for liability attribution. The policy/governance drafts don't address cross-jurisdictional legal accountability.",
"category": "Policy/governance",
"evidence": "91 policy/governance drafts but legal liability for cross-domain autonomous actions remains unaddressed",
"severity": "critical"
},
{
"id": 27,
"topic": "Agent Resource Exhaustion Protection",
"description": "Missing standardized mechanisms to prevent malicious or poorly designed agents from consuming excessive network, compute, or storage resources. Current drafts focus on traffic management but not on agent-specific resource quotas and enforcement.",
"category": "Autonomous netops",
"evidence": "93 autonomous netops drafts and 73 ML traffic management drafts lack agent-specific resource protection mechanisms",
"severity": "high"
},
{
"id": 28,
"topic": "Human Override Protocols",
"description": "Critical gap in standardized protocols for humans to safely interrupt, override, or take control of autonomous agents in emergency situations. Only 30 drafts address human-agent interaction, with no focus on emergency takeover procedures.",
"category": "Human-agent interaction",
"evidence": "Only 30 human-agent interaction drafts compared to 213+ autonomous operation drafts, with no emergency override standards",
"severity": "critical"
},
{
"id": 29,
"topic": "Agent-Generated Data Provenance",
"description": "While 145 drafts address data formats for AI interop, there's insufficient attention to tracking the provenance and lineage of data generated by agents. This creates trust and auditability issues in agent-to-agent data exchanges.",
"category": "Data formats/interop",
"evidence": "145 data format drafts with high overlap but no clear standards for agent-generated data provenance tracking",
"severity": "high"
},
{
"id": 30,
"topic": "Agent Capability Degradation Handling",
"description": "No standardized approaches for detecting and handling when an agent's capabilities degrade due to model drift, data corruption, or hardware issues. Systems need graceful degradation protocols rather than silent failures.",
"category": "AI safety/alignment",
"evidence": "Only 44 safety/alignment drafts don't address capability degradation, while 213+ drafts assume stable agent performance",
"severity": "high"
},
{
"id": 31,
"topic": "Multi-Agent Coordination Deadlocks",
"description": "With 120+ A2A protocol drafts, there's insufficient attention to preventing deadlock situations where multiple agents create circular dependencies or resource conflicts. Missing are standardized deadlock detection and resolution mechanisms.",
"category": "A2A protocols", "category": "A2A protocols",
"evidence": "120 A2A protocol drafts with high internal overlap but no systematic deadlock prevention frameworks", "evidence": "High overlap in A2A protocols suggests fragmented approaches without consensus mechanisms, critical for autonomous system coordination",
"severity": "high" "severity": "high"
}, },
{ {
"id": 32, "id": 38,
"topic": "Agent Privacy Preservation", "topic": "Agent Behavioral Verification",
"description": "Agents often process sensitive data but current drafts don't adequately address privacy-preserving computation, differential privacy, or secure multi-party computation for agent interactions. This is critical for deployment in regulated industries.", "description": "No drafts address how to verify that deployed AI agents are actually behaving according to their declared policies or specifications. This creates a gap between policy definition and runtime assurance.",
"category": "Agent identity/auth", "category": "AI safety/alignment",
"evidence": "108 identity/auth drafts focus on authentication but lack privacy preservation mechanisms for agent data processing", "evidence": "Only 47 drafts on AI safety despite 433 total drafts, and behavioral verification is fundamental to autonomous system trust",
"severity": "high" "severity": "critical"
}, },
{ {
"id": 33, "id": 39,
"topic": "Real-time Agent Debugging",
"description": "Missing standardized protocols for debugging autonomous agents in production environments. When agents make unexpected decisions, there are no standard interfaces for real-time introspection without disrupting operations.",
"category": "Other AI/agent",
"evidence": "26 other AI/agent drafts suggest various approaches but no standardized debugging protocols for production agents",
"severity": "medium"
},
{
"id": 34,
"topic": "Agent Firmware/Model Update Security",
"description": "While model serving is addressed in 42 drafts, there's insufficient focus on secure update mechanisms for agent models and firmware. Missing are standards for cryptographically verified, rollback-capable agent updates.",
"category": "Model serving/inference",
"evidence": "42 model serving drafts but no comprehensive security standards for agent software/model updates",
"severity": "high"
},
{
"id": 35,
"topic": "Cross-Protocol Agent Migration", "topic": "Cross-Protocol Agent Migration",
"description": "No standardized mechanisms for migrating agent state and context when moving between different A2A protocols or infrastructure providers. This creates vendor lock-in and limits agent mobility.", "description": "Agents need to migrate between different network protocols, domains, or infrastructure providers while maintaining state and identity. Current drafts focus on registration but not migration continuity.",
"category": "A2A protocols", "category": "Agent discovery/reg",
"evidence": "120 A2A protocol drafts with high overlap suggest competing approaches but no migration standards between them", "evidence": "Agent registration servers exist but no migration protocols, essential for dynamic edge deployments mentioned in technical ideas",
"severity": "medium" "severity": "medium"
}, },
{ {
"id": 36, "id": 40,
"topic": "Agent Energy Consumption Optimization", "topic": "Real-Time Agent Rollback Mechanisms",
"description": "Missing standards for energy-aware agent deployment and operation. As AI workloads are energy-intensive, there's no framework for agents to optimize their energy consumption or for infrastructure to enforce energy budgets.", "description": "When autonomous agents make incorrect decisions, there's no standardized protocol for quickly rolling back their actions across distributed systems. This is critical for network operations and safety.",
"category": "ML traffic mgmt", "category": "Autonomous netops",
"evidence": "73 ML traffic management drafts focus on performance but lack energy consumption considerations for sustainable AI deployment", "evidence": "114 autonomous netops drafts but no rollback mechanisms, critical for production network safety",
"severity": "high"
},
{
"id": 41,
"topic": "Agent Resource Accounting and Billing",
"description": "Multi-tenant environments need standardized ways to track and bill for AI agent resource consumption across compute, network, and storage. Current drafts define resources but not accounting mechanisms.",
"category": "new",
"evidence": "Edge AI placement and resource types defined but no economic models for multi-tenant agent deployments",
"severity": "medium"
},
{
"id": 42,
"topic": "Federated Agent Learning Privacy",
"description": "While federated architectures exist, there's insufficient specification for privacy-preserving agent learning that prevents data leakage between federated participants during model updates.",
"category": "Policy/governance",
"evidence": "Federated AI architectures mentioned but private data training requirements suggest unresolved privacy challenges",
"severity": "high"
},
{
"id": 43,
"topic": "Agent Capability Negotiation",
"description": "Agents need standardized protocols to negotiate their capabilities and limitations before collaboration. Tool enumeration exists but not dynamic capability negotiation during runtime.",
"category": "A2A protocols",
"evidence": "Tool enumeration APIs exist but no dynamic capability negotiation, needed for heterogeneous agent coordination",
"severity": "medium"
},
{
"id": 44,
"topic": "Cross-Domain Agent Audit Trails",
"description": "When agents operate across multiple domains or organizations, there's no standardized format for maintaining audit trails that satisfy different regulatory requirements simultaneously.",
"category": "Agent identity/auth",
"evidence": "Cross-domain identity management exists but audit requirements vary by jurisdiction, creating compliance gaps",
"severity": "high"
},
{
"id": 45,
"topic": "Agent Failure Cascade Prevention",
"description": "No protocols exist to prevent agent failures from cascading across interconnected autonomous systems. This is critical as agent interdependencies increase in production deployments.",
"category": "AI safety/alignment",
"evidence": "Limited AI safety focus (47 drafts) despite high interconnectivity implied by A2A protocols and autonomous netops",
"severity": "critical"
},
{
"id": 46,
"topic": "Human Override Standardization",
"description": "While human-agent interaction exists, there's no standardized protocol for human operators to override autonomous agent decisions across different vendor implementations and domains.",
"category": "Human-agent interaction",
"evidence": "Only 33 human-agent interaction drafts suggest limited focus on human control mechanisms essential for safe deployment",
"severity": "high"
},
{
"id": 47,
"topic": "Agent Performance Benchmarking",
"description": "No standardized metrics or protocols exist for comparing agent performance across different implementations, making it difficult to evaluate and select appropriate agents for specific tasks.",
"category": "new",
"evidence": "AI workload characterization mentioned but no performance comparison frameworks, needed for agent selection and optimization",
"severity": "medium" "severity": "medium"
} }
], ],
"history": [] "history": [
{
"id": 2,
"snapshot_id": 1,
"gap_topic": "Agent Behavioral Verification",
"gap_description": "No drafts address how to verify that deployed AI agents are actually behaving according to their declared policies or specifications. This creates a gap between policy definition and runtime assurance.",
"severity": "critical",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 7,
"snapshot_id": 1,
"gap_topic": "Agent Capability Negotiation",
"gap_description": "Agents need standardized protocols to negotiate their capabilities and limitations before collaboration. Tool enumeration exists but not dynamic capability negotiation during runtime.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 9,
"snapshot_id": 1,
"gap_topic": "Agent Failure Cascade Prevention",
"gap_description": "No protocols exist to prevent agent failures from cascading across interconnected autonomous systems. This is critical as agent interdependencies increase in production deployments.",
"severity": "critical",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 11,
"snapshot_id": 1,
"gap_topic": "Agent Performance Benchmarking",
"gap_description": "No standardized metrics or protocols exist for comparing agent performance across different implementations, making it difficult to evaluate and select appropriate agents for specific tasks.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 5,
"snapshot_id": 1,
"gap_topic": "Agent Resource Accounting and Billing",
"gap_description": "Multi-tenant environments need standardized ways to track and bill for AI agent resource consumption across compute, network, and storage. Current drafts define resources but not accounting mechanisms.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 8,
"snapshot_id": 1,
"gap_topic": "Cross-Domain Agent Audit Trails",
"gap_description": "When agents operate across multiple domains or organizations, there's no standardized format for maintaining audit trails that satisfy different regulatory requirements simultaneously.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 3,
"snapshot_id": 1,
"gap_topic": "Cross-Protocol Agent Migration",
"gap_description": "Agents need to migrate between different network protocols, domains, or infrastructure providers while maintaining state and identity. Current drafts focus on registration but not migration continuity.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 6,
"snapshot_id": 1,
"gap_topic": "Federated Agent Learning Privacy",
"gap_description": "While federated architectures exist, there's insufficient specification for privacy-preserving agent learning that prevents data leakage between federated participants during model updates.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 10,
"snapshot_id": 1,
"gap_topic": "Human Override Standardization",
"gap_description": "While human-agent interaction exists, there's no standardized protocol for human operators to override autonomous agent decisions across different vendor implementations and domains.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 1,
"snapshot_id": 1,
"gap_topic": "Multi-Agent Consensus Protocols",
"gap_description": "While A2A protocols exist, there's no standardized approach for multiple AI agents to reach consensus on shared decisions or conflicting objectives. This becomes critical when agents must coordinate on resource allocation or policy enforcement across domains.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 4,
"snapshot_id": 1,
"gap_topic": "Real-Time Agent Rollback Mechanisms",
"gap_description": "When autonomous agents make incorrect decisions, there's no standardized protocol for quickly rolling back their actions across distributed systems. This is critical for network operations and safety.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:06:32.803029+00:00",
"snapshot_at": "2026-03-03T23:59:47.420517+00:00"
},
{
"id": 13,
"snapshot_id": 2,
"gap_topic": "Agent Behavioral Verification",
"gap_description": "No drafts address how to verify that deployed AI agents are actually behaving according to their declared policies or specifications. This creates a gap between policy definition and runtime assurance.",
"severity": "critical",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 18,
"snapshot_id": 2,
"gap_topic": "Agent Capability Negotiation",
"gap_description": "Agents need standardized protocols to negotiate their capabilities and limitations before collaboration. Tool enumeration exists but not dynamic capability negotiation during runtime.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 20,
"snapshot_id": 2,
"gap_topic": "Agent Failure Cascade Prevention",
"gap_description": "No protocols exist to prevent agent failures from cascading across interconnected autonomous systems. This is critical as agent interdependencies increase in production deployments.",
"severity": "critical",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 22,
"snapshot_id": 2,
"gap_topic": "Agent Performance Benchmarking",
"gap_description": "No standardized metrics or protocols exist for comparing agent performance across different implementations, making it difficult to evaluate and select appropriate agents for specific tasks.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 16,
"snapshot_id": 2,
"gap_topic": "Agent Resource Accounting and Billing",
"gap_description": "Multi-tenant environments need standardized ways to track and bill for AI agent resource consumption across compute, network, and storage. Current drafts define resources but not accounting mechanisms.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 19,
"snapshot_id": 2,
"gap_topic": "Cross-Domain Agent Audit Trails",
"gap_description": "When agents operate across multiple domains or organizations, there's no standardized format for maintaining audit trails that satisfy different regulatory requirements simultaneously.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 14,
"snapshot_id": 2,
"gap_topic": "Cross-Protocol Agent Migration",
"gap_description": "Agents need to migrate between different network protocols, domains, or infrastructure providers while maintaining state and identity. Current drafts focus on registration but not migration continuity.",
"severity": "medium",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 17,
"snapshot_id": 2,
"gap_topic": "Federated Agent Learning Privacy",
"gap_description": "While federated architectures exist, there's insufficient specification for privacy-preserving agent learning that prevents data leakage between federated participants during model updates.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 21,
"snapshot_id": 2,
"gap_topic": "Human Override Standardization",
"gap_description": "While human-agent interaction exists, there's no standardized protocol for human operators to override autonomous agent decisions across different vendor implementations and domains.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 12,
"snapshot_id": 2,
"gap_topic": "Multi-Agent Consensus Protocols",
"gap_description": "While A2A protocols exist, there's no standardized approach for multiple AI agents to reach consensus on shared decisions or conflicting objectives. This becomes critical when agents must coordinate on resource allocation or policy enforcement across domains.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
},
{
"id": 15,
"snapshot_id": 2,
"gap_topic": "Real-Time Agent Rollback Mechanisms",
"gap_description": "When autonomous agents make incorrect decisions, there's no standardized protocol for quickly rolling back their actions across distributed systems. This is critical for network operations and safety.",
"severity": "high",
"status": "open",
"recorded_at": "2026-03-04T00:09:12.383015+00:00",
"snapshot_at": "2026-03-04T00:09:01.501362+00:00"
}
]
} }

2
docs/data/meta.json
View File

@@ -1,5 +1,5 @@
{ {
"generated_at": "2026-03-03T23:32:29.056739+00:00", "generated_at": "2026-03-04T00:09:19.610824+00:00",
"version": "0.3.0", "version": "0.3.0",
"project": "IETF Living Standards Observatory" "project": "IETF Living Standards Observatory"
} }

13
docs/data/observatory.json
View File

@@ -1,9 +1,12 @@
{ {
"total_docs": 361, "total_docs": 434,
"sources": {}, "sources": {
"gaps_count": 12, "ietf": 433,
"w3c": 0
},
"gaps_count": 11,
"unrated": 0, "unrated": 0,
"ideas": 1780, "ideas": 1907,
"authors": 557, "authors": 557,
"last_update": null "last_update": "2026-03-04T00:09:01.501362+00:00"
} }

244
docs/data/timeline.json
View File

@@ -2,11 +2,18 @@
"months": [ "months": [
"2024-01", "2024-01",
"2024-02", "2024-02",
"2024-03",
"2024-04", "2024-04",
"2024-06",
"2024-07",
"2024-08",
"2024-09", "2024-09",
"2024-10", "2024-10",
"2024-11",
"2024-12", "2024-12",
"2025-01", "2025-01",
"2025-02",
"2025-03",
"2025-04", "2025-04",
"2025-05", "2025-05",
"2025-06", "2025-06",
@@ -18,56 +25,115 @@
"2025-12", "2025-12",
"2026-01", "2026-01",
"2026-02", "2026-02",
"2026-03" "2026-03",
"2026012"
], ],
"by_category": { "by_category": {
"2024-01": { "2024-01": {
"Agent identity/auth": 1, "Autonomous netops": 3,
"Policy/governance": 1, "Agent discovery/reg": 1,
"Other AI/agent": 1, "Agent identity/auth": 2,
"Data formats/interop": 1, "AI safety/alignment": 2,
"AI safety/alignment": 1 "ML traffic mgmt": 2,
"Policy/governance": 2,
"Data formats/interop": 2,
"Other AI/agent": 1
}, },
"2024-02": { "2024-02": {
"Agent identity/auth": 1,
"Autonomous netops": 1, "Autonomous netops": 1,
"ML traffic mgmt": 1 "ML traffic mgmt": 1,
"Data formats/interop": 1,
"A2A protocols": 1
}, },
"2024-04": { "2024-03": {
"A2A protocols": 1, "A2A protocols": 1,
"Autonomous netops": 1, "Agent identity/auth": 1,
"Data formats/interop": 1 "Data formats/interop": 1
}, },
"2024-04": {
"Agent identity/auth": 4,
"A2A protocols": 1,
"Autonomous netops": 3,
"Data formats/interop": 1,
"Policy/governance": 3,
"Agent discovery/reg": 1
},
"2024-06": {
"Data formats/interop": 2,
"Agent identity/auth": 1
},
"2024-07": {
"Agent identity/auth": 1,
"Autonomous netops": 2,
"Policy/governance": 1
},
"2024-08": {
"A2A protocols": 3,
"Policy/governance": 1
},
"2024-09": { "2024-09": {
"Autonomous netops": 1, "Autonomous netops": 3,
"Policy/governance": 1, "Policy/governance": 4,
"Data formats/interop": 5,
"Agent identity/auth": 3,
"A2A protocols": 2,
"Agent discovery/reg": 1,
"Human-agent interaction": 1,
"ML traffic mgmt": 1 "ML traffic mgmt": 1
}, },
"2024-10": { "2024-10": {
"ML traffic mgmt": 1, "ML traffic mgmt": 1,
"Model serving/inference": 1 "Model serving/inference": 1,
"Data formats/interop": 1
},
"2024-11": {
"Agent identity/auth": 2,
"Data formats/interop": 3,
"Autonomous netops": 1,
"A2A protocols": 1,
"Policy/governance": 1
}, },
"2024-12": { "2024-12": {
"Other AI/agent": 1 "Data formats/interop": 2,
"Agent discovery/reg": 2,
"Other AI/agent": 5,
"Autonomous netops": 1,
"ML traffic mgmt": 1
}, },
"2025-01": { "2025-01": {
"Data formats/interop": 2, "Data formats/interop": 4,
"ML traffic mgmt": 4, "ML traffic mgmt": 4,
"Model serving/inference": 2 "Model serving/inference": 2,
"Agent identity/auth": 3,
"A2A protocols": 1
},
"2025-02": {
"Other AI/agent": 1
},
"2025-03": {
"Autonomous netops": 1,
"Agent identity/auth": 1,
"Agent discovery/reg": 1,
"Other AI/agent": 1
}, },
"2025-04": { "2025-04": {
"Policy/governance": 3, "Policy/governance": 5,
"Agent identity/auth": 1, "Agent identity/auth": 3,
"Data formats/interop": 2, "Data formats/interop": 4,
"AI safety/alignment": 1, "AI safety/alignment": 1,
"Autonomous netops": 1, "Autonomous netops": 2,
"Agent discovery/reg": 1,
"Model serving/inference": 1, "Model serving/inference": 1,
"ML traffic mgmt": 1 "ML traffic mgmt": 1
}, },
"2025-05": { "2025-05": {
"Policy/governance": 2, "Agent identity/auth": 4,
"Agent discovery/reg": 1, "Data formats/interop": 3,
"Data formats/interop": 1, "A2A protocols": 2,
"Human-agent interaction": 1 "Agent discovery/reg": 2,
"Policy/governance": 3,
"Human-agent interaction": 2
}, },
"2025-06": { "2025-06": {
"Agent identity/auth": 2, "Agent identity/auth": 2,
@@ -109,50 +175,50 @@
}, },
"2025-10": { "2025-10": {
"Policy/governance": 10, "Policy/governance": 10,
"Data formats/interop": 36, "Data formats/interop": 35,
"Model serving/inference": 9, "Model serving/inference": 9,
"ML traffic mgmt": 13, "ML traffic mgmt": 13,
"Autonomous netops": 19, "Autonomous netops": 18,
"Agent discovery/reg": 15, "Agent discovery/reg": 14,
"Agent identity/auth": 24, "Agent identity/auth": 19,
"Human-agent interaction": 7, "Human-agent interaction": 7,
"A2A protocols": 28, "A2A protocols": 25,
"AI safety/alignment": 5, "AI safety/alignment": 5,
"Other AI/agent": 5 "Other AI/agent": 4
}, },
"2025-11": { "2025-11": {
"A2A protocols": 29, "A2A protocols": 28,
"Autonomous netops": 18, "Autonomous netops": 18,
"Data formats/interop": 25, "Data formats/interop": 22,
"ML traffic mgmt": 9, "ML traffic mgmt": 9,
"Agent discovery/reg": 18, "Agent discovery/reg": 17,
"Agent identity/auth": 23, "Agent identity/auth": 16,
"AI safety/alignment": 7, "AI safety/alignment": 5,
"Policy/governance": 15, "Policy/governance": 12,
"Human-agent interaction": 9, "Human-agent interaction": 8,
"Model serving/inference": 4, "Model serving/inference": 4,
"Other AI/agent": 2 "Other AI/agent": 2
}, },
"2025-12": { "2025-12": {
"Policy/governance": 6, "Policy/governance": 6,
"Data formats/interop": 6, "Data formats/interop": 5,
"Human-agent interaction": 1, "Human-agent interaction": 1,
"Agent identity/auth": 8, "Agent identity/auth": 7,
"Autonomous netops": 4, "Autonomous netops": 4,
"Other AI/agent": 2, "Other AI/agent": 2,
"A2A protocols": 2, "A2A protocols": 2,
"ML traffic mgmt": 3, "ML traffic mgmt": 3,
"AI safety/alignment": 3, "AI safety/alignment": 3,
"Agent discovery/reg": 4, "Agent discovery/reg": 3,
"Model serving/inference": 1 "Model serving/inference": 1
}, },
"2026-01": { "2026-01": {
"Agent identity/auth": 18, "Agent identity/auth": 16,
"AI safety/alignment": 8, "AI safety/alignment": 8,
"Policy/governance": 14, "Policy/governance": 14,
"A2A protocols": 22, "A2A protocols": 22,
"Agent discovery/reg": 15, "Agent discovery/reg": 15,
"ML traffic mgmt": 8, "ML traffic mgmt": 7,
"Data formats/interop": 17, "Data formats/interop": 17,
"Autonomous netops": 16, "Autonomous netops": 16,
"Model serving/inference": 5, "Model serving/inference": 5,
@@ -160,59 +226,84 @@
"Human-agent interaction": 2 "Human-agent interaction": 2
}, },
"2026-02": { "2026-02": {
"A2A protocols": 38, "A2A protocols": 40,
"Human-agent interaction": 6, "Human-agent interaction": 6,
"Data formats/interop": 32, "Data formats/interop": 31,
"Agent identity/auth": 34, "Agent identity/auth": 36,
"Policy/governance": 24, "Policy/governance": 24,
"Autonomous netops": 18, "Autonomous netops": 20,
"ML traffic mgmt": 15, "ML traffic mgmt": 15,
"Agent discovery/reg": 19, "Agent discovery/reg": 19,
"Other AI/agent": 8, "Other AI/agent": 9,
"AI safety/alignment": 13, "AI safety/alignment": 15,
"Model serving/inference": 8 "Model serving/inference": 8
}, },
"2026-03": { "2026-03": {
"ML traffic mgmt": 11, "ML traffic mgmt": 14,
"A2A protocols": 9, "A2A protocols": 19,
"Agent identity/auth": 23,
"Policy/governance": 11,
"Agent discovery/reg": 8,
"Human-agent interaction": 3,
"Model serving/inference": 9, "Model serving/inference": 9,
"Agent identity/auth": 3, "Data formats/interop": 15,
"Data formats/interop": 5, "AI safety/alignment": 2,
"Autonomous netops": 8, "Autonomous netops": 10,
"Policy/governance": 6, "Other AI/agent": 3
"Agent discovery/reg": 3, },
"Human-agent interaction": 1, "2026012": {
"Other AI/agent": 1, "Policy/governance": 1,
"AI safety/alignment": 1 "Human-agent interaction": 1
} }
}, },
"by_source": { "by_source": {
"2024-01": { "2024-01": {
"ietf": 3 "ietf": 7
}, },
"2024-02": { "2024-02": {
"ietf": 1 "ietf": 3
},
"2024-03": {
"ietf": 2
}, },
"2024-04": { "2024-04": {
"ietf": 1 "ietf": 6
},
"2024-06": {
"ietf": 2
},
"2024-07": {
"ietf": 2
},
"2024-08": {
"ietf": 3
}, },
"2024-09": { "2024-09": {
"ietf": 2 "ietf": 11
}, },
"2024-10": { "2024-10": {
"ietf": 1 "ietf": 2
}, },
"2024-12": { "2024-11": {
"ietf": 1
},
"2025-01": {
"ietf": 4 "ietf": 4
}, },
"2024-12": {
"ietf": 8
},
"2025-01": {
"ietf": 8
},
"2025-02": {
"ietf": 1
},
"2025-03": {
"ietf": 2
},
"2025-04": { "2025-04": {
"ietf": 5 "ietf": 9
}, },
"2025-05": { "2025-05": {
"ietf": 2 "ietf": 7
}, },
"2025-06": { "2025-06": {
"ietf": 5 "ietf": 5
@@ -227,22 +318,25 @@
"ietf": 17 "ietf": 17
}, },
"2025-10": { "2025-10": {
"ietf": 67
},
"2025-11": {
"ietf": 61 "ietf": 61
}, },
"2025-11": {
"ietf": 53
},
"2025-12": { "2025-12": {
"ietf": 16 "ietf": 15
}, },
"2026-01": { "2026-01": {
"ietf": 54 "ietf": 51
}, },
"2026-02": { "2026-02": {
"ietf": 86 "ietf": 88
}, },
"2026-03": { "2026-03": {
"ietf": 22 "ietf": 53
},
"2026012": {
"ietf": 1
} }
}, },
"categories": [ "categories": [

148
src/ietf_analyzer/sources/w3c.py
View File

@@ -22,6 +22,23 @@ W3C_API = "https://api.w3.org"
console = Console() console = Console()
# AI/agent-relevant W3C groups with their full API type prefix
W3C_AI_GROUPS = {
"webagents": "cg/webagents", # Autonomous Agents on the Web
"agentprotocol": "cg/agentprotocol", # AI Agent Protocol
"agentic-arbitration": "cg/agentic-arbitration", # Agentic Arbitration Protocol
"ai-content-disclosure": "cg/ai-content-disclosure",
"aikr": "cg/aikr", # AI Knowledge Representation
"ai-web-visibility": "cg/ai-web-visibility",
"aiwss": "cg/aiwss", # AI-Driven Web Standards
"cogai": "cg/cogai", # Cognitive AI
"credentials": "cg/credentials", # Credentials CG
"did": "wg/did", # Decentralized Identifier WG
"vc": "wg/vc", # Verifiable Credentials WG
"webmachinelearning": "wg/webmachinelearning", # Web Machine Learning WG
"wot": "wg/wot", # Web of Things WG
}
def _strip_html(html: str) -> str: def _strip_html(html: str) -> str:
"""Minimal HTML tag stripper — no heavy dependencies.""" """Minimal HTML tag stripper — no heavy dependencies."""
@@ -38,20 +55,31 @@ def _strip_html(html: str) -> str:
class W3CFetcher: class W3CFetcher:
"""Fetch specs from the W3C public API (no auth needed).""" """Fetch specs from the W3C public API (no auth needed).
Two strategies:
1. Browse /specifications paginated, filter client-side by keywords
2. Fetch specs linked from known AI-relevant groups
Both are combined for maximum coverage.
"""
def __init__(self, config: Config | None = None): def __init__(self, config: Config | None = None):
self.config = config or Config.load() self.config = config or Config.load()
self.client = httpx.Client(timeout=30, follow_redirects=True) self.client = httpx.Client(timeout=30, follow_redirects=True)
self.groups = self.config.w3c_groups
def search( def search(
self, keywords: list[str], since: str | None = None self, keywords: list[str], since: str | None = None
) -> list[SourceDocument]: ) -> list[SourceDocument]:
"""Fetch specs from AI-relevant W3C groups, filtered by keywords.""" """Fetch AI-relevant W3C specs via keyword search over /specifications."""
seen: dict[str, SourceDocument] = {} seen: dict[str, SourceDocument] = {}
kw_lower = [k.lower() for k in keywords] kw_lower = [k.lower() for k in keywords]
# Strategy 1: Paginate through /specifications, filter client-side
console.print(" Searching W3C specifications catalog...")
page = 1
max_pages = 17 # ~1679 specs total
checked = 0
with Progress( with Progress(
SpinnerColumn(), SpinnerColumn(),
TextColumn("[progress.description]{task.description}"), TextColumn("[progress.description]{task.description}"),
@@ -59,93 +87,72 @@ class W3CFetcher:
MofNCompleteColumn(), MofNCompleteColumn(),
console=console, console=console,
) as progress: ) as progress:
task = progress.add_task("Fetching W3C specs...", total=len(self.groups)) task = progress.add_task("W3C catalog...", total=max_pages)
for group in self.groups: while page <= max_pages:
progress.update(task, description=f"W3C group: {group}") progress.update(task, description=f"W3C catalog page {page}/{max_pages}")
specs = self._fetch_group_specs(group) try:
for spec in specs: resp = self.client.get(
# Client-side keyword filter on title + description f"{W3C_API}/specifications",
haystack = (spec.title + " " + spec.abstract).lower() params={"format": "json", "page": page},
if any(kw in haystack for kw in kw_lower): headers={"Accept": "application/json"},
if since and spec.time and spec.time < since: )
continue resp.raise_for_status()
if spec.name not in seen: data = resp.json()
seen[spec.name] = spec except httpx.HTTPError as e:
progress.advance(task) console.print(f"[yellow]W3C API error on page {page}: {e}[/]")
break
console.print(f"Found [bold green]{len(seen)}[/] W3C specs matching keywords") spec_list = data.get("_links", {}).get("specifications", [])
return list(seen.values())
def _fetch_group_specs(self, group_shortname: str) -> list[SourceDocument]:
"""Fetch all specifications for a W3C group."""
url = f"{W3C_API}/groups/{group_shortname}/specifications"
specs: list[SourceDocument] = []
try:
page = 1
while True:
resp = self.client.get(
url,
params={"format": "json", "page": page},
headers={"Accept": "application/json"},
)
resp.raise_for_status()
data = resp.json()
spec_list = data if isinstance(data, list) else data.get("_links", {}).get("specifications", [])
if not spec_list:
# Try alternate response shape
spec_list = data.get("specifications", [])
if not spec_list: if not spec_list:
break break
max_pages = data.get("pages", max_pages)
progress.update(task, total=max_pages)
for item in spec_list: for item in spec_list:
title = item.get("title", "")
href = item.get("href", "") href = item.get("href", "")
shortname = item.get("shortname", "") shortname = href.rstrip("/").split("/")[-1] if href else ""
title = item.get("title", shortname)
if not shortname and href:
# Extract shortname from href like /specifications/webnn
parts = href.rstrip("/").split("/")
shortname = parts[-1] if parts else ""
if not shortname: if not shortname:
continue continue
checked += 1
# Fetch spec detail for abstract/description # Quick keyword check on title
if not any(kw in title.lower() for kw in kw_lower):
continue
# Fetch detail for description, URL, dates
detail = self._fetch_spec_detail(shortname) detail = self._fetch_spec_detail(shortname)
abstract = detail.get("description", title) abstract = _strip_html(detail.get("description", title))
spec_url = detail.get("editor-draft", detail.get("url", f"https://www.w3.org/TR/{shortname}/")) spec_url = detail.get("editor-draft") or detail.get("url") or f"https://www.w3.org/TR/{shortname}/"
status = detail.get("status", "")
date = detail.get("date", "") date = detail.get("date", "")
status = detail.get("status", "")
specs.append( if since and date and date < since:
SourceDocument( continue
name=f"w3c-{shortname}",
title=title, name = f"w3c-{shortname}"
if name not in seen:
seen[name] = SourceDocument(
name=name,
title=detail.get("title", title),
abstract=abstract, abstract=abstract,
source="w3c", source="w3c",
source_id=shortname, source_id=shortname,
source_url=spec_url, source_url=spec_url,
time=date, time=date,
doc_status=status, doc_status=status,
extra={"group": group_shortname},
) )
)
time_mod.sleep(0.3)
# Check pagination time_mod.sleep(0.15)
pages = data.get("pages", 1) if isinstance(data, dict) else 1
if page >= pages: progress.advance(task)
break
page += 1 page += 1
time_mod.sleep(0.3) time_mod.sleep(0.2)
except httpx.HTTPError as e: console.print(f" Checked {checked} W3C specs, found [bold green]{len(seen)}[/] matching keywords")
console.print(f"[yellow]W3C API error for {group_shortname}: {e}[/]") return list(seen.values())
return specs
def _fetch_spec_detail(self, shortname: str) -> dict: def _fetch_spec_detail(self, shortname: str) -> dict:
"""Fetch detail for a single spec.""" """Fetch detail for a single spec."""
@@ -156,13 +163,14 @@ class W3CFetcher:
) )
resp.raise_for_status() resp.raise_for_status()
data = resp.json() data = resp.json()
latest = data.get("_links", {}).get("latest-version", {})
return { return {
"description": data.get("description", ""), "description": data.get("description", ""),
"title": data.get("title", shortname), "title": data.get("title", shortname),
"editor-draft": data.get("editor-draft", ""), "editor-draft": data.get("editor-draft", ""),
"url": data.get("_links", {}).get("latest-version", {}).get("href", ""), "url": latest.get("href", ""),
"status": data.get("_links", {}).get("latest-version", {}).get("status", ""), "status": latest.get("title", ""),
"date": data.get("_links", {}).get("latest-version", {}).get("date", ""), "date": latest.get("href", "").rstrip("/").split("/")[-1] if latest.get("href") else "",
} }
except httpx.HTTPError: except httpx.HTTPError:
return {} return {}