Fix security, data integrity, and accuracy issues from 4-perspective review
Security fixes: - Fix SQL injection in db.py:update_generation_run (column name whitelist) - Flask SECRET_KEY from env var instead of hardcoded - Add LLM rating bounds validation (_clamp_rating, 1-10) - Fix JSON extraction trailing whitespace handling Data integrity: - Normalize 21 legacy category names to 11 canonical short forms - Add false_positive column, flag 73 non-AI drafts (361 relevant remain) - Document verified counts: 434 total/361 relevant drafts, 557 authors, 419 ideas, 11 gaps Code quality: - Fix version string 0.1.0 → 0.2.0 - Add close()/context manager to Embedder class - Dynamic matrix size instead of hardcoded "260x260" Blog accuracy: - Fix EU AI Act timeline (enforcement Aug 2026, not "18 months") - Distinguish OAuth consent from GDPR Einwilligung - Add EU AI Act Annex III context to hospital scenario - Add FIPA, eIDAS 2.0 references where relevant Methodology: - Add methodology.md documenting pipeline, limitations, rating rubric - Add LLM-as-judge caveats to analyzer.py - Document clustering threshold rationale Reviews from: legal (German/EU law), statistics, development, science perspectives. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -12,11 +12,11 @@ This is the story of who is writing the rules for AI agents, what their collabor
|
||||
|
||||
## The Numbers Behind the Names
|
||||
|
||||
Our analysis mapped **557 unique authors** from **230 organizations** across the 361 AI/agent drafts in the IETF pipeline. But those topline numbers mask extreme concentration.
|
||||
Our analysis mapped **557 unique authors** from **230 organizations** across the 434 AI/agent drafts in the IETF pipeline. But those topline numbers mask extreme concentration.
|
||||
|
||||
| Organization | Authors | Drafts |
|
||||
|-------------|--------:|-------:|
|
||||
| Huawei | 53 | 66 |
|
||||
| Huawei | 53 | 69 |
|
||||
| China Mobile | 24 | 35 |
|
||||
| Cisco | 24 | 26 |
|
||||
| Independent | 19 | 25 |
|
||||
@@ -27,7 +27,7 @@ Our analysis mapped **557 unique authors** from **230 organizations** across the
|
||||
| Five9 | 1 | 10 |
|
||||
| Ericsson | 4 | 9 |
|
||||
|
||||
One company -- Huawei -- contributes 18% of all drafts. The top six Chinese-linked organizations together contribute over 160 authors. This is not a general pattern across the IETF; it is specific to the AI agent space, and it tells a story about who considers these standards strategically important.
|
||||
One company -- Huawei -- contributes about 16% of all drafts (69 across all Huawei-named entities, consolidated from Huawei, Huawei Technologies, Huawei Canada, etc.). The top six Chinese-linked organizations together contribute over 160 authors. This is not a general pattern across the IETF; it is specific to the AI agent space, and it tells a story about who considers these standards strategically important.
|
||||
|
||||
## The Huawei Drafting Machine
|
||||
|
||||
@@ -51,7 +51,7 @@ Their 22 drafts cover a specific territory: agent networking frameworks for ente
|
||||
|
||||
Two deeper metrics reveal the nature of this operation:
|
||||
|
||||
**Volume over iteration.** Across the entire corpus, **55% of all 361 drafts** have never been revised beyond their first submission (rev-00). But the rate varies dramatically by organization. Of Huawei's drafts, **65% are at rev-00**. Compare that to Ericsson (11%), Siemens (0%), Nokia (20%), or Boeing (0%). The most serious iterators -- Boeing (avg 28.2 revisions per draft), Siemens (17.2), Sandelman Software (14.3) -- submit far fewer drafts but iterate relentlessly. Western companies submit fewer drafts but revise heavily -- incorporating feedback, advancing toward maturity. Huawei's pattern is the opposite: submit at volume, iterate rarely. Submitting a draft is cheap. Iterating it signals genuine investment.
|
||||
**Volume over iteration.** Across the entire corpus, **55% of all 434 drafts** have never been revised beyond their first submission (rev-00). But the rate varies dramatically by organization. Of Huawei's drafts, **65% are at rev-00**. Compare that to Ericsson (11%), Siemens (0%), Nokia (20%), or Boeing (0%). The most serious iterators -- Boeing (avg 28.2 revisions per draft), Siemens (17.2), Sandelman Software (14.3) -- submit far fewer drafts but iterate relentlessly. Western companies submit fewer drafts but revise heavily -- incorporating feedback, advancing toward maturity. Huawei's pattern is the opposite: submit at volume, iterate rarely. Submitting a draft is cheap. Iterating it signals genuine investment.
|
||||
|
||||
**Campaign timing.** Of Huawei's drafts, **43 were submitted in the four weeks before IETF 121 Dublin** -- 62% of the company's entire output, packed into a single pre-meeting window. For context, the entire corpus had 107 drafts in that period. Huawei alone accounted for **40% of all pre-IETF 121 submissions**. This is not organic growth. It is a coordinated submission campaign timed for maximum standards-body impact.
|
||||
|
||||
@@ -146,7 +146,7 @@ The one exception is Fraunhofer SIT's Henk Birkholz and Tradeverifyd's Orie Stee
|
||||
|
||||
Three implications emerge from the authorship data:
|
||||
|
||||
**1. Volume and influence are not the same thing.** Huawei's 66 drafts represent 18% of the corpus, but 65% have never been revised. The IETF rewards sustained engagement -- drafts that iterate through feedback cycles, reach working group adoption, and mature toward RFC status. A campaign that optimizes for volume at a pre-meeting deadline is playing a different game than one that optimizes for adoption. The quality scores bear this out: Huawei's team averages around 3.1, respectable but not exceptional. The organizations doing the deepest work (Ericsson at 4.8 average revision, Siemens at 17.2) submit far fewer drafts but iterate relentlessly.
|
||||
**1. Volume and influence are not the same thing.** Huawei's 69 drafts represent about 16% of the corpus, but 65% have never been revised. The IETF rewards sustained engagement -- drafts that iterate through feedback cycles, reach working group adoption, and mature toward RFC status. A campaign that optimizes for volume at a pre-meeting deadline is playing a different game than one that optimizes for adoption. The quality scores bear this out: Huawei's team averages around 3.1, respectable but not exceptional. The organizations doing the deepest work (Ericsson at 4.8 average revision, Siemens at 17.2) submit far fewer drafts but iterate relentlessly.
|
||||
|
||||
**2. The safety work comes from unexpected places.** The highest-quality safety and accountability drafts come not from the high-volume drafters but from smaller, specialized teams: Aylward (independent), Birkholz/Steele (Fraunhofer/Tradeverifyd), Rosenberg/White (Five9/Bitwave), and the JPMorgan-led multi-org team. The organizations doing the most drafting are focused on capability; the organizations doing the best safety work are doing the least drafting.
|
||||
|
||||
@@ -156,7 +156,7 @@ Three implications emerge from the authorship data:
|
||||
|
||||
### Key Takeaways
|
||||
|
||||
- **Huawei dominates** with 53 authors on 66 drafts (18% of corpus); their 13-person core team co-authors 22 drafts at 94% cohesion -- but 65% of those drafts have never been revised, and 43 were submitted in a single 4-week pre-meeting window
|
||||
- **Huawei dominates** with 53 authors on 69 drafts (~16% of corpus); their 13-person core team co-authors 22 drafts at 94% cohesion -- but 65% of those drafts have never been revised, and 43 were submitted in a single 4-week pre-meeting window
|
||||
- **Chinese institutions** collectively contribute 160+ of 557 authors; they form a tightly interconnected collaboration ecosystem
|
||||
- **Google has 9 drafts but Microsoft and Apple are largely absent** from AI agent standardization -- a notable strategic gap
|
||||
- **18 team blocs** detected; cross-team collaboration is sparse, with most cross-bloc pairs sharing only 1 draft
|
||||
@@ -167,4 +167,4 @@ Three implications emerge from the authorship data:
|
||||
|
||||
---
|
||||
|
||||
*Data from the IETF Draft Analyzer, covering 361 drafts, 557 authors, and 18 detected team blocs. Co-authorship analysis uses 70% pairwise draft overlap threshold with 3+ shared drafts.*
|
||||
*Data from the IETF Draft Analyzer, covering 434 drafts, 557 authors, and 18 detected team blocs. Co-authorship analysis uses 70% pairwise draft overlap threshold with 3+ shared drafts.*
|
||||
|
||||
Reference in New Issue
Block a user