Platform upgrade: semantic search, citations, readiness, tests, Docker
Major features added by 5 parallel agent teams: - Semantic "Ask" (NL queries via FTS5 + embeddings + Claude synthesis) - Global search across drafts, ideas, authors, gaps - REST API expansion (14 endpoints, up from 3) with CSV/JSON export - Citation graph visualization (D3.js, 440 nodes, 2422 edges) - Standards readiness scoring (0-100 composite from 6 factors) - Side-by-side draft comparison view with shared/unique analysis - Annotation system (notes + tags per draft, DB-persisted) - Docker deployment (Dockerfile + docker-compose with Ollama) - Scheduled updates (cron script with log rotation) - Pipeline health dashboard (stage progress bars, cost tracking) - Test suite foundation (54 pytest tests covering DB, models, web data) Fixes: compare_drafts() stubbed→working, get_authors_for_draft() bug, source-aware analysis prompts, config env var overrides + validation, resilient batch error handling with --retry-failed, observatory --dry-run Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
182
data/reports/platform-improvement-plan.md
Normal file
182
data/reports/platform-improvement-plan.md
Normal file
@@ -0,0 +1,182 @@
|
||||
# Platform Improvement Plan: IETF Draft Analyzer → Standards Intelligence Platform
|
||||
|
||||
*Generated 2026-03-07 — Based on full codebase audit and architectural analysis*
|
||||
|
||||
---
|
||||
|
||||
## Current State Summary
|
||||
|
||||
| Dimension | What Exists | Assessment |
|
||||
|-----------|-------------|------------|
|
||||
| **Data** | 434 drafts, 557 authors, 419 ideas, 11 gaps, 4231 refs to 694 RFCs | Strong foundation |
|
||||
| **CLI** | 20+ commands (fetch, analyze, embed, ideas, gaps, report, viz, wg, etc.) | Feature-rich |
|
||||
| **Web UI** | 16 Flask pages with D3.js/Plotly visualizations | Good but disconnected |
|
||||
| **Pipeline** | Observatory class with multi-source support | Built but manual |
|
||||
| **Multi-SDO** | IETF complete, W3C fetcher written but unused | Partially built |
|
||||
| **Tests** | Zero tests | Critical gap |
|
||||
| **Deployment** | Manual Python setup | No containerization |
|
||||
|
||||
## The Transformation
|
||||
|
||||
**From**: A powerful CLI analysis tool that an expert runs manually
|
||||
**To**: A living intelligence platform that monitors, alerts, and answers questions
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Quick Wins (All Parallelizable)
|
||||
|
||||
### 1.1 Global Search [S: 3-5h]
|
||||
Add a unified search bar to the web UI that queries across drafts (FTS5), ideas, authors, and gaps simultaneously. Currently search is isolated to the drafts page.
|
||||
|
||||
- **Files**: `src/webui/app.py` (new `/search` route), `src/webui/data.py` (new `global_search()`), `src/webui/templates/base.html` (search in sidebar), new `search_results.html`
|
||||
- **Why first**: Most common user action on any data platform
|
||||
|
||||
### 1.2 REST API [S: 2-3h]
|
||||
Expose all existing `data.py` functions as JSON endpoints. Only 3 API endpoints exist today — extend to all 15+ data views.
|
||||
|
||||
- **Files**: `src/webui/app.py` (add `/api/ideas`, `/api/gaps`, `/api/ratings`, `/api/timeline`, `/api/landscape`, `/api/similarity`, `/api/drafts/<name>`, etc.)
|
||||
- **Why**: Enables programmatic access, third-party tools, decouples data from presentation
|
||||
|
||||
### 1.3 Export (CSV/JSON) [S: 3-4h]
|
||||
Add export buttons to web UI pages + `ietf export` CLI command.
|
||||
|
||||
- **Files**: `src/ietf_analyzer/cli.py` (new `export` command), API endpoints get `?format=csv` support
|
||||
- **Depends on**: 1.2
|
||||
|
||||
### 1.4 Annotation System [M: 4-6h]
|
||||
Add private notes and custom tags per draft, persisted in DB.
|
||||
|
||||
- **Files**: `src/ietf_analyzer/db.py` (new `annotations` table), `src/webui/app.py` (POST endpoint), `src/webui/templates/draft_detail.html` (inline edit), `src/ietf_analyzer/cli.py` (new `annotate` command)
|
||||
- **Why**: Analysts need to layer their own context onto the data
|
||||
|
||||
### 1.5 Test Suite Foundation [M: 6-8h]
|
||||
Create pytest infrastructure with ~30 tests covering DB layer, models, and web data functions.
|
||||
|
||||
- **Files**: new `tests/conftest.py`, `tests/test_db.py`, `tests/test_models.py`, `tests/test_data.py`, update `pyproject.toml`
|
||||
- **Why**: Zero tests today. Every future change benefits from a safety net
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Core Platform
|
||||
|
||||
### 2.1 Semantic Search / "Ask" [M: 1-2 days] — THE SIGNATURE FEATURE
|
||||
Natural language queries: "Which drafts address agent authentication?" → synthesized answer with citations.
|
||||
|
||||
- Embed the query via Ollama, compute cosine similarity against all 434 draft embeddings
|
||||
- Merge with FTS5 keyword results using reciprocal rank fusion
|
||||
- Optionally synthesize answer via Claude with top-K context
|
||||
- **Files**: new `src/ietf_analyzer/search.py`, `src/ietf_analyzer/cli.py` (`ietf ask`), `src/webui/app.py` (`/ask` route), new template
|
||||
|
||||
### 2.2 Competitive Landscape Mapping [M: 1-2 days]
|
||||
Auto-detect and compare competing proposals in the same problem space.
|
||||
|
||||
- Group drafts by similarity clusters, enrich with rating comparisons and WG adoption status
|
||||
- Show head-to-head comparisons: where they agree, where they diverge
|
||||
- **Files**: new `src/ietf_analyzer/competition.py`, `src/webui/app.py` (`/competition` route), new template
|
||||
|
||||
### 2.3 Standards Readiness Scoring [M: 1 day]
|
||||
Composite 0-100 "readiness" score: WG adoption, revision count, reference density, cited-by count, author track record.
|
||||
|
||||
- **Files**: new `src/ietf_analyzer/readiness.py`, update `src/webui/templates/draft_detail.html` (gauge chart), update drafts listing
|
||||
|
||||
### 2.4 Scheduled Updates + Pipeline Health [M: 1 day]
|
||||
Cron-based auto-fetch using existing Observatory, plus monitoring dashboard.
|
||||
|
||||
- **Files**: new `scripts/scheduled-update.sh`, enhance `src/webui/templates/monitor.html` (stage breakdown, cost tracking, failure log)
|
||||
|
||||
### 2.5 Comparison View [M: 1 day]
|
||||
Side-by-side comparison of 2+ drafts with rating radar overlay, shared/unique ideas, shared/unique references.
|
||||
|
||||
- **Files**: `src/webui/app.py` (`/compare?drafts=...`), `src/webui/data.py` (`get_comparison_data()`), new template, add checkboxes to drafts listing
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Intelligence
|
||||
|
||||
### 3.1 Trend Forecasting [L: 2-3 days]
|
||||
Predict which areas will grow based on submission velocity, revision activity, WG adoption signals.
|
||||
|
||||
- Per-category momentum signals, linear/exponential extrapolation, "Hot/Cooling/Emerging" tags
|
||||
- **Files**: new `src/ietf_analyzer/forecasting.py`, new web page `/trends`
|
||||
|
||||
### 3.2 Change Detection & Diffing [L: 2-3 days]
|
||||
Track what changed between draft revisions, summarize changes via Claude.
|
||||
|
||||
- New `draft_revisions` table to archive old versions
|
||||
- Section-level diff with Claude-generated change summaries
|
||||
- **Files**: new `src/ietf_analyzer/diff.py`, update `src/ietf_analyzer/db.py`, `src/ietf_analyzer/fetcher.py`
|
||||
- **Depends on**: 2.4 (scheduled updates to catch new revisions)
|
||||
|
||||
### 3.3 Citation Graph [M: 1-2 days]
|
||||
Visual dependency tree from the 4231 existing cross-references to 694 RFCs.
|
||||
|
||||
- D3.js force-directed graph (pattern from authors.html), PageRank-style influence scores
|
||||
- **Files**: `src/webui/data.py` (`get_citation_graph()`), new template `citations.html`
|
||||
|
||||
### 3.4 Newsletter Generation [M: 1-2 days]
|
||||
Automated weekly/monthly digest with new drafts, significant changes, trend shifts.
|
||||
|
||||
- **Files**: new `src/ietf_analyzer/newsletter.py`, `src/ietf_analyzer/cli.py` (`ietf newsletter`)
|
||||
- **Depends on**: 3.2 (for change content)
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Scale
|
||||
|
||||
### 4.1 Docker Deployment [S: 3-4h]
|
||||
Dockerfile + docker-compose.yml with Flask app + Ollama.
|
||||
|
||||
### 4.2 Complete W3C Integration [M: 1 day]
|
||||
Wire existing W3C fetcher into Observatory pipeline, make prompts source-aware, add source filter to web UI.
|
||||
|
||||
### 4.3 IEEE + 3GPP Sources [L: 3-5 days]
|
||||
New source fetchers following `SourceFetcher` protocol. Depends on 4.2 validating the pipeline.
|
||||
|
||||
### 4.4 Cross-SDO Analysis [L: 2-3 days]
|
||||
Compare work across standards bodies. Embedding similarity between IETF/W3C/IEEE specs, gap analysis for topics only one body covers.
|
||||
|
||||
### 4.5 Plugin Architecture [L: 2-3 days]
|
||||
Formalize extension points for sources, analyzers, report types via Python entry points.
|
||||
|
||||
---
|
||||
|
||||
## Recommended Agent Team Assignments
|
||||
|
||||
### Immediate Sprint (Phase 1 — all in parallel)
|
||||
|
||||
| Agent | Items | Focus |
|
||||
|-------|-------|-------|
|
||||
| **Coder A** | 1.1 + 1.2 | Global search + REST API |
|
||||
| **Coder B** | 1.4 + 1.3 | Annotations + Export |
|
||||
| **Coder C** | 1.5 | Test suite foundation |
|
||||
|
||||
### Next Sprint (Phase 2 — mostly parallel)
|
||||
|
||||
| Agent | Items | Focus |
|
||||
|-------|-------|-------|
|
||||
| **Coder A** | 2.1 | Semantic search / Ask (signature feature) |
|
||||
| **Coder B** | 2.2 + 2.5 | Competition mapping + Comparison view |
|
||||
| **Coder C** | 2.3 + 2.4 | Readiness scoring + Pipeline health |
|
||||
|
||||
---
|
||||
|
||||
## Impact vs Effort Matrix
|
||||
|
||||
```
|
||||
HIGH IMPACT
|
||||
|
|
||||
| 2.1 Ask 3.2 Diffing
|
||||
| 1.1 Search 3.1 Forecasting
|
||||
| 2.2 Competition 4.4 Cross-SDO
|
||||
| 2.5 Compare
|
||||
| 1.2 API
|
||||
| 2.3 Readiness 3.3 Citations
|
||||
| 1.4 Annotations 3.4 Newsletter
|
||||
| 1.3 Export 4.2 W3C
|
||||
| 1.5 Tests 4.1 Docker
|
||||
| 4.5 Plugins
|
||||
| 4.3 IEEE/3GPP
|
||||
LOW IMPACT
|
||||
+------------------------------------------
|
||||
LOW EFFORT HIGH EFFORT
|
||||
```
|
||||
Reference in New Issue
Block a user