c/ietf-draft-analyzer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
ietf-draft-analyzer/FEATURE_BACKLOG.md
Christian Nennemann 61cdab16b9 fix: dev mode auth regression from blueprint refactor 
The _initialized singleton in auth.py prevented hooks from registering
on the correct app instance when create_app() was called twice (once
eagerly at import, once from __main__). Removed the guard and made
the module-level app lazy. Also adds feature backlog and architecture
assessment from the review team.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 03:52:02 +01:00

17 KiB
Raw Permalink Blame History

IETF Draft Analyzer — Feature & UX Backlog

Scope: Analysis of web UI (33 templates, 66 routes), CLI (36 commands), data layer (100+ computed functions), and DB schema (10 core tables).

Date: 2026-03-09 Analyst: UX Scout Task #3

Summary

The application is feature-rich but siloed: extensive data exists in the DB and is computed in data.py, but only ~60% is exposed in the web UI. Cross-linking between pages is incomplete (e.g., author pages don't link to gap gaps, drafts don't link to related proposals). CLI has powerful analysis commands (embed-ideas, idea-overlap, co-occurrence) but no web UI counterparts.

Quick wins: 5 features that reuse existing infrastructure with minimal effort. Medium effort: 7 features requiring new pages or moderate refactoring. Design debt: Navigation flows have dead ends (e.g., single idea clusters → nowhere, gaps → proposals missing web UI).

Detailed Backlog

🟢 HIGH IMPACT × SMALL EFFORT

1. Author detail page with draft linkage [IMPACT: HIGH | EFFORT: SMALL]

  • Current: Authors page shows network graph only; clicking author in draft_detail searches "author name" in drafts
  • Missing: Individual author page (/authors/<author_id>) showing:
    • Author metadata (affiliation, co-authors)
    • List of drafts authored (with ratings/scores)
    • Co-author graph (2-hop collaboration network)
    • Ideas contributed (via their drafts)
    • Gaps they could address (category match with drafts)
  • Why: Closes gap between draft_detail → author → their other work. Reuses existing get_author_network_full() and co-author logic.
  • Effort: New template + 2 route handlers + 1 data.py function to fetch author profile + co-authors.
  • Reuses: authors.py, get_coauthor_network(), existing draft query logic.

2. Idea detail page with "related ideas" clustering [IMPACT: HIGH | EFFORT: SMALL]

  • Current: Idea clusters shown as page, but no single-idea view; clicking idea in draft_detail has no action
  • Missing: /ideas/<idea_id> page showing:
    • Idea title, type, description, novelty score
    • Source draft (link)
    • 5–10 semantically similar ideas (from other drafts)
    • Ideas this relates to (same cluster)
    • Gaps this could address (keyword + semantic match)
  • Why: Idea-centric navigation. Currently all idea access is through drafts or clusters.
  • Effort: New template + route + 1 data.py function (k-nearest neighbors in idea embedding space).
  • Reuses: Idea embeddings already computed; get_idea_clusters() logic.

3. Gap-to-draft reverse link [IMPACT: HIGH | EFFORT: SMALL]

  • Current: Gap detail shows what drafts exist on that topic; no link back to gaps from draft_detail
  • Missing: Add section to draft_detail showing:
    • Which gaps this draft helps address (topic keyword match + semantic)
    • Related proposals (from /proposals system)
  • Why: Closes loop: gap → drafts and draft → gaps. Helps user understand how a draft fits into the roadmap.
  • Effort: New section in draft_detail template + 1 data.py function (get_gaps_for_draft(draft_name)).
  • Reuses: Existing gap data + keyword/semantic matching.

4. Inline draft comparison widget in drafts list [IMPACT: MEDIUM | EFFORT: SMALL]

  • Current: /compare page exists but requires manual draft selection via checkboxes, no shortcuts
  • Missing:
    • Add "Compare" checkbox to each draft row in /drafts list
    • Sticky "Compare Selected" button at bottom
    • Quick-compare overlay showing side-by-side: score, category, authors, novelty
  • Why: Low-friction way to discover overlaps without navigating away.
  • Effort: Template changes + 1 JS handler + reuse existing /api/compare endpoint.

5. Export audit trail: "What changed?" in monitoring [IMPACT: MEDIUM | EFFORT: SMALL]

  • Current: Monitor page shows last run stats (new drafts, duration) but no detailed change log
  • Missing: Expandable section per run showing:
    • New drafts added (names, count)
    • False positives removed
    • Ideas extracted (by category)
    • API costs (tokens used)
  • Why: Transparency into pipeline. Supports cost tracking and debugging.
  • Effort: New template section + 1 data.py function to fetch run details from monitoring tables.
  • Reuses: Existing monitoring data; minor DB schema extension (one audit_log table).

🟡 HIGH IMPACT × MEDIUM EFFORT

6. Category detail pages [IMPACT: HIGH | EFFORT: MEDIUM]

  • Current: Category counts shown in overview; drafts filterable by category in /drafts?cat=X; no dedicated page
  • Missing: /categories/<category> page showing:
    • Category description, count, average score
    • All drafts in category (sorted by score)
    • Top ideas in category (clustered)
    • Top authors in category (by draft count)
    • Trends: are drafts in this category getting more diverse? Maturing faster?
    • Gaps specific to this category (e.g., "Authentication" category lacks "revocation" ideas)
  • Why: Makes categories first-class entities; easier content discovery.
  • Effort: 1 template + 1 route + 3 data.py functions (top ideas, top authors, trends).
  • Reuses: Existing category logic, idea clustering, author ranking.

7. Proposal detail → drafts linkage (web UI) [IMPACT: HIGH | EFFORT: MEDIUM]

  • Current: /proposals exists (admin-only) but is disconnected from public UI; no way to see "which drafts might form a proposal?"
  • Missing:
    • Make /proposals public (read-only)
    • Add /proposals/<slug> route showing:
      • Proposal title, status, description, intended WG
      • Linked gaps (current feature)
      • Suggested/related drafts (category + author match)
      • Links to existing or draft RFCs/BCP refs
    • Add "Proposal intake" link from gap detail: "This gap could become a proposal. Start here →"
  • Why: Connects gap analysis to standards development workflow.
  • Effort: Route changes + template + 2 data.py functions (related drafts, related RFCs).
  • Requires: Remove admin_required on /proposals routes (review auth first).

8. Citation/reference graph improvements [IMPACT: MEDIUM | EFFORT: MEDIUM]

  • Current: Citation page shows RFC/draft references as graph
  • Missing:
    • Reverse citations: "Which other drafts cite this one?"
    • Citation patterns: "What RFC/BCP are most cited in agent drafts?"
    • Missing references: "This draft should cite RFC X because it covers the same topic" (suggest related RFCs)
  • Why: Helps identify foundational standards and co-dependencies.
  • Effort: 2 new data functions + template enhancements.
  • Reuses: Existing draft_refs table; parsing logic for RFC extraction.

9. Dashboard export to Obsidian/Markdown [IMPACT: MEDIUM | EFFORT: MEDIUM]

  • Current: /export/obsidian exists but exports only notes, not full dashboard state
  • Missing: Export multiple formats:
    • Markdown vault: Obsidian-compatible markdown with frontmatter (category, score, authors, ideas)
    • JSON snapshot: Full state dump (drafts + ratings + ideas + gaps) for offline analysis
    • CSV multi-export: Separate CSVs for each entity type (drafts, authors, ideas, gaps, proposals)
  • Why: Supports research workflows (offline reading, external tool integration).
  • Effort: 2 new route handlers + 2 data.py functions (schema converters).
  • Reuses: Existing obsidian_export.py; CSV export logic already in place.

10. Search filters: advanced query builder [IMPACT: MEDIUM | EFFORT: MEDIUM]

  • Current: Search page shows simple text box and keyword search; /drafts has basic filters (category, score, source)
  • Missing: Advanced query UI:
    • Filter by: category, score range, publication date, author, WG, false positive status
    • Sort by: score, novelty, overlap, momentum, relevance, date
    • Save/name favorite searches
    • Full-text search highlighting in results
  • Why: Power users (researchers, standards bodies) need fine-grained filtering.
  • Effort: New template (filter form) + route enhancements + 1 data.py function (advanced search builder).
  • Reuses: Existing FTS5 search; current filter parameters.

🔵 MEDIUM IMPACT × MEDIUM EFFORT

11. Local LLM (Ollama) analysis shortcuts [IMPACT: MEDIUM | EFFORT: MEDIUM]

  • Current: Ollama used for embeddings + classification only; Claude handles all analysis
  • Missing: Expose local-only analysis tools in web UI:
    • Idea re-classification: "Reclassify ideas for this draft using local model" (cheaper)
    • Draft similarity (local embeddings): "Find drafts most similar to this one"
    • Batch idea extraction preview: "Show how ideas would be extracted from this draft (preview, no API call)"
  • Why: Reduces Claude API costs; good for data exploration before expensive operations.
  • Effort: 3 new API routes + Flask handlers.
  • Reuses: Existing embeddings.py + classifier.py + Ollama integration.

12. Idea overlap/gap matrix UI [IMPACT: MEDIUM | EFFORT: MEDIUM]

  • Current: CLI has idea-overlap and co-occurrence commands (generate reports); no web UI
  • Missing: Interactive web page at /ideas/overlap:
    • Matrix showing which ideas overlap (appear in multiple drafts)
    • Co-occurrence frequency (if ideas A and B frequently appear together, high signal)
    • Suggests "merge these ideas" or "these might be the same standard"
  • Why: Helps identify redundant/overlapping work; supports standards consolidation.
  • Effort: 1 new route + 1 data.py function + 1 Plotly heatmap template.
  • Reuses: CLI logic from idea-overlap and co-occurrence commands.

13. Trends analysis page [IMPACT: MEDIUM | EFFORT: MEDIUM]

  • Current: get_trends_data() computed but not exposed (used by complexity page); data.py has trends functions
  • Missing: Dedicated /trends page showing:
    • New drafts per month (trend line)
    • Score distribution over time (are recent drafts more mature?)
    • Category growth trends
    • Author activity (new vs. returning)
    • API cost trend (tokens/month)
  • Why: Understand field maturity and adoption patterns.
  • Effort: 1 route + 1 template + use existing get_trends_data().
  • Reuses: Existing data; Plotly timeseries.

14. Readiness scorer exposé [IMPACT: MEDIUM | EFFORT: MEDIUM]

  • Current: readiness module exists; shown in CLI show command; not in web UI
  • Missing:
    • Add readiness score to draft_detail page (already computed)
    • /readiness page: heatmap of all drafts × readiness factors
    • "How close to standardization?" indicator
  • Why: Helps standards bodies identify which proposals are most mature.
  • Effort: 1 template update + 1 route + reuse existing readiness module.

15. Geospatial author view [IMPACT: LOW | EFFORT: MEDIUM]

  • Current: Author affiliations stored in DB (org) but not geospatial
  • Missing: If org data can be geo-coded (rough):
    • Map showing author distribution (bubbles by region/country)
    • "Where is AI/agent standardization happening?"
  • Why: Novel visualization; supports diversity analysis.
  • Effort: Requires org → geo mapping (manual or external API); map widget setup.
  • Blocker: Geo-coding affiliation strings is non-trivial.

🟠 MEDIUM/LOW IMPACT × LARGE EFFORT

16. Real-time monitoring dashboard [IMPACT: MEDIUM | EFFORT: LARGE]

  • Current: Monitor page is static; shows last run results
  • Missing: WebSocket-based live updates during fetch/analyze/embed pipeline runs
    • Real-time progress bars
    • Live log streaming
    • Ability to pause/cancel runs
  • Why: Better UX for long-running operations.
  • Effort: WebSocket layer (Flask-SocketIO), background task queue (Celery), run state machine.
  • Note: High complexity; consider only if pipeline runs are frequent bottleneck.

17. Semantic search (vs. full-text) [IMPACT: MEDIUM | EFFORT: LARGE]

  • Current: Search uses FTS5 (lexical); embeddings exist but not used for search
  • Missing:
    • "Ask" page queries: should embed question, find semantically similar drafts
    • Extend to all search endpoints (global search)
  • Why: Better recall for conceptual queries ("How do drafts handle agent authorization?")
  • Effort: Requires re-querying embeddings for every search (perf issue) or building vector index (Faiss/Milvus).
  • Note: Good for future; current FTS5 is sufficient.

18. Collaborative annotations & tags [IMPACT: LOW | EFFORT: LARGE]

  • Current: Admin-only notes & tags on draft_detail
  • Missing:
    • Multi-user collaborative notes (comments, threads)
    • Tag suggestions (based on similar drafts)
    • Tag autocomplete
  • Why: Supports community review of drafts.
  • Effort: Auth rework, comments table, collaborative editing.
  • Note: Out of scope unless multi-user access is planned.

19. Custom report builder [IMPACT: MEDIUM | EFFORT: LARGE]

  • Current: Fixed reports (overview, landscape, digest, gaps, etc.)
  • Missing: UI to build custom reports:
    • Select entities (drafts, ideas, authors, gaps)
    • Choose dimensions (score, category, date)
    • Choose output format (PDF, HTML, Markdown)
  • Why: Non-technical users (standards committees) can create reports without CLI.
  • Effort: Report template DSL, PDF generation (weasyprint), export logic.

20. Draft family/lineage tracking [IMPACT: MEDIUM | EFFORT: LARGE]

  • Current: pipeline/family.py exists; tracks draft families (related drafts across revisions/forks)
  • Missing: Web UI showing draft lineage:
    • "This draft evolved from draft-X, also related to draft-Y"
    • Family tree visualization
    • Automatic linking of superseding drafts
  • Why: Helps track standardization progress (e.g., "agent-framework-v1 → v2 → RFC").
  • Effort: Draft relationship inferencing, visualization.
  • Note: Requires careful matching logic; already partially implemented in family.py.

Data Utilization Gaps

Functions computed but not exposed:

  • get_trends_data() → only in complexity page
  • get_complexity_data() → complexity page only
  • get_citation_influence() → citations page, could be more prominent
  • get_bcp_analysis() → no dedicated page
  • get_false_positive_profile() → part of monitor, could be own page
  • get_source_comparison() → no dedicated page (only API)

Quick fix: Create /analytics/trends and /analytics/sources pages to expose these.

Navigation & Cross-Linking Issues

Page Outbound links Gaps
Draft detail Gap, Author search, Idea clusters No: "I'm an author, show all my drafts" link; "Reverse gaps" (gaps this draft addresses)
Author network Co-authors, drafts (search) No: Author detail page; no link to gaps this author could address
Gap detail Related drafts No: Reverse link to draft; no "start proposal" CTA
Idea cluster Drafts (list) No: Individual idea page; no link to related gaps
Proposal (admin) Gaps No: Public proposal discovery; no related drafts; no standards WG link
/ask Search results No: Persistent query history; no "save search"

Low-Hanging Fruit (Quick Wins)

  1. ✨ Add "Source draft" link to idea_detail page (if page created) — 15 min
  2. ✨ Add readiness score to draft_detail — 10 min (already computed)
  3. ✨ Show proposals on gap_detail (reverse link) — 20 min
  4. ✨ Author search in draft_detail → author detail page (if page created) — 30 min
  5. ✨ Export monitor audit log to CSV — 15 min

Recommended Priority

Phase 1 (Week 1–2): Navigation foundation

  1. Author detail page
  2. Idea detail page
  3. Gap-to-draft reverse links
  4. Make proposals public + improve detail page

Phase 2 (Week 3–4): Discoverability

  1. Category detail pages
  2. Search filters (advanced query)
  3. Trends/analytics pages

Phase 3 (Optional): Depth

  1. Idea overlap matrix
  2. Citation improvements
  3. Obsidian/JSON export

Architecture Notes

  • DB schema ready: All needed relationships exist (draft_refs, draft_authors, ideas, gaps, generated_drafts)
  • Data layer mature: Most data functions exist in data.py; just need exposure
  • Template patterns consistent: Easy to add new pages following existing conventions (Tailwind + Plotly)
  • CLI-to-web asymmetry: Some powerful CLI commands (idea-overlap, draft-gen) have no web UI
  • Performance: Current app uses ~10–15 queries per page load; acceptable for 361 drafts. No N+1 issues observed; caching in place via _cached() with 5-min TTL.

Cost/Effort Estimates

  • Small: ~15–30 min, no new DB tables, reuses existing functions
  • Medium: 1–3 hours, possible minor DB updates, new routes + templates
  • Large: 4+ hours, significant refactoring or new infrastructure (WebSockets, embeddings, auth)

Report compiled by: UX Scout (Task #3) Date: 2026-03-09

Reference in New Issue View Git Blame Copy Permalink