From ed821097de4fb17d527c71f357fb504492a63176 Mon Sep 17 00:00:00 2001 From: Christian Nennemann Date: Thu, 2 Apr 2026 18:32:18 +0000 Subject: [PATCH] feat: add 3-Sets agent diagnostic and attention filters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New skill: agent-diagnostic — applies the 3-Sets framework (Tool-Set, Skill-Set, Mind-Set) to agent orchestration: - Pre-orchestration diagnostic: check each agent's configuration across three dimensions, fix the weakest set first - Chain principle: weakest set caps output (Opus + bad prompt = waste) - Alignment principle: modest aligned agents beat excellent misaligned ones - Attention filters: each archetype reads only relevant artifacts - Post-orchestration learning: extract learnings to persistent memory structured by the three sets Based on the 3-Sets Method diagnostic framework. --- README.md | 19 +++++ skills/agent-diagnostic/SKILL.md | 136 +++++++++++++++++++++++++++++++ skills/using-archeflow/SKILL.md | 1 + 3 files changed, 156 insertions(+) create mode 100644 skills/agent-diagnostic/SKILL.md diff --git a/README.md b/README.md index 8f93d79..42345da 100644 --- a/README.md +++ b/README.md @@ -88,6 +88,7 @@ archeflow/ │ ├── do-phase/ # Maker implementation rules │ ├── check-phase/ # Reviewer protocols (all 4) │ ├── shadow-detection/ # Recognizing and correcting dysfunction +│ ├── agent-diagnostic/ # 3-Sets analysis for agent configuration │ ├── autonomous-mode/ # Unattended overnight sessions │ ├── custom-archetypes/ # Creating domain-specific roles │ └── workflow-design/ # Designing custom workflows @@ -143,6 +144,22 @@ pdca: act: { exit_when: all_approved, max_cycles: 2 } ``` +## Agent Diagnostic — The 3-Sets Lens + +Before orchestrating, ArcheFlow diagnoses each agent across three dimensions: + +| Set | Question | Fix | +|-----|----------|-----| +| **Tool-Set** | Does the agent have the right capabilities? | Add missing tools, remove noisy ones | +| **Skill-Set** | Is the model tier matched to the task? | Adjust model, don't compensate with tools | +| **Mind-Set** | Is the archetype prompt focused and aligned? | Sharpen prompt, don't compensate with model | + +**The chain principle:** The weakest set caps the output. An Opus model with a vague prompt wastes money. A Haiku with a focused archetype and the right tools outperforms it at 1/50th the cost. + +**The alignment principle:** Three modestly configured agents that are aligned outperform three individually excellent but misaligned agents. + +Based on the [3-Sets Method](https://git.xorwell.de/chris/workspace) — an integrative diagnostic framework for Tool-Set, Skill-Set, and Mind-Set alignment. + ## Philosophy ArcheFlow is built on three beliefs: @@ -153,6 +170,8 @@ ArcheFlow is built on three beliefs: 3. **Autonomy needs structure.** Agents left to their own devices produce mediocre results. Agents given clear roles, typed communication, and quality gates produce exceptional work — even overnight, even unattended. +4. **Fix the weakest set, not the strongest.** Don't upgrade the model when the problem is a bad prompt. Don't add tools when the problem is wrong model tier. Diagnose first, then invest where it matters. + ## License MIT diff --git a/skills/agent-diagnostic/SKILL.md b/skills/agent-diagnostic/SKILL.md new file mode 100644 index 0000000..98e64b0 --- /dev/null +++ b/skills/agent-diagnostic/SKILL.md @@ -0,0 +1,136 @@ +--- +name: agent-diagnostic +description: Use before orchestration to diagnose agent configuration, and after orchestration to extract learnings. Applies the 3-Sets diagnostic (Tool-Set, Skill-Set, Mind-Set) to optimize agent alignment. +--- + +# Agent Diagnostic — 3-Sets Analysis + +Before spawning agents, diagnose their configuration across three dimensions. The weakest dimension caps the agent's output. Alignment across dimensions matters more than excellence in any single one. + +## The Three Sets + +| Set | What It Is | Agent Equivalent | +|-----|-----------|-----------------| +| **Tool-Set** | What the agent can access | File read/write, git, bash, MCP servers | +| **Skill-Set** | What the agent's model can do | Haiku (fast/cheap), Sonnet (balanced), Opus (deep reasoning) | +| **Mind-Set** | How the agent approaches the task | Archetype definition, system prompt focus | + +## Pre-Orchestration Diagnostic + +Before each orchestration, run a quick check per agent: + +### Tool-Set Check +- Does the agent have the tools it needs for its role? +- Explorer needs: file read, grep, git log — NOT file write +- Maker needs: file read/write, git, bash, test runner +- Guardian needs: file read, git diff — NOT file write +- Does the agent have tools it DOESN'T need? Remove them. Excess tools create noise and distraction. + +**Bottleneck signal:** Agent can't perform its core task due to missing capability. +**Fix:** Add the missing tool. Don't upgrade the model — it won't compensate. + +### Skill-Set Check +- Is the model tier matched to the cognitive demand? +- Research, filtering, pattern matching → Haiku (cheap, fast) +- Design, code generation, structured review → Sonnet (balanced) +- Holistic judgment, complex trade-offs, architecture → Opus (deep, expensive) + +| Archetype | Default Tier | Why | +|-----------|-------------|-----| +| Explorer | Haiku | Pattern matching and synthesis — breadth over depth | +| Creator | Sonnet | Design requires reasoning but not deep judgment | +| Maker | Sonnet | Code generation is Sonnet's sweet spot | +| Guardian | Sonnet | Security review needs structured reasoning | +| Skeptic | Sonnet | Assumption challenging needs analytical depth | +| Trickster | Haiku | Edge case generation is fast, creative work | +| Sage | Sonnet | Quality review needs good judgment; Opus only for large changes | + +**Bottleneck signal:** Agent produces shallow output on complex tasks, or expensive model on simple tasks. +**Fix:** Adjust model tier. Don't add more tools — they won't compensate for reasoning limits. + +### Mind-Set Check +- Is the archetype prompt focused on the right concern? +- Does the prompt contain contradictions? ("Be thorough" + "Be fast") +- Is the shadow definition specific enough to be detectable? +- Is the prompt appropriately sized? (Under 500 words — longer prompts dilute focus) + +**Bottleneck signal:** Agent produces generic output, misses its archetype's core concern, or falls into shadow immediately. +**Fix:** Sharpen the prompt. Don't upgrade the model — a vague prompt stays vague on any model. + +## The Chain Principle + +The weakest set determines the result: + +``` +Tool-Set: 90 Skill-Set: 90 Mind-Set: 30 → Output: ~30 +``` + +An Opus model (Skill-Set: 100) with a vague prompt (Mind-Set: 30) wastes money. A Haiku model (Skill-Set: 60) with a perfectly focused archetype (Mind-Set: 90) and the right tools (Tool-Set: 80) produces better results at 1/50th the cost. + +**Always fix the weakest set first.** + +## The Alignment Principle + +Three agents with modest but aligned configurations outperform three individually excellent but misaligned agents. + +Signs of misalignment: +- Explorer researches topics the Creator doesn't use in the proposal (Mind-Set mismatch) +- Maker has tools the proposal doesn't reference (Tool-Set excess) +- Guardian reviews at threat level inappropriate to the context (Mind-Set miscalibration) +- Expensive model on a task that doesn't need it (Skill-Set waste) + +## Post-Orchestration Learning + +After each orchestration, extract learnings to `.archeflow/memory/`: + +### What to Record + +**Tool-Set learnings:** +- "This project uses pnpm, not npm" → future Makers know +- "The test runner is vitest, not jest" → future Makers and Sages know +- "No database access in CI" → future Guardians adjust threat model + +**Skill-Set learnings:** +- "Complex type inference in this codebase requires Sonnet minimum" → future routing +- "Haiku was sufficient for all Check phase reviews in this project" → cost savings + +**Mind-Set learnings:** +- "Guardian was paranoid on auth module — auth tests are comprehensive, calibrate to normal risk" → future calibration +- "Explorer rabbit-holed in the monorepo — add 10-file cap for this codebase" → future shadow tuning + +### Memory Format + +Write to `.archeflow/memory/.md`: + +```markdown +## Tool-Set +- Package manager: pnpm (not npm) +- Test runner: vitest +- CI: GitHub Actions, no DB access in CI + +## Skill-Set +- Type-heavy modules need Sonnet minimum +- Standard CRUD routes work fine with Haiku review + +## Mind-Set +- Auth module: well-tested, normal risk level (don't over-guard) +- Payment module: no tests, elevated risk (Guardian should be thorough) +``` + +Keep entries factual and specific. No opinions, no predictions. Update after each orchestration — don't append endlessly, revise what changed. + +## Attention Filters + +Each archetype reads only what's relevant from shared context: + +| Archetype | Reads | Ignores | +|-----------|-------|---------| +| Explorer | Task description, codebase | Prior proposals | +| Creator | Explorer's research, task description | Implementation details | +| Maker | Creator's proposal | Explorer's research, reviews | +| Guardian | Maker's git diff, proposal's risk section | Explorer's research | +| Skeptic | Creator's proposal (assumptions) | Git diff details | +| Trickster | Maker's git diff only | Everything else | +| Sage | Proposal + implementation + diff | Explorer's raw research | + +When spawning agents, pass only the relevant artifacts — not everything. This reduces context window waste and sharpens focus. diff --git a/skills/using-archeflow/SKILL.md b/skills/using-archeflow/SKILL.md index 144f716..1a1c0cc 100644 --- a/skills/using-archeflow/SKILL.md +++ b/skills/using-archeflow/SKILL.md @@ -46,5 +46,6 @@ Act → All approved? Merge. Issues? Cycle back to Plan. - **archeflow:orchestration** — Step-by-step execution guide - **archeflow:plan-phase** / **do-phase** / **check-phase** — Phase protocols - **archeflow:shadow-detection** — Recognizing dysfunction +- **archeflow:agent-diagnostic** — 3-Sets analysis (Tool-Set, Skill-Set, Mind-Set) for agent configuration - **archeflow:autonomous-mode** — Unattended sessions - **archeflow:custom-archetypes** / **workflow-design** — Extending ArcheFlow