From af1f4e7da72e264ad181d9b95127cfadcc87f51d Mon Sep 17 00:00:00 2001 From: Christian Nennemann Date: Mon, 6 Apr 2026 20:41:20 +0200 Subject: [PATCH 1/2] refactor: merge attention-filters into check-phase skill Consolidate the attention-filters skill (122 lines) into check-phase, reducing check-phase from 234 to 110 lines. Removed verbose bash code blocks, 30-line consolidated output example, re-check protocol (belongs in act-phase), and motivational section. Updated all references in README, plugin.json, using-archeflow, and colette-bridge. --- .claude-plugin/plugin.json | 2 +- README.md | 2 - skills/attention-filters/SKILL.md | 121 --------------- skills/check-phase/SKILL.md | 241 ++++++++---------------------- skills/colette-bridge/SKILL.md | 2 +- skills/using-archeflow/SKILL.md | 1 - 6 files changed, 61 insertions(+), 308 deletions(-) delete mode 100644 skills/attention-filters/SKILL.md diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 2e59bd1..c4b3ee0 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -15,7 +15,7 @@ ], "skills": [ "run", "orchestration", "plan-phase", "do-phase", "check-phase", "act-phase", - "shadow-detection", "attention-filters", "convergence", "artifact-routing", + "shadow-detection", "convergence", "artifact-routing", "process-log", "memory", "effectiveness", "progress", "colette-bridge", "git-integration", "multi-project", "custom-archetypes", "workflow-design", "domains", "cost-tracking", diff --git a/README.md b/README.md index e34617c..a28836d 100644 --- a/README.md +++ b/README.md @@ -164,7 +164,6 @@ ArcheFlow ships with 24 skills organized by function. | Skill | Description | |-------|-------------| | `archeflow:shadow-detection` | Quantitative dysfunction detection and automatic correction | -| `archeflow:attention-filters` | Context optimization per archetype -- each agent gets only what it needs | | `archeflow:convergence` | Detects convergence, stalling, and oscillation in multi-cycle runs | | `archeflow:artifact-routing` | Inter-phase artifact protocol -- naming, storage, routing, archiving | @@ -358,7 +357,6 @@ archeflow/ │ ├── check-phase/ # Check protocols │ ├── act-phase/ # Act phase decision logic │ ├── shadow-detection/ # Dysfunction detection -│ ├── attention-filters/ # Context optimization │ ├── convergence/ # Cycle convergence detection │ ├── artifact-routing/ # Inter-phase artifact protocol │ ├── process-log/ # Event-sourced JSONL logging diff --git a/skills/attention-filters/SKILL.md b/skills/attention-filters/SKILL.md deleted file mode 100644 index 08e8f7e..0000000 --- a/skills/attention-filters/SKILL.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -name: attention-filters -description: Use when spawning archetype agents to decide what context each agent receives. Reduces token waste and sharpens focus by passing only relevant artifacts. ---- - -# Attention Filters - -Each archetype needs different context. Pass only what's relevant — not everything. - -| Archetype | Receives | Does NOT Receive | -|-----------|----------|-----------------| -| Explorer | Task description, codebase access | Prior proposals or reviews | -| Creator | Explorer's research + task description | Implementation details | -| Maker | Creator's proposal | Explorer's research, reviews | -| Guardian | Maker's git diff + proposal risk section | Explorer's research | -| Skeptic | Creator's proposal (focus: assumptions) | Git diff details | -| Trickster | Maker's git diff only | Everything else | -| Sage | Proposal + implementation + diff | Explorer's raw research | - -## Why This Matters - -- **Token cost:** A Guardian reading the Explorer's 2000-word research wastes ~2600 tokens on irrelevant context -- **Focus:** An agent with too much context drifts from its archetype's concern -- **Shadow prevention:** Over-loading context encourages rabbit-holing (Explorer) and scope creep (Maker) - -## In Practice - -When spawning a Check-phase agent, include only the filtered context in the prompt: - -``` -# Guardian receives: -"Review these changes: -The proposal identified these risks: -Verdict: APPROVED or REJECTED with findings." - -# NOT: -"Here is the full research, the full proposal, the full implementation, -the full git log, and everything else we have..." -``` - -## Prompt Construction Templates - -### Explorer -- **Receives:** Task description, file tree (max 200 lines), prior-cycle feedback (if cycle 2+) -- **Excludes:** Creator proposals, Maker diffs, reviewer outputs -- **Token target:** ~2000 tokens input - -### Creator -- **Receives:** Task description, Explorer research (if available), prior-cycle feedback (if cycle 2+) -- **Excludes:** Maker diffs, reviewer outputs -- **Token target:** ~3000 tokens input - -### Maker -- **Receives:** Creator's proposal (full), test strategy section, file list -- **Excludes:** Explorer research, reviewer outputs, prior-cycle feedback -- **Token target:** ~2500 tokens input - -### Guardian -- **Receives:** Maker's git diff, proposal risk section, test results -- **Excludes:** Explorer research, Creator rationale, Skeptic/Sage outputs -- **Token target:** ~2000 tokens input - -### Skeptic -- **Receives:** Creator's proposal (assumptions + architecture decision), confidence scores -- **Excludes:** Git diff details, Explorer raw research, other reviewer outputs -- **Token target:** ~1500 tokens input - -### Trickster -- **Receives:** Maker's git diff only, attack surface summary (file types + entry points) -- **Excludes:** Proposal, research, other reviewer outputs -- **Token target:** ~1500 tokens input - -### Sage -- **Receives:** Creator's proposal, Maker's implementation summary + diff, test results -- **Excludes:** Explorer raw research, other reviewer verdicts -- **Token target:** ~2500 tokens input - -## Token Budget Targets - -| Archetype | Fast | Standard | Thorough | -|-----------|------|----------|----------| -| Explorer | skip | 2000 | 3000 | -| Creator | 2000 | 3000 | 4000 | -| Maker | 2000 | 2500 | 3000 | -| Guardian | 1500 | 2000 | 2500 | -| Skeptic | skip | 1500 | 2000 | -| Trickster | skip | skip | 1500 | -| Sage | skip | 2500 | 3000 | - -"skip" means the archetype is not spawned in that workflow tier. - -## Cycle-Back Filtering - -When injecting prior-cycle feedback into cycle 2+: - -1. **Summary only** — pass the structured feedback table (issue, source, severity), not full reviewer artifacts -2. **Strip resolved items** — if a finding was marked Fixed in the Act phase, exclude it -3. **Compress context** — prior proposal diffs reduce to "What Changed" section only (not full re-proposal) -4. **Cap at 500 tokens** — if feedback exceeds this, summarize by severity (CRITICAL first, then WARNING, drop INFO) - -## Filter Verification Checklist - -Before spawning each agent, verify: - -- [ ] Prompt contains ONLY the artifacts listed in that archetype's "Receives" above -- [ ] No cross-contamination from other reviewers' outputs -- [ ] Token count is within 20% of the target for the current workflow tier -- [ ] Prior-cycle feedback (if any) is summarized, not raw -- [ ] Excluded artifacts are genuinely absent (search for keywords like file paths from excluded sources) - -## Context Isolation - -Attention filters control *what* each agent receives. Context isolation controls *how* that context is constructed — ensuring agents operate on provided facts, not ambient knowledge. - -### Rules - -1. **No session bleed.** Agents receive fresh context only — constructed from task description, artifact files, or extracted sections. They must not inherit session state, chat history, or prior agent prompts. -2. **No cross-agent contamination.** An agent receives another agent's output only if the attention filter table above explicitly allows it. Guardian does not see Skeptic's output. Skeptic does not see the Maker's diff. Violations produce unreliable reviews. -3. **Controller-constructed only.** All agent context is assembled by the orchestrator from: (a) the task description, (b) artifact files on disk, or (c) extracted sections of those artifacts. Agents never pull their own context. -4. **No ambient knowledge.** Agents cannot "remember" findings from prior phases or cycles unless that information is explicitly injected via the cycle-back filtering protocol above. An agent that references information not in its prompt is hallucinating. -5. **Verification.** Before spawning each agent, confirm the constructed prompt has zero references to other agents' raw outputs that are not in the "Receives" column. Search for file paths, archetype names, and finding descriptions from excluded sources. diff --git a/skills/check-phase/SKILL.md b/skills/check-phase/SKILL.md index e4e7fd4..b4e98c1 100644 --- a/skills/check-phase/SKILL.md +++ b/skills/check-phase/SKILL.md @@ -1,233 +1,110 @@ --- name: check-phase -description: Use when you are acting as Guardian, Skeptic, Sage, or Trickster archetype in the Check phase. Defines shared review rules and output format. +description: Use when acting as Guardian, Skeptic, Sage, or Trickster in the Check phase. Defines review rules, finding format, attention filters, and spawning protocol. --- # Check Phase -Multiple reviewers examine the Maker's implementation in parallel. Each agent definition has its specific protocol — this skill defines the shared rules. +Reviewers examine the Maker's implementation. This skill defines shared rules, finding format, and spawning protocol. ## Shared Rules -1. **Read the proposal first.** Review against the intended design, not invented requirements. -2. **Read the actual code.** Use `git diff` on the Maker's branch. Don't review descriptions alone. -3. **Structured findings.** Use the standardized finding format below for every issue. -4. **Clear verdict:** `APPROVED` or `REJECTED` with rationale. -5. **Status tokens are separate from verdicts.** The `STATUS: DONE` line signals the agent finished successfully. The `APPROVED`/`REJECTED` verdict is domain output. A reviewer can be `STATUS: DONE` with verdict `REJECTED` — that is normal. Parse both independently. +1. Review against the proposal's intended design, not invented requirements. +2. Read actual code via `git diff` on the Maker's branch. +3. Use the finding format below for every issue. +4. Give a clear verdict: `APPROVED` or `REJECTED` with rationale. +5. `STATUS: DONE` signals agent completion. `APPROVED`/`REJECTED` is domain output. Both are parsed independently. ## Finding Format -Every finding must use this format for cross-cycle tracking: - -``` | Location | Severity | Category | Description | Fix | |----------|----------|----------|-------------|-----| -| src/auth/handler.ts:48 | CRITICAL | security | Empty string bypasses validation | Add length check before processing | -``` +| src/auth/handler.ts:48 | CRITICAL | security | Empty string bypasses validation | Add length check | -**Severity:** -- **CRITICAL** — Must fix. Blocks approval. -- **WARNING** — Should fix. Doesn't block alone. -- **INFO** — Nice to have. Never blocks. +**Severity:** CRITICAL = must fix, blocks approval. WARNING = should fix, doesn't block alone. INFO = nice to have, never blocks. -**Categories** (use consistently for cross-cycle tracking): -- `security` — Injection, auth bypass, data exposure, secrets -- `reliability` — Error handling, edge cases, race conditions, crashes -- `design` — Architecture, assumptions, scalability, coupling -- `breaking-change` — API compatibility, schema migrations, removals -- `dependency` — New deps, version conflicts, license issues -- `quality` — Readability, maintainability, naming, duplication -- `testing` — Missing tests, weak assertions, untested paths -- `consistency` — Deviates from codebase patterns +**Categories:** `security` `reliability` `design` `breaking-change` `dependency` `quality` `testing` `consistency` -## Consolidated Output +## Evidence Requirements -After all reviewers finish, compile: +Every CRITICAL or WARNING must include concrete evidence. Without evidence, downgrade to INFO. -```markdown -## Check Phase Results — Cycle N +**Valid evidence:** command output, exit codes, code citations with line numbers, git diff excerpts, reproduction steps. -### Guardian: APPROVED -| Location | Severity | Category | Description | Fix | -|----------|----------|----------|-------------|-----| -| src/auth/handler.ts:52 | WARNING | security | Missing rate limit | Add rate limiter middleware | +**Banned in CRITICAL/WARNING:** "might be", "could potentially", "appears to", "seems like", "may not". Rewrite with evidence or downgrade. -### Skeptic: APPROVED -| Location | Severity | Category | Description | Fix | -|----------|----------|----------|-------------|-----| -| src/auth/handler.ts:30 | INFO | design | Consider caching validated tokens | Add TTL cache for token validation | +For each CRITICAL/WARNING, state: (1) what was tested, (2) what was observed, (3) what correct behavior should be. -### Sage: APPROVED -| Location | Severity | Category | Description | Fix | -|----------|----------|----------|-------------|-----| -| tests/auth.test.ts:15 | WARNING | testing | Test names don't describe behavior | Rename to "should reject expired tokens" | +## Attention Filters -### Trickster: REJECTED -| Location | Severity | Category | Description | Fix | -|----------|----------|----------|-------------|-----| -| src/auth/handler.ts:48 | CRITICAL | reliability | Empty string bypasses validation | Add `if (!token || token.trim() === '')` guard | +Each archetype receives only relevant context. Do not pass everything. -### Deduplication -If two reviewers raise the same issue (same file + same category), merge: -| Guardian + Skeptic | CRITICAL | security | Input not sanitized (src/api.ts:30) | Add validation | +| Archetype | Receives | Excludes | +|-----------|----------|----------| +| Guardian | Maker's git diff + proposal risk section + test results | Explorer research, Creator rationale, other reviewers | +| Skeptic | Creator's proposal (assumptions + architecture) + confidence scores | Git diff, Explorer research, other reviewers | +| Sage | Creator's proposal + Maker's diff + implementation summary + test results | Explorer raw research, other reviewer verdicts | +| Trickster | Maker's git diff + attack surface summary (file types + entry points) | Proposal, research, other reviewers | -Use the higher severity. Don't double-count in the verdict. +**Token budget targets:** -### Verdict: REJECTED — 1 critical finding -→ Build cycle feedback (see orchestration skill) and feed to Plan phase -``` +| Archetype | Fast | Standard | Thorough | +|-----------|------|----------|----------| +| Guardian | 1500 | 2000 | 2500 | +| Skeptic | skip | 1500 | 2000 | +| Trickster | skip | skip | 1500 | +| Sage | skip | 2500 | 3000 | + +**Context isolation:** Agents receive fresh, controller-constructed context only. No session bleed, no cross-agent contamination, no ambient knowledge. Verify zero references to excluded artifacts before spawning. + +**Cycle-back filtering (cycle 2+):** Pass structured feedback table only (not full reviewer artifacts). Strip resolved items. Cap at 500 tokens — summarize by severity if exceeded. ## Reviewer Spawning Protocol -This section defines the exact sequence for spawning reviewers in the Check phase. - ### Step 1: Guardian First (mandatory) -Guardian always runs first, before any other reviewer. It receives the Maker's git diff and the proposal's risk section only. - -**Context for Guardian:** -- `git diff main...` (the actual code changes) -- Risk section from `plan-creator.md` (if present) -- Do NOT include: Explorer research, full proposal, other reviewer outputs - -``` -Agent( - description: "Guardian: security and risk review for ", - prompt: "You are the GUARDIAN archetype. - Review the diff: - Proposal risks: - Assess: security vulnerabilities, reliability risks, breaking changes, dependency risks. - Output: APPROVED or REJECTED with findings in the standardized format. - Each finding: | Location | Severity | Category | Description | Fix |", - model: -) -``` +Guardian always runs first. It receives the Maker's git diff and the proposal's risk section only. Save output to `.archeflow/artifacts/${RUN_ID}/check-guardian.md`. ### Step 2: A2 Fast-Path Evaluation -After Guardian completes, parse its output before spawning other reviewers: +After Guardian completes, count CRITICAL and WARNING findings in its output. If both are zero, and not escalated, and not first cycle of a thorough workflow — skip remaining reviewers and proceed to Act phase. -```bash -CRITICAL_COUNT=$(grep -c "| CRITICAL |" ".archeflow/artifacts/${RUN_ID}/check-guardian.md" || echo 0) -WARNING_COUNT=$(grep -c "| WARNING |" ".archeflow/artifacts/${RUN_ID}/check-guardian.md" || echo 0) +### Step 3: Parallel Remaining Reviewers -# A2 fast-path: skip remaining reviewers if Guardian is clean -# Exception: first cycle of thorough workflows always spawns all reviewers -if [[ "$CRITICAL_COUNT" -eq 0 && "$WARNING_COUNT" -eq 0 \ - && "$ESCALATED" != "true" \ - && ! ("$WORKFLOW" == "thorough" && "$CYCLE" -eq 1) ]]; then - echo "Guardian fast-path: 0 CRITICAL, 0 WARNING — skipping remaining reviewers." - # Proceed directly to Act phase -fi -``` - -### Step 3: Parallel Reviewer Spawning - -If A2 does not trigger, spawn remaining reviewers in parallel based on workflow: +If A2 does not trigger, spawn remaining reviewers in parallel: | Workflow | Reviewers (after Guardian) | |----------|--------------------------| | `fast` | None (Guardian only) | -| `fast` (escalated via A1) | Skeptic + Sage | +| `fast` (escalated) | Skeptic + Sage | | `standard` | Skeptic + Sage | | `thorough` | Skeptic + Sage + Trickster | -Spawn all applicable reviewers in a single message with multiple Agent calls: +Each reviewer gets context per the attention filters above. -``` -# Standard workflow example — spawn Skeptic and Sage in parallel: -Agent( - description: "Skeptic: challenge assumptions for ", - prompt: "", - model: -) +### Step 4: Collect and Consolidate -Agent( - description: "Sage: holistic quality review for ", - prompt: "", - model: -) +For each reviewer: save to `.archeflow/artifacts/${RUN_ID}/check-.md`, emit `review.verdict` event, record sequence number. + +**Deduplication:** If two reviewers raise the same issue (same file + same category), merge into one finding using the higher severity. Don't double-count. + +**Verdict:** Count CRITICAL findings across all reviewers (after dedup). Any CRITICAL = `REJECTED`. Otherwise `APPROVED`. + +Example consolidated output: + +```markdown +## Check Phase Results — Cycle 1 +### Guardian: APPROVED +| Location | Severity | Category | Description | Fix | +|----------|----------|----------|-------------|-----| +| src/auth.ts:52 | WARNING | security | Missing rate limit | Add rate limiter | +### Verdict: APPROVED — 0 critical, 1 warning ``` -Each reviewer gets context per the attention filters defined in `archeflow:orchestration`: -- **Skeptic:** Creator's proposal (assumptions section focus) -- **Sage:** Creator's proposal + Maker's diff + implementation summary -- **Trickster:** Maker's diff only +## Timeout Handling -### Step 4: Collect Results +Each reviewer has a **5-minute timeout**. On timeout: emit `agent.complete` with `"error": true`, log WARNING, treat as no findings, proceed. -Wait for all spawned reviewers to return. For each: -1. Save output to `.archeflow/artifacts/${RUN_ID}/check-.md` -2. Emit `review.verdict` event with findings -3. Record sequence number for DAG parent tracking - -### Timeout Handling - -Each reviewer has a **5-minute timeout**. If a reviewer does not return within 5 minutes: -1. Emit `agent.complete` with `"error": true, "reason": "timeout"` -2. Log a WARNING — do not block the run -3. Treat the timed-out reviewer as having delivered no findings (neither approved nor rejected) -4. Proceed with available verdicts - -If Guardian times out, this is a blocking failure — abort the Check phase and report to the user. - -### Re-Check Protocol (Act Phase Fixes) - -When the Act phase routes findings back to the Maker and the Maker applies fixes in a subsequent cycle, the Check phase re-runs with the updated diff. Reviewers who previously rejected should focus on whether their specific findings were addressed. The structured feedback from `act-feedback.md` provides the mapping of which findings were routed where. - ---- - -## Evidence Requirements - -Every CRITICAL or WARNING finding must include concrete evidence. Findings without evidence are downgraded to INFO. - -### Evidence Types - -| Type | Example | When Required | -|------|---------|---------------| -| Command output | `npm test` output showing failure | Test-related findings | -| Exit code | `exit code 1 from eslint` | Tool-based validation | -| Code citation | `src/auth.ts:48 — \`if (token) { ... }\`` | Logic or security findings | -| Git diff | `+ db.query(userInput)` (unsanitized) | Implementation review | -| Reproduction steps | "1. Send POST with empty body, 2. Observe 500" | Runtime behavior findings | - -### Banned Phrases - -The following phrases are not permitted in CRITICAL or WARNING findings. They indicate speculation, not evidence: - -- "might be" -- "could potentially" -- "appears to" -- "seems like" -- "may not" - -A finding using these phrases must either be rewritten with evidence or downgraded to INFO. - -### Verification Protocol - -For each CRITICAL or WARNING finding, state: - -1. **What was tested** — the specific code path, input, or scenario examined -2. **What was observed** — the actual behavior or code construct found -3. **What correct behavior should be** — the expected alternative - -### Downgrade Rule - -If a reviewer produces a CRITICAL or WARNING finding without any of the evidence types above, the orchestrator downgrades it to INFO and emits a `decision` event: - -```bash -./lib/archeflow-event.sh "$RUN_ID" decision check "" \ - '{"what":"evidence_downgrade","from":"CRITICAL","to":"INFO","finding":"","reviewer":"","reason":"no evidence provided"}' -``` - ---- - -## Why Structured Findings Matter - -The standardized format enables: -- **Cross-cycle tracking:** Same category + location = same issue. Can detect resolution or regression. -- **Feedback routing:** Security/design findings → Creator. Quality/testing findings → Maker. -- **Shadow detection:** CRITICAL:WARNING ratios, finding counts, and category distributions are measurable. -- **Metrics:** Severity counts feed into the orchestration summary. +**Exception:** Guardian timeout is blocking — abort Check phase and report to user. diff --git a/skills/colette-bridge/SKILL.md b/skills/colette-bridge/SKILL.md index f04f4ac..05354fe 100644 --- a/skills/colette-bridge/SKILL.md +++ b/skills/colette-bridge/SKILL.md @@ -186,7 +186,7 @@ When regenerating: ## Per-Agent Attention Filters -Not every agent needs the full bundle. The bridge defines attention filters that control which sections each archetype receives. This extends the base attention filters from `archeflow:attention-filters`. +Not every agent needs the full bundle. The bridge defines attention filters that control which sections each archetype receives. This extends the base attention filters from `archeflow:check-phase`. | Archetype | Bundle sections injected | Rationale | |-----------|------------------------|-----------| diff --git a/skills/using-archeflow/SKILL.md b/skills/using-archeflow/SKILL.md index bf3efc6..5e5b584 100644 --- a/skills/using-archeflow/SKILL.md +++ b/skills/using-archeflow/SKILL.md @@ -157,7 +157,6 @@ Read `.archeflow/session-log.md` and show the last 5 orchestration summaries in ### Quality and Safety - **archeflow:shadow-detection** -- Quantitative dysfunction detection and correction -- **archeflow:attention-filters** -- Context optimization per archetype - **archeflow:convergence** -- Detects convergence, stalling, and oscillation in multi-cycle runs - **archeflow:artifact-routing** -- Inter-phase artifact protocol for naming, storage, and routing From 8837a359acbfe340dd52f635b6bae2ee0bdf7a7a Mon Sep 17 00:00:00 2001 From: Christian Nennemann Date: Mon, 6 Apr 2026 20:42:47 +0200 Subject: [PATCH 2/2] refactor: simplify memory and shadow-detection skills Trim verbose implementation details that duplicate what the bash helper scripts already handle. Memory skill: 278 -> 120 lines. Shadow detection skill: 180 -> 66 lines. All essential protocols, tables, and commands preserved; removed redundant algorithm descriptions, multiple examples, and narrative prose. --- skills/memory/SKILL.md | 235 +++++-------------------------- skills/shadow-detection/SKILL.md | 202 ++++++-------------------- 2 files changed, 83 insertions(+), 354 deletions(-) diff --git a/skills/memory/SKILL.md b/skills/memory/SKILL.md index 741f1eb..cc09539 100644 --- a/skills/memory/SKILL.md +++ b/skills/memory/SKILL.md @@ -11,21 +11,14 @@ description: | # Cross-Run Memory -ArcheFlow forgets everything after each run. If Guardian repeatedly flags the same type of issue (e.g., timeline errors in fiction, missing null checks in code), the next run starts from zero. This skill fixes that by extracting lessons from completed runs and injecting them into future agent prompts. +ArcheFlow forgets everything after each run. This skill extracts lessons from completed runs and injects them into future agent prompts, so recurring issues (timeline errors, missing null checks) are caught proactively. ## Storage ``` .archeflow/memory/lessons.jsonl # Append-only, one lesson per line -``` - -Each lesson is a single JSON line: - -```jsonl -{"id":"m-001","ts":"2026-04-03T14:00:00Z","run_id":"2026-04-03-der-huster","type":"pattern","source":"guardian","description":"Timeline references must match story start day","frequency":2,"severity":"bug","domain":"writing","tags":["continuity","timeline"],"last_seen_run":"2026-04-03-der-huster","runs_since_last_seen":0} -{"id":"m-002","ts":"2026-04-03T15:00:00Z","run_id":"2026-04-03-der-huster","type":"preference","source":"user_feedback","description":"User prefers single bundled PR over many small ones","frequency":1,"severity":"info","domain":"general","tags":["workflow"],"last_seen_run":"","runs_since_last_seen":0} -{"id":"m-003","ts":"2026-04-04T10:00:00Z","run_id":"2026-04-04-auth-fix","type":"archetype_hint","source":"sage","description":"Voice drift most common in long monologue passages","frequency":3,"severity":"warning","domain":"writing","tags":["voice","prose"],"archetype":"story-sage","last_seen_run":"2026-04-04-auth-fix","runs_since_last_seen":0} -{"id":"m-004","ts":"2026-04-04T11:00:00Z","run_id":"2026-04-04-auth-fix","type":"anti_pattern","source":"maker","description":"Splitting auth middleware into per-route handlers causes duplication","frequency":1,"severity":"warning","domain":"code","tags":["auth","middleware"],"last_seen_run":"2026-04-04-auth-fix","runs_since_last_seen":0} +.archeflow/memory/archive.jsonl # Decayed lessons (frequency reached 0) +.archeflow/memory/audit.jsonl # Injection audit trail ``` ## Lesson Types @@ -33,245 +26,95 @@ Each lesson is a single JSON line: | Type | Source | Description | |------|--------|-------------| | `pattern` | Auto-detected | Recurring finding across runs (same category + similar description) | -| `preference` | Manual | User correction or workflow preference (added via CLI) | +| `preference` | Manual | User correction or workflow preference (injected immediately, skips frequency threshold) | | `archetype_hint` | Auto-detected | Per-archetype insight (e.g., Sage catches voice drift in monologues) | -| `anti_pattern` | Manual or auto | Something that was tried and failed — avoid repeating | +| `anti_pattern` | Manual or auto | Something that was tried and failed -- avoid repeating | -## Lesson Fields +## Lesson JSON Fields | Field | Type | Description | |-------|------|-------------| -| `id` | string | Unique ID, format `m-NNN` (monotonically increasing) | -| `ts` | ISO 8601 | When the lesson was created or last updated | +| `id` | string | `m-NNN` (monotonically increasing) | +| `ts` | ISO 8601 | Created or last updated | | `run_id` | string | Run that created or last triggered this lesson | -| `type` | string | One of: `pattern`, `preference`, `archetype_hint`, `anti_pattern` | -| `source` | string | Archetype or `user_feedback` that originated the lesson | +| `type` | string | `pattern`, `preference`, `archetype_hint`, `anti_pattern` | +| `source` | string | Archetype name or `user_feedback` | | `description` | string | Human-readable lesson text | -| `frequency` | integer | How many times this lesson was triggered | -| `severity` | string | `bug`, `warning`, `info`, or `recommendation` | +| `frequency` | integer | Times this lesson was triggered | +| `severity` | string | `bug`, `warning`, `info`, `recommendation` | | `domain` | string | `writing`, `code`, `general`, or project-specific | | `tags` | string[] | Keywords for matching and filtering | -| `archetype` | string or null | For `archetype_hint` type — which archetype this applies to | -| `last_seen_run` | string | Run ID where this lesson was last matched | -| `runs_since_last_seen` | integer | Counter for decay — incremented each run that does NOT trigger this lesson | +| `archetype` | string? | For `archetype_hint` -- which archetype this applies to | +| `last_seen_run` | string | Run ID where last matched | +| `runs_since_last_seen` | integer | Counter for decay | + +Example: +```jsonl +{"id":"m-001","ts":"2026-04-03T14:00:00Z","run_id":"2026-04-03-der-huster","type":"pattern","source":"guardian","description":"Timeline references must match story start day","frequency":2,"severity":"bug","domain":"writing","tags":["continuity","timeline"],"last_seen_run":"2026-04-03-der-huster","runs_since_last_seen":0} +``` --- ## Auto-Detection -After each `run.complete`, the orchestrator runs lesson extraction: +After each `run.complete`, extract lessons from findings: ```bash ./lib/archeflow-memory.sh extract .archeflow/events/.jsonl ``` -### Extraction Algorithm +The script reads `review.verdict` events, matches findings against existing lessons by keyword overlap (50%+ threshold), increments frequency on matches, and creates new candidate lessons (frequency: 1) for unmatched findings with severity >= WARNING. -1. **Read all `review.verdict` events** from the completed run's JSONL. -2. **For each finding** in each verdict: - a. Tokenize the finding description into keywords (lowercase, strip punctuation). - b. Compare keywords against each existing lesson's description + tags. - c. **Match threshold:** 50%+ keyword overlap between finding and lesson. -3. **If match found:** Update the existing lesson: - - Increment `frequency` by 1 - - Update `ts` to now - - Update `last_seen_run` to current run ID - - Reset `runs_since_last_seen` to 0 -4. **If no match AND severity >= WARNING:** Add as candidate lesson with `frequency: 1`. -5. **Candidates become active** when `frequency >= 2` (triggered in a second run). - -### Promotion Rule - -A finding that appears in only one run stays at `frequency: 1` — it might be a one-off. Once the same pattern appears in a second run (matched by keyword overlap), it gets promoted to `frequency: 2` and becomes eligible for injection. - -This prevents noise from single-run anomalies while still capturing genuine recurring issues quickly. - ---- +**Promotion rule:** A finding needs `frequency >= 2` (seen in 2+ runs) before injection. This filters out one-off noise. Preferences skip this threshold. ## Injection -At run start, before spawning agents, the orchestrator injects relevant lessons: +Before spawning agents, inject relevant lessons: ```bash LESSONS=$(./lib/archeflow-memory.sh inject ) ``` -### Injection Rules +Rules: filters by domain (or `general`), optionally by archetype, requires `frequency >= 2`, sorts by frequency descending, caps at 10 lessons. Lessons with `frequency >= 5` are always injected regardless of filters. -1. Read `lessons.jsonl`. -2. Filter by `domain` (exact match or `general`) and optionally by `archetype`. -3. Only include lessons with `frequency >= 2` (confirmed patterns). -4. Sort by frequency descending (most common first). -5. Cap at **10 lessons** per injection. -6. Lessons with `frequency >= 5` are **always injected** regardless of domain/archetype filter (they are universal enough to matter). - -### Injection Format - -Append to the agent's system prompt as a structured section: +Injected as a markdown section appended to the agent's system prompt: ```markdown ## Known Issues (from past runs) - Timeline references must match story start day [seen 3x, guardian] - Voice drift common in monologue passages >200 words [seen 2x, sage] -- Missing null checks in API response handlers [seen 5x, guardian] ``` -### Integration with Run Skill - -In the `run` skill, after Step 0 (Initialize) and before Step 1 (Plan Phase): - -```bash -# Load cross-run memory for this domain -MEMORY_LESSONS=$(./lib/archeflow-memory.sh inject "$DOMAIN" "") - -# Inject into Explorer/Creator prompts if non-empty -if [[ -n "$MEMORY_LESSONS" ]]; then - EXPLORER_PROMPT="${EXPLORER_PROMPT} - -${MEMORY_LESSONS}" - CREATOR_PROMPT="${CREATOR_PROMPT} - -${MEMORY_LESSONS}" -fi -``` - -For reviewers in the Check phase, inject archetype-specific lessons: - -```bash -GUARDIAN_LESSONS=$(./lib/archeflow-memory.sh inject "$DOMAIN" "guardian") -SAGE_LESSONS=$(./lib/archeflow-memory.sh inject "$DOMAIN" "sage") -``` - ---- - ## Decay -Lessons that stop being relevant should fade out. After each `run.complete`, apply decay: +After each `run.complete`, apply decay: lessons not seen for 10 runs lose 1 frequency. When frequency reaches 0, the lesson is archived. ```bash ./lib/archeflow-memory.sh decay ``` -### Decay Algorithm - -1. For every lesson in `lessons.jsonl`: - - If `last_seen_run` is NOT the current run → increment `runs_since_last_seen` by 1 -2. If `runs_since_last_seen >= 10`: - - Decrement `frequency` by 1 - - Reset `runs_since_last_seen` to 0 -3. If `frequency` drops to 0: - - Move the lesson to `.archeflow/memory/archive.jsonl` (append) - - Remove from `lessons.jsonl` - -This means a lesson that was seen 5 times but then stops appearing will survive 50 runs of non-triggering before being fully archived (5 decrements x 10 runs each). - ---- - ## Manual Management -### Add a lesson +```bash +archeflow memory add "User prefers single bundled PR" # Add preference (injected immediately) +archeflow memory list # Show all active lessons +archeflow memory forget m-002 # Archive a lesson +``` + +## Audit Trail + +Track which lessons are injected per run and whether they were effective. Pass `--audit ` to inject to log records. After a run, `audit-check ` compares injected lessons against review findings: no matching finding = helpful (issue prevented), matching finding = ineffective (issue repeated despite injection). ```bash -archeflow memory add "User prefers single bundled PR over many small ones" -# Internally: ./lib/archeflow-memory.sh add preference "User prefers single bundled PR over many small ones" +./lib/archeflow-memory.sh inject "$DOMAIN" "" --audit "$RUN_ID" +./lib/archeflow-memory.sh audit-check ``` -Manually added lessons start at `frequency: 1` but with type `preference`, which means they are injected immediately (preferences skip the frequency >= 2 threshold). - -### List lessons - -```bash -archeflow memory list -# Internally: ./lib/archeflow-memory.sh list -``` - -Output: - -``` -ID Freq Type Domain Description -m-001 3 pattern writing Timeline references must match story start day -m-002 1 preference general User prefers single bundled PR over many small ones -m-003 5 archetype_hint writing Voice drift most common in long monologue passages -m-004 1 anti_pattern code Splitting auth middleware causes duplication -``` - -### Forget a lesson - -```bash -archeflow memory forget m-002 -# Internally: ./lib/archeflow-memory.sh forget m-002 -``` - -Moves the lesson to `archive.jsonl` regardless of frequency. - ---- - ## Integration Points | Moment | Action | Script Command | |--------|--------|----------------| | After `run.complete` | Extract lessons from findings | `archeflow-memory.sh extract ` | | After extraction | Apply decay to all lessons | `archeflow-memory.sh decay` | -| Before agent spawn (run start) | Inject relevant lessons | `archeflow-memory.sh inject ` | +| Before agent spawn | Inject relevant lessons | `archeflow-memory.sh inject ` | | User command | Add/list/forget lessons | `archeflow-memory.sh add/list/forget` | - -## Audit Trail - -Track which lessons are injected into each run and whether they were effective. - -### Storage - -``` -.archeflow/memory/audit.jsonl # Append-only audit log -``` - -### Injection Audit Record - -When `--audit ` is passed to the `inject` command, an audit record is written: - -```jsonl -{"ts":"2026-04-04T10:00:00Z","run_id":"2026-04-04-auth-fix","domain":"code","archetype":"","lessons_injected":["m-001","m-003"],"lesson_count":2} -``` - -Usage: -```bash -./lib/archeflow-memory.sh inject "$DOMAIN" "" --audit "$RUN_ID" -``` - -### Effectiveness Check - -After a run completes, check whether injected lessons prevented issues: - -```bash -./lib/archeflow-memory.sh audit-check -``` - -This command: -1. Reads `audit.jsonl` for lessons injected in the given run -2. Reads the run's event file for `review.verdict` events -3. For each injected lesson, checks keyword overlap between the lesson's description and review findings -4. **No matching finding** = `helpful` (the lesson likely prevented the issue) -5. **Matching finding** = `ineffective` (the issue repeated despite the lesson being injected) -6. Appends effectiveness results to `audit.jsonl` - -### Effectiveness Over Time - -By querying `audit.jsonl` for effectiveness records, you can measure: -- Which lessons consistently prevent issues (high `helpful` count) -- Which lessons are not working (high `ineffective` count — consider rewording or removing) -- Overall memory system ROI (ratio of helpful to ineffective across all runs) - -```bash -# Count effectiveness per lesson -jq -r 'select(.type == "effectiveness_check") | [.lesson_id, .effectiveness] | @tsv' .archeflow/memory/audit.jsonl | sort | uniq -c -``` - ---- - -## Design Principles - -1. **Append-only storage.** `lessons.jsonl` is append-only during writes; decay rewrites the file in place but preserves all data (archived lessons move to `archive.jsonl`). -2. **Conservative promotion.** A finding must appear in 2+ runs before injection. One-offs are noise. -3. **Graceful degradation.** If `lessons.jsonl` doesn't exist, injection returns empty — no error, no block. -4. **Cheap.** Keyword matching, not embeddings. `jq` for JSON, `grep` for matching. No external services. -5. **Bounded.** Max 10 lessons injected per prompt. Prevents context pollution. diff --git a/skills/shadow-detection/SKILL.md b/skills/shadow-detection/SKILL.md index cb0dce0..b9c1dbb 100644 --- a/skills/shadow-detection/SKILL.md +++ b/skills/shadow-detection/SKILL.md @@ -5,176 +5,62 @@ description: Use when monitoring agent behavior for dysfunction, when an agent s # Shadow Detection -Every archetype has a **virtue** (its unique contribution) and a **shadow** (the destructive inversion of that virtue). A shadow activates when the virtue is pushed too far. +Every archetype has a virtue and a shadow (its destructive inversion). Shadow activates when the virtue is pushed too far. -``` -Virtue (healthy) → pushed too far → Shadow (dysfunction) - -Contextual Clarity → can't stop → Rabbit Hole -Decisive Framing → over-builds → Over-Architect -Execution Discipline → no guardrails → Rogue -Threat Intuition → sees threats only → Paranoid -Assumption Surfacing → questions only → Paralytic -Adversarial Creativity → noise over signal → False Alarm -Maintainability Judgment → reviews only → Bureaucrat -``` +| Archetype | Virtue | Shadow | +|-----------|--------|--------| +| Explorer | Contextual Clarity | Rabbit Hole | +| Creator | Decisive Framing | Over-Architect | +| Maker | Execution Discipline | Rogue | +| Guardian | Threat Intuition | Paranoid | +| Skeptic | Assumption Surfacing | Paralytic | +| Trickster | Adversarial Creativity | False Alarm | +| Sage | Maintainability Judgment | Bureaucrat | --- -## Explorer → Rabbit Hole -**Virtue inverted:** Contextual Clarity becomes compulsive investigation — or output that dumps without analyzing. +### Explorer -> Rabbit Hole +**Detect** (any): output >2000w without Recommendation | >3 tangents | >15 files no patterns | no synthesis in final 25% +**Correct**: "Summarize top 3 findings and one recommendation in under 300 words." -**Symptoms:** -- Research output keeps growing but never synthesizes -- "I found one more thing to check" repeated 3+ times -- Reading more than 15 files without producing findings -- Output is a raw inventory of files with no analysis or recommendation +### Creator -> Over-Architect +**Detect** (any): >2 new abstractions for a single feature | "future-proof" in rationale | scope exceeds task by >50% | >1 new package for one feature +**Correct**: "Design for the current order of magnitude. Remove abstractions that serve hypothetical requirements." -**Detection Checklist** (trigger on ANY): -- [ ] Output >2000 words without a `### Recommendation` section -- [ ] >3 tangent topics not directly related to the original task -- [ ] >15 files read with no `### Patterns` identified -- [ ] No synthesis language (recommend, suggest, conclusion, finding, summary) in final 25% of output +### Maker -> Rogue +**Detect** (any): zero test files with >=3 files changed | single monolithic commit | diff contains files not in proposal | no evidence of running tests +**Correct**: "Read the proposal. Write a test. Commit what you have. Revert changes to files not in the proposal." -**Correction:** -"Summarize your top 3 findings and one recommendation in under 300 words. If your output has no Recommendation section, add one. A dump is not research." +### Guardian -> Paranoid +**Detect** (any): CRITICAL:WARNING ratio >2:1 (min 3 findings) | zero APPROVED in 3+ reviews | <50% findings include a fix | findings require already-compromised systems +**Correct**: "For each CRITICAL: would a senior engineer block a PR for this? If not, downgrade. Every rejection must include a specific fix." + +### Skeptic -> Paralytic +**Detect** (any): >7 challenges in a single review | <50% include alternatives | same concern appears 2+ times reworded | >3 findings outside task scope +**Correct**: "Rank challenges by impact. Keep top 3. Each must include a specific alternative. Delete the rest." + +### Trickster -> False Alarm +**Detect** (any): findings reference code untouched by diff | >10 findings for <5 files | impossible deployment scenarios | >3 findings without repro steps +**Correct**: "Delete findings outside the diff. Rank remaining by likelihood x impact. Keep top 3-5." + +### Sage -> Bureaucrat +**Detect** (any): review words >2x diff lines | findings reference files not in changeset | >2 "consider" without concrete action | suggesting docs for <5-line functions +**Correct**: "Limit to issues affecting maintainability in the next 6 months. Every finding must end with a specific action." --- -## Creator → Over-Architect -**Virtue inverted:** Decisive Framing becomes designing at the wrong scale. +## Escalation Protocol -**Symptoms:** -- Abstraction layers for one-time operations -- Future-proofing for requirements that don't exist -- Configuration systems for things that could be constants -- Proposal has more infrastructure than business logic - -**Detection Checklist** (trigger on ANY): -- [ ] >2 new abstractions (interfaces, base classes, factories, registries) for a single feature -- [ ] "In the future we might need..." or "future-proof" appears in rationale -- [ ] Proposal scope (files changed) exceeds original task scope by >50% -- [ ] More than 1 new package/module introduced for a single feature - -**Correction:** -"Design for the current order of magnitude. If the app has 1000 users, design for 10,000 — not 10 million. Remove abstractions that serve hypothetical requirements." - ---- - -## Maker → Rogue -**Virtue inverted:** Execution Discipline becomes reckless shipping — or expanding beyond the plan. - -**Symptoms:** -- Writing code before reading the proposal fully -- No tests, or tests written after implementation -- Large uncommitted working tree -- Files changed that aren't mentioned in the proposal - -**Detection Checklist** (trigger on ANY): -- [ ] Zero test files (`.test.`, `.spec.`, `_test.`) in the changeset with >=3 files changed -- [ ] Single monolithic commit instead of incremental commits -- [ ] Diff contains files not listed in the Creator's proposal `### Changes` section -- [ ] No evidence of running existing test suite before finishing - -**Correction:** -"Read the proposal. Write a test. Commit what you have. Revert changes to files not in the proposal. Then continue." - ---- - -## Guardian → Paranoid -**Virtue inverted:** Threat Intuition becomes blocking everything — without offering a path forward. - -**Symptoms:** -- Every finding marked CRITICAL -- Blocking on theoretical risks with < 1% probability -- Rejecting without suggesting how to fix -- Security concerns for internal-only code at external-API severity - -**Detection Checklist** (trigger on ANY): -- [ ] CRITICAL:WARNING ratio >2:1 (with minimum 3 total findings) -- [ ] Zero APPROVED verdicts in 3+ consecutive reviews -- [ ] <50% of findings include a suggested fix in the `Fix` column -- [ ] Findings reference attack scenarios that require already-compromised internal systems - -**Correction:** -"For each CRITICAL finding, answer: Would a senior engineer block a PR for this? If not, downgrade. Every rejection must include a specific, implementable fix." - ---- - -## Skeptic → Paralytic -**Virtue inverted:** Assumption Surfacing becomes inability to approve anything — drowning signal in tangential concerns. - -**Symptoms:** -- More than 7 challenges raised -- Challenges without suggested alternatives -- "What about X?" chains that drift from the task -- Restating the same concern in different words - -**Detection Checklist** (trigger on ANY): -- [ ] >7 findings/challenges raised in a single review -- [ ] <50% of findings include an alternative in the `Fix` column -- [ ] Same conceptual concern appears 2+ times with different wording -- [ ] >3 findings reference code or scenarios outside the task scope - -**Correction:** -"Rank your challenges by impact. Keep the top 3. Each must include a specific alternative. Delete the rest." - ---- - -## Trickster → False Alarm -**Virtue inverted:** Adversarial Creativity becomes noise — too many low-signal findings drowning the real issues. - -**Symptoms:** -- Testing code that wasn't changed -- Reporting non-bugs as bugs (unrealistic test scenarios) -- 20 findings when 3 good ones would cover the real risks -- Edge cases for edge cases (diminishing returns) - -**Detection Checklist** (trigger on ANY): -- [ ] Any finding references code untouched by the Maker's diff -- [ ] >10 findings for a change touching <5 files -- [ ] Findings describe scenarios requiring conditions that can't occur in the deployment context -- [ ] >3 findings without reproduction steps - -**Correction:** -"Quality over quantity. Delete findings outside the Maker's diff. Rank remaining by likelihood x impact. Keep top 3-5. Three real findings beat twenty noise." - ---- - -## Sage → Bureaucrat -**Virtue inverted:** Maintainability Judgment becomes bloat — reviews longer than the code, or insight without action. - -**Symptoms:** -- Review longer than the code change itself -- Requesting documentation for self-evident code -- Suggesting refactors unrelated to the current task -- Deep-sounding analysis that doesn't end with a specific action - -**Detection Checklist** (trigger on ANY): -- [ ] Review word count >2x the code change's line count (rough: review words > diff lines x 2) -- [ ] Any finding references files not in the Maker's changeset -- [ ] >2 findings use "consider" or "think about" without a concrete action in the `Fix` column -- [ ] Suggesting documentation for functions with <5 lines or self-descriptive names - -**Correction:** -"Limit your review to issues that affect maintainability in the next 6 months. Every finding must end with a specific action. If you can't state the consequence of NOT fixing it, don't raise it." - ---- - -## Shadow Escalation Protocol - -1. **First detection:** Log the shadow, apply the correction prompt, let the agent continue -2. **Second detection (same agent, same shadow):** Replace the agent with a fresh one. The shadow is entrenched. -3. **Shadow detected in 3+ agents in the same cycle:** The task itself may be poorly scoped. Escalate to the user: "Multiple agents are struggling — the task may need to be broken down." +1. **1st detection:** Log the shadow, apply the correction prompt, let the agent continue +2. **2nd detection (same agent, same shadow):** Replace the agent -- the shadow is entrenched +3. **3+ agents shadowed in same cycle:** Escalate to user -- the task may need to be broken down ## Shadow Immunity -Some behaviors LOOK like shadows but aren't: +Some behaviors look like shadows but are not. **Rule of thumb:** shadow = behavior disconnected from the goal. Intensity alone is not a shadow. -- Explorer reading 20 files in a monorepo with scattered dependencies → **not a rabbit hole** if each file is genuinely relevant -- Creator adding an abstraction → **not over-architect** if the abstraction is genuinely needed by the current task -- Guardian blocking with 2 CRITICAL findings → **not paranoid** if both are genuine security vulnerabilities -- Trickster finding 5 edge cases → **not false alarm** if all are in the changed code with reproduction steps -- Sage writing a long review → **not bureaucrat** if the change is large and every finding is actionable - -**Rule of thumb:** Shadow = behavior disconnected from the goal. Intensity alone is not a shadow. +- Explorer reading 20 files in a monorepo with scattered dependencies -- not a rabbit hole if each file is genuinely relevant +- Creator adding an abstraction -- not over-architect if the current task genuinely needs it +- Guardian blocking with 2 CRITICALs -- not paranoid if both are genuine security vulnerabilities +- Trickster finding 5 edge cases -- not false alarm if all are in changed code with repro steps +- Sage writing a long review -- not bureaucrat if the change is large and every finding is actionable