--- name: process-log description: | Event-based process logging for ArcheFlow orchestrations. Captures every phase transition, agent output, decision, and fix as structured JSONL events. Enables post-hoc reports, dashboards, and process archaeology. Automatically loaded during orchestration User: "Show me how this story was made" --- # Process Log — Event-Sourced Orchestration History Every ArcheFlow orchestration writes structured events to a JSONL file. Events are the **single source of truth** — all reports (Markdown, dashboards, timelines) are generated views. ## Event Storage ``` .archeflow/events/.jsonl # One file per orchestration run .archeflow/events/index.jsonl # Run index (one line per run, for listing) ``` **Run ID format:** `-` (e.g., `2026-04-03-der-huster`) ## When to Emit Events Emit an event at each of these points during orchestration: | Moment | Event Type | Trigger | |--------|-----------|---------| | Orchestration starts | `run.start` | After workflow selection, before first agent | | Agent spawned | `agent.start` | Before each Agent tool call | | Agent completes | `agent.complete` | After each Agent returns | | Phase transition | `phase.transition` | Plan→Do, Do→Check, Check→Act | | Decision made | `decision` | Plot direction chosen, fix applied, workflow adapted | | Review verdict | `review.verdict` | Guardian/Sage/Skeptic delivers verdict | | Fix applied | `fix.applied` | After each edit that addresses a review finding | | Cycle boundary | `cycle.boundary` | End of PDCA cycle, before next (or exit) | | Shadow detected | `shadow.detected` | Shadow threshold triggered | | Orchestration ends | `run.complete` | After final Act phase | ## Event Schema Every event is one JSON line with these required fields: ```jsonl { "ts": "2026-04-03T14:32:07Z", "run_id": "2026-04-03-der-huster", "seq": 4, "parent": [2], "type": "agent.complete", "phase": "plan", "agent": "creator", "data": { ... } } ``` | Field | Type | Description | |-------|------|-------------| | `ts` | ISO 8601 | Timestamp | | `run_id` | string | Unique run identifier | | `seq` | integer | Monotonically increasing sequence number within run | | `parent` | int[] | Seq numbers of causal parent events. Forms a DAG. `[]` for root events. | | `type` | string | Event type (see table above) | | `phase` | string | Current PDCA phase: `plan`, `do`, `check`, `act` | | `agent` | string or null | Agent archetype that triggered the event | | `data` | object | Event-type-specific payload (see below) | ### Parent Relationships (DAG) The `parent` field turns the flat event stream into a directed acyclic graph (agent call graph). This enables: - **Causal reconstruction:** which agent output caused which downstream action - **Parallel visualization:** agents sharing a parent ran concurrently - **Blame tracking:** trace a fix back through review → draft → outline → research Rules: - `run.start` has `parent: []` (root node) - An agent has `parent: [seq of event that triggered it]` - A phase transition has `parent: [seq of all completing events in prior phase]` - A fix has `parent: [seq of the review that found the issue]` - A decision has `parent: [seq of the agent that produced the alternatives]` - Parallel agents share the same parent (fan-out), phase transitions collect them (fan-in) Example DAG from a writing workflow: ``` #1 run.start [] ├── #2 agent.complete (explorer) [1] │ └── #3 decision (plot direction) [2] ├── #4 agent.complete (creator) [2] ← explorer informs creator ├── #5 phase.transition (plan→do) [3,4] ← fan-in │ └── #6 agent.complete (maker) [5] ├── #7 phase.transition (do→check) [6] │ ├── #8 review (guardian) [7] ← parallel (fan-out) │ └── #9 review (sage) [7] ← parallel (fan-out) ├── #10 phase.transition (check→act) [8,9] ← fan-in ├── #11 fix (timeline) [8] ← caused by guardian ├── #12 fix (voice drift) [9] ← caused by sage └── #18 run.complete [17] ``` ## Event Payloads by Type ### `run.start` ```json { "task": "Write short story 'Der Huster'", "workflow": "kurzgeschichte", "team": "story-development", "max_cycles": 2, "config": { "voice_profile": "vp-giesing-gschichten-v1", "persona": "giesinger", "target_words": 6000 } } ``` ### `agent.start` ```json { "archetype": "story-explorer", "model": "haiku", "prompt_summary": "Research premise, find emotional core, suggest 3 plot directions" } ``` ### `agent.complete` ```json { "archetype": "story-explorer", "duration_ms": 87605, "tokens": 21645, "artifacts": ["docs/01-der-huster-research.md"], "summary": "3 plot directions developed, recommended C (Mo krank + Koffer)" } ``` ### `decision` ```json { "what": "plot_direction", "chosen": "C — Mo krank + Koffer aus B", "alternatives": [ {"id": "A", "label": "Mo ist weg", "reason_rejected": "Zu passiv für 6k-Story"}, {"id": "B", "label": "Huster gehört nicht Mo", "reason_rejected": "Zu Krimi-nah"} ], "rationale": "Stärkster emotionaler Kern, passt zum Voice Profile" } ``` ### `review.verdict` ```json { "archetype": "guardian", "verdict": "approved_with_fixes", "findings": [ {"severity": "bug", "description": "Timeline: 'Montag' referenced but story starts Dienstag", "fix_required": true}, {"severity": "recommendation", "description": "Gentrification monologue too long for Alex register", "fix_required": false} ] } ``` ### `fix.applied` ```json { "source": "guardian", "finding": "Timeline: Montag → Dienstag", "file": "stories/01-der-huster.md", "line": 302, "before": "das Gegenteil von Montag", "after": "das Gegenteil von Dienstag" } ``` ### `phase.transition` ```json { "from": "plan", "to": "do", "artifacts_so_far": ["research.md", "outline.md"], "notes": "Explorer recommended direction C, Creator produced 6-scene outline" } ``` ### `cycle.boundary` ```json { "cycle": 1, "max_cycles": 2, "exit_condition": "all_approved", "met": true, "fixes_applied": 6, "next_action": "complete" } ``` ### `shadow.detected` ```json { "archetype": "story-explorer", "shadow": "endless_research", "trigger": "output >2000 words without recommendation", "action": "correction_prompt_applied", "occurrence": 1 } ``` ### `run.complete` ```json { "status": "completed", "cycles": 1, "agents_total": 5, "fixes_total": 6, "shadows": 0, "duration_ms": 1295519, "artifacts": [ "docs/01-der-huster-research.md", "docs/01-der-huster-outline.md", "stories/01-der-huster.md", "docs/01-der-huster-guardian-review.md", "docs/01-der-huster-sage-review.md", "docs/01-der-huster-process.md" ] } ``` ## How to Emit Events During orchestration, write events using this pattern: ```bash # Append one event to the run's JSONL file echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","run_id":"RUN_ID","seq":SEQ,"type":"TYPE","phase":"PHASE","agent":"AGENT","data":{...}}' >> .archeflow/events/RUN_ID.jsonl ``` Or use the helper script: ```bash ./lib/archeflow-event.sh RUN_ID TYPE PHASE AGENT '{"key":"value"}' ``` The orchestration skill should call the event emitter at each trigger point listed in the table above. ## Generating Reports After orchestration completes (or during, for live progress): ```bash # Generate markdown process report ./lib/archeflow-report.sh .archeflow/events/2026-04-03-der-huster.jsonl > docs/process-report.md # List all runs cat .archeflow/events/index.jsonl | jq -r '[.run_id, .status, .task] | @tsv' ``` ## Run Index After each `run.complete`, append a summary line to `.archeflow/events/index.jsonl`: ```jsonl {"run_id":"2026-04-03-der-huster","ts":"2026-04-03T16:00:00Z","task":"Write Der Huster","workflow":"kurzgeschichte","status":"completed","cycles":1,"agents":5,"fixes":6,"duration_ms":1295519} ``` ## Integration with Existing Skills - **`orchestration`**: Emit events at phase transitions and after each agent - **`shadow-detection`**: Emit `shadow.detected` when thresholds trigger - **`autonomous-mode`**: Use `index.jsonl` for session summaries instead of separate session-log - **`workflow-design`**: Custom workflows inherit logging automatically ## Design Principles 1. **Append-only.** Never modify or delete events. They are immutable facts. 2. **Self-contained.** Each event has enough context to be understood alone (no forward references). 3. **Cheap.** One `echo >>` per event. No database, no service, no dependencies. 4. **Optional.** If events dir doesn't exist, orchestration works fine without logging. Events are observation, not control flow.