Files
claude-archeflow-plugin/skills/process-log/SKILL.md
Christian Nennemann 1753e69a9f feat: add process logging with DAG-based event sourcing
Event-sourced orchestration logging: JSONL events with parent
relationships form a DAG for causal reconstruction of agent flows.
Includes bash event emitter (jq-based) and markdown report generator.
2026-04-03 11:06:02 +02:00

8.7 KiB

name, description
name description
process-log Event-based process logging for ArcheFlow orchestrations. Captures every phase transition, agent output, decision, and fix as structured JSONL events. Enables post-hoc reports, dashboards, and process archaeology. <example>Automatically loaded during orchestration</example> <example>User: "Show me how this story was made"</example>

Process Log — Event-Sourced Orchestration History

Every ArcheFlow orchestration writes structured events to a JSONL file. Events are the single source of truth — all reports (Markdown, dashboards, timelines) are generated views.

Event Storage

.archeflow/events/<run-id>.jsonl     # One file per orchestration run
.archeflow/events/index.jsonl        # Run index (one line per run, for listing)

Run ID format: <date>-<slug> (e.g., 2026-04-03-der-huster)

When to Emit Events

Emit an event at each of these points during orchestration:

Moment Event Type Trigger
Orchestration starts run.start After workflow selection, before first agent
Agent spawned agent.start Before each Agent tool call
Agent completes agent.complete After each Agent returns
Phase transition phase.transition Plan→Do, Do→Check, Check→Act
Decision made decision Plot direction chosen, fix applied, workflow adapted
Review verdict review.verdict Guardian/Sage/Skeptic delivers verdict
Fix applied fix.applied After each edit that addresses a review finding
Cycle boundary cycle.boundary End of PDCA cycle, before next (or exit)
Shadow detected shadow.detected Shadow threshold triggered
Orchestration ends run.complete After final Act phase

Event Schema

Every event is one JSON line with these required fields:

{
  "ts": "2026-04-03T14:32:07Z",
  "run_id": "2026-04-03-der-huster",
  "seq": 4,
  "parent": [2],
  "type": "agent.complete",
  "phase": "plan",
  "agent": "creator",
  "data": { ... }
}
Field Type Description
ts ISO 8601 Timestamp
run_id string Unique run identifier
seq integer Monotonically increasing sequence number within run
parent int[] Seq numbers of causal parent events. Forms a DAG. [] for root events.
type string Event type (see table above)
phase string Current PDCA phase: plan, do, check, act
agent string or null Agent archetype that triggered the event
data object Event-type-specific payload (see below)

Parent Relationships (DAG)

The parent field turns the flat event stream into a directed acyclic graph (agent call graph). This enables:

  • Causal reconstruction: which agent output caused which downstream action
  • Parallel visualization: agents sharing a parent ran concurrently
  • Blame tracking: trace a fix back through review → draft → outline → research

Rules:

  • run.start has parent: [] (root node)
  • An agent has parent: [seq of event that triggered it]
  • A phase transition has parent: [seq of all completing events in prior phase]
  • A fix has parent: [seq of the review that found the issue]
  • A decision has parent: [seq of the agent that produced the alternatives]
  • Parallel agents share the same parent (fan-out), phase transitions collect them (fan-in)

Example DAG from a writing workflow:

#1 run.start []
├── #2 agent.complete (explorer)       [1]
│   └── #3 decision (plot direction)   [2]
├── #4 agent.complete (creator)        [2]     ← explorer informs creator
├── #5 phase.transition (plan→do)      [3,4]   ← fan-in
│   └── #6 agent.complete (maker)      [5]
├── #7 phase.transition (do→check)     [6]
│   ├── #8 review (guardian)           [7]     ← parallel (fan-out)
│   └── #9 review (sage)              [7]     ← parallel (fan-out)
├── #10 phase.transition (check→act)   [8,9]   ← fan-in
├── #11 fix (timeline)                 [8]     ← caused by guardian
├── #12 fix (voice drift)             [9]     ← caused by sage
└── #18 run.complete                   [17]

Event Payloads by Type

run.start

{
  "task": "Write short story 'Der Huster'",
  "workflow": "kurzgeschichte",
  "team": "story-development",
  "max_cycles": 2,
  "config": {
    "voice_profile": "vp-giesing-gschichten-v1",
    "persona": "giesinger",
    "target_words": 6000
  }
}

agent.start

{
  "archetype": "story-explorer",
  "model": "haiku",
  "prompt_summary": "Research premise, find emotional core, suggest 3 plot directions"
}

agent.complete

{
  "archetype": "story-explorer",
  "duration_ms": 87605,
  "tokens": 21645,
  "artifacts": ["docs/01-der-huster-research.md"],
  "summary": "3 plot directions developed, recommended C (Mo krank + Koffer)"
}

decision

{
  "what": "plot_direction",
  "chosen": "C — Mo krank + Koffer aus B",
  "alternatives": [
    {"id": "A", "label": "Mo ist weg", "reason_rejected": "Zu passiv für 6k-Story"},
    {"id": "B", "label": "Huster gehört nicht Mo", "reason_rejected": "Zu Krimi-nah"}
  ],
  "rationale": "Stärkster emotionaler Kern, passt zum Voice Profile"
}

review.verdict

{
  "archetype": "guardian",
  "verdict": "approved_with_fixes",
  "findings": [
    {"severity": "bug", "description": "Timeline: 'Montag' referenced but story starts Dienstag", "fix_required": true},
    {"severity": "recommendation", "description": "Gentrification monologue too long for Alex register", "fix_required": false}
  ]
}

fix.applied

{
  "source": "guardian",
  "finding": "Timeline: Montag → Dienstag",
  "file": "stories/01-der-huster.md",
  "line": 302,
  "before": "das Gegenteil von Montag",
  "after": "das Gegenteil von Dienstag"
}

phase.transition

{
  "from": "plan",
  "to": "do",
  "artifacts_so_far": ["research.md", "outline.md"],
  "notes": "Explorer recommended direction C, Creator produced 6-scene outline"
}

cycle.boundary

{
  "cycle": 1,
  "max_cycles": 2,
  "exit_condition": "all_approved",
  "met": true,
  "fixes_applied": 6,
  "next_action": "complete"
}

shadow.detected

{
  "archetype": "story-explorer",
  "shadow": "endless_research",
  "trigger": "output >2000 words without recommendation",
  "action": "correction_prompt_applied",
  "occurrence": 1
}

run.complete

{
  "status": "completed",
  "cycles": 1,
  "agents_total": 5,
  "fixes_total": 6,
  "shadows": 0,
  "duration_ms": 1295519,
  "artifacts": [
    "docs/01-der-huster-research.md",
    "docs/01-der-huster-outline.md",
    "stories/01-der-huster.md",
    "docs/01-der-huster-guardian-review.md",
    "docs/01-der-huster-sage-review.md",
    "docs/01-der-huster-process.md"
  ]
}

How to Emit Events

During orchestration, write events using this pattern:

# Append one event to the run's JSONL file
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","run_id":"RUN_ID","seq":SEQ,"type":"TYPE","phase":"PHASE","agent":"AGENT","data":{...}}' >> .archeflow/events/RUN_ID.jsonl

Or use the helper script:

./lib/archeflow-event.sh RUN_ID TYPE PHASE AGENT '{"key":"value"}'

The orchestration skill should call the event emitter at each trigger point listed in the table above.

Generating Reports

After orchestration completes (or during, for live progress):

# Generate markdown process report
./lib/archeflow-report.sh .archeflow/events/2026-04-03-der-huster.jsonl > docs/process-report.md

# List all runs
cat .archeflow/events/index.jsonl | jq -r '[.run_id, .status, .task] | @tsv'

Run Index

After each run.complete, append a summary line to .archeflow/events/index.jsonl:

{"run_id":"2026-04-03-der-huster","ts":"2026-04-03T16:00:00Z","task":"Write Der Huster","workflow":"kurzgeschichte","status":"completed","cycles":1,"agents":5,"fixes":6,"duration_ms":1295519}

Integration with Existing Skills

  • orchestration: Emit events at phase transitions and after each agent
  • shadow-detection: Emit shadow.detected when thresholds trigger
  • autonomous-mode: Use index.jsonl for session summaries instead of separate session-log
  • workflow-design: Custom workflows inherit logging automatically

Design Principles

  1. Append-only. Never modify or delete events. They are immutable facts.
  2. Self-contained. Each event has enough context to be understood alone (no forward references).
  3. Cheap. One echo >> per event. No database, no service, no dependencies.
  4. Optional. If events dir doesn't exist, orchestration works fine without logging. Events are observation, not control flow.