Event-sourced orchestration logging: JSONL events with parent relationships form a DAG for causal reconstruction of agent flows. Includes bash event emitter (jq-based) and markdown report generator.
8.7 KiB
name, description
| name | description |
|---|---|
| process-log | Event-based process logging for ArcheFlow orchestrations. Captures every phase transition, agent output, decision, and fix as structured JSONL events. Enables post-hoc reports, dashboards, and process archaeology. <example>Automatically loaded during orchestration</example> <example>User: "Show me how this story was made"</example> |
Process Log — Event-Sourced Orchestration History
Every ArcheFlow orchestration writes structured events to a JSONL file. Events are the single source of truth — all reports (Markdown, dashboards, timelines) are generated views.
Event Storage
.archeflow/events/<run-id>.jsonl # One file per orchestration run
.archeflow/events/index.jsonl # Run index (one line per run, for listing)
Run ID format: <date>-<slug> (e.g., 2026-04-03-der-huster)
When to Emit Events
Emit an event at each of these points during orchestration:
| Moment | Event Type | Trigger |
|---|---|---|
| Orchestration starts | run.start |
After workflow selection, before first agent |
| Agent spawned | agent.start |
Before each Agent tool call |
| Agent completes | agent.complete |
After each Agent returns |
| Phase transition | phase.transition |
Plan→Do, Do→Check, Check→Act |
| Decision made | decision |
Plot direction chosen, fix applied, workflow adapted |
| Review verdict | review.verdict |
Guardian/Sage/Skeptic delivers verdict |
| Fix applied | fix.applied |
After each edit that addresses a review finding |
| Cycle boundary | cycle.boundary |
End of PDCA cycle, before next (or exit) |
| Shadow detected | shadow.detected |
Shadow threshold triggered |
| Orchestration ends | run.complete |
After final Act phase |
Event Schema
Every event is one JSON line with these required fields:
{
"ts": "2026-04-03T14:32:07Z",
"run_id": "2026-04-03-der-huster",
"seq": 4,
"parent": [2],
"type": "agent.complete",
"phase": "plan",
"agent": "creator",
"data": { ... }
}
| Field | Type | Description |
|---|---|---|
ts |
ISO 8601 | Timestamp |
run_id |
string | Unique run identifier |
seq |
integer | Monotonically increasing sequence number within run |
parent |
int[] | Seq numbers of causal parent events. Forms a DAG. [] for root events. |
type |
string | Event type (see table above) |
phase |
string | Current PDCA phase: plan, do, check, act |
agent |
string or null | Agent archetype that triggered the event |
data |
object | Event-type-specific payload (see below) |
Parent Relationships (DAG)
The parent field turns the flat event stream into a directed acyclic graph (agent call graph). This enables:
- Causal reconstruction: which agent output caused which downstream action
- Parallel visualization: agents sharing a parent ran concurrently
- Blame tracking: trace a fix back through review → draft → outline → research
Rules:
run.starthasparent: [](root node)- An agent has
parent: [seq of event that triggered it] - A phase transition has
parent: [seq of all completing events in prior phase] - A fix has
parent: [seq of the review that found the issue] - A decision has
parent: [seq of the agent that produced the alternatives] - Parallel agents share the same parent (fan-out), phase transitions collect them (fan-in)
Example DAG from a writing workflow:
#1 run.start []
├── #2 agent.complete (explorer) [1]
│ └── #3 decision (plot direction) [2]
├── #4 agent.complete (creator) [2] ← explorer informs creator
├── #5 phase.transition (plan→do) [3,4] ← fan-in
│ └── #6 agent.complete (maker) [5]
├── #7 phase.transition (do→check) [6]
│ ├── #8 review (guardian) [7] ← parallel (fan-out)
│ └── #9 review (sage) [7] ← parallel (fan-out)
├── #10 phase.transition (check→act) [8,9] ← fan-in
├── #11 fix (timeline) [8] ← caused by guardian
├── #12 fix (voice drift) [9] ← caused by sage
└── #18 run.complete [17]
Event Payloads by Type
run.start
{
"task": "Write short story 'Der Huster'",
"workflow": "kurzgeschichte",
"team": "story-development",
"max_cycles": 2,
"config": {
"voice_profile": "vp-giesing-gschichten-v1",
"persona": "giesinger",
"target_words": 6000
}
}
agent.start
{
"archetype": "story-explorer",
"model": "haiku",
"prompt_summary": "Research premise, find emotional core, suggest 3 plot directions"
}
agent.complete
{
"archetype": "story-explorer",
"duration_ms": 87605,
"tokens": 21645,
"artifacts": ["docs/01-der-huster-research.md"],
"summary": "3 plot directions developed, recommended C (Mo krank + Koffer)"
}
decision
{
"what": "plot_direction",
"chosen": "C — Mo krank + Koffer aus B",
"alternatives": [
{"id": "A", "label": "Mo ist weg", "reason_rejected": "Zu passiv für 6k-Story"},
{"id": "B", "label": "Huster gehört nicht Mo", "reason_rejected": "Zu Krimi-nah"}
],
"rationale": "Stärkster emotionaler Kern, passt zum Voice Profile"
}
review.verdict
{
"archetype": "guardian",
"verdict": "approved_with_fixes",
"findings": [
{"severity": "bug", "description": "Timeline: 'Montag' referenced but story starts Dienstag", "fix_required": true},
{"severity": "recommendation", "description": "Gentrification monologue too long for Alex register", "fix_required": false}
]
}
fix.applied
{
"source": "guardian",
"finding": "Timeline: Montag → Dienstag",
"file": "stories/01-der-huster.md",
"line": 302,
"before": "das Gegenteil von Montag",
"after": "das Gegenteil von Dienstag"
}
phase.transition
{
"from": "plan",
"to": "do",
"artifacts_so_far": ["research.md", "outline.md"],
"notes": "Explorer recommended direction C, Creator produced 6-scene outline"
}
cycle.boundary
{
"cycle": 1,
"max_cycles": 2,
"exit_condition": "all_approved",
"met": true,
"fixes_applied": 6,
"next_action": "complete"
}
shadow.detected
{
"archetype": "story-explorer",
"shadow": "endless_research",
"trigger": "output >2000 words without recommendation",
"action": "correction_prompt_applied",
"occurrence": 1
}
run.complete
{
"status": "completed",
"cycles": 1,
"agents_total": 5,
"fixes_total": 6,
"shadows": 0,
"duration_ms": 1295519,
"artifacts": [
"docs/01-der-huster-research.md",
"docs/01-der-huster-outline.md",
"stories/01-der-huster.md",
"docs/01-der-huster-guardian-review.md",
"docs/01-der-huster-sage-review.md",
"docs/01-der-huster-process.md"
]
}
How to Emit Events
During orchestration, write events using this pattern:
# Append one event to the run's JSONL file
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","run_id":"RUN_ID","seq":SEQ,"type":"TYPE","phase":"PHASE","agent":"AGENT","data":{...}}' >> .archeflow/events/RUN_ID.jsonl
Or use the helper script:
./lib/archeflow-event.sh RUN_ID TYPE PHASE AGENT '{"key":"value"}'
The orchestration skill should call the event emitter at each trigger point listed in the table above.
Generating Reports
After orchestration completes (or during, for live progress):
# Generate markdown process report
./lib/archeflow-report.sh .archeflow/events/2026-04-03-der-huster.jsonl > docs/process-report.md
# List all runs
cat .archeflow/events/index.jsonl | jq -r '[.run_id, .status, .task] | @tsv'
Run Index
After each run.complete, append a summary line to .archeflow/events/index.jsonl:
{"run_id":"2026-04-03-der-huster","ts":"2026-04-03T16:00:00Z","task":"Write Der Huster","workflow":"kurzgeschichte","status":"completed","cycles":1,"agents":5,"fixes":6,"duration_ms":1295519}
Integration with Existing Skills
orchestration: Emit events at phase transitions and after each agentshadow-detection: Emitshadow.detectedwhen thresholds triggerautonomous-mode: Useindex.jsonlfor session summaries instead of separate session-logworkflow-design: Custom workflows inherit logging automatically
Design Principles
- Append-only. Never modify or delete events. They are immutable facts.
- Self-contained. Each event has enough context to be understood alone (no forward references).
- Cheap. One
echo >>per event. No database, no service, no dependencies. - Optional. If events dir doesn't exist, orchestration works fine without logging. Events are observation, not control flow.