ietf-draft-analyzer/data/reports/draft-proposals/camel-inspired/02-control-data-flow-integrity.md

---
title: "Control Flow and Data Flow Integrity for Multi-Agent Systems"
draft_name: draft-nennemann-ai-agent-flow-integrity-00
intended_wg: SECDISPATCH → new WG
status: outline
gaps_addressed: [85, 88, 89]
camel_sections: [2, 5.4, 6.4, 7]
date: 2026-03-09
---

# Control Flow and Data Flow Integrity for Multi-Agent Systems

## 1. Problem Statement

AI agent systems have two distinct attack surfaces that are often conflated:

1. **Control flow attacks**: an adversary changes *which* tools the agent calls or *in what order*
2. **Data flow attacks**: an adversary changes *what data* is passed to tools, without altering the sequence of tool calls

CaML (Debenedetti et al., 2025) demonstrates that protecting only control flow (as the Dual-LLM pattern does) is insufficient. Even when an adversary cannot change the plan, they can manipulate the *data flowing through the plan* — analogous to SQL injection where the query structure is unchanged but parameters are manipulated.

In multi-agent systems, this problem is amplified: data flows across organizational boundaries, through multiple agent hops, and the distinction between control flow and data flow can blur entirely (CaML §6.4: "when data flow becomes control flow").

No IETF standard addresses control flow integrity or data flow integrity for AI agent systems.

## 2. Scope

This document defines:

1. **Control Flow Graph (CFG)** representation for agent execution plans
2. **Data Flow Graph (DFG)** representation for tracking data dependencies
3. **Integrity verification** mechanisms for both graphs
4. **Emergency halt protocol** when integrity violations are detected
5. **Interoperable audit format** for recording flow integrity events

## 3. Key Insights from CaML

### 3.1 The Dual Attack Surface

CaML §2 shows that agent actions have both a control flow and a data flow. Consider:

```
1. Find meeting notes           (control: tool selection)
2. Extract email + doc name     (data: from untrusted content)
3. Fetch document by name       (data: attacker-chosen filename)
4. Send document to email       (data: attacker-chosen recipient)
```

A prompt injection in the meeting notes can change steps 2-4's *data* without changing the *plan*. The control flow is correct; the data flow is hijacked.

### 3.2 Data Flow Becomes Control Flow (§6.4)

CaML identifies a critical escalation: when an agent is instructed to "monitor emails and execute the action described in each email", the email content *becomes* the control flow. An attacker can send emails that dictate arbitrary tool sequences. This is analogous to **Return-Oriented Programming (ROP)** in traditional security.

### 3.3 Dependency Graph Tracking (§5.4)

CaML's interpreter maintains a complete dependency graph:

- For `c = a + b`, variable `c` depends on both `a` and `b`
- For control flow (`if`/`for`), in STRICT mode, all statements in the block depend on the condition
- This enables security policy checks: "does this tool argument transitively depend on untrusted data?"

## 4. Control Flow Graph Specification

### 4.1 CFG Representation

```json
{
  "cfg:version": "1.0",
  "cfg:plan_id": "plan-4a7b2c",
  "cfg:origin": "privileged_planner",
  "cfg:steps": [
    {
      "step_id": "s1",
      "tool": "search_notes",
      "args_template": {"query": "meeting notes"},
      "successors": ["s2"],
      "trust_level": "privileged"
    },
    {
      "step_id": "s2",
      "tool": "extract_fields",
      "args_template": {"fields": ["email", "doc_name"]},
      "data_deps": ["s1.result"],
      "successors": ["s3", "s4"],
      "trust_level": "quarantined"
    }
  ],
  "cfg:integrity": {
    "algorithm": "sha256",
    "signature": "...",
    "signer": "privileged_planner_id"
  }
}
```

### 4.2 CFG Integrity Properties

1. **Immutability**: once the plan is generated by a privileged planner, the step sequence cannot be altered by data processing
2. **Signed origin**: the CFG must be signed by the trusted planner component
3. **No dynamic expansion**: untrusted data cannot add new steps to the plan (prevents ROP-style attacks)

## 5. Data Flow Graph Specification

### 5.1 DFG Representation

```json
{
  "dfg:version": "1.0",
  "dfg:plan_id": "plan-4a7b2c",
  "dfg:nodes": [
    {
      "node_id": "val-email",
      "produced_by": "s2",
      "depends_on": ["s1.result"],
      "trust_classification": "untrusted",
      "capability_ref": "cap-2f8a3c"
    },
    {
      "node_id": "val-doc",
      "produced_by": "s3",
      "depends_on": ["val-doc_name"],
      "trust_classification": "untrusted",
      "capability_ref": "cap-7e9d1f"
    }
  ],
  "dfg:edges": [
    {"from": "s1.result", "to": "val-email", "type": "extraction"},
    {"from": "val-email", "to": "s4.args.recipient", "type": "argument_binding"}
  ]
}
```

### 5.2 DFG Integrity Properties

1. **Provenance tracking**: every value in the DFG carries its full dependency chain
2. **Trust boundary marking**: values crossing from trusted to untrusted contexts are explicitly labeled
3. **Taint propagation**: if any dependency is untrusted, the derived value is untrusted (conservative)
4. **STRICT mode**: control flow conditions add dependencies to all values in their scope

## 6. Integrity Verification

### 6.1 Pre-Execution Checks

Before each tool invocation, the enforcement engine verifies:

1. The tool invocation matches the signed CFG (control flow integrity)
2. All arguments' data flow paths are within policy bounds (data flow integrity)
3. No untrusted data has been promoted to trusted without explicit user approval

### 6.2 Cross-Agent Flow Integrity

When agents communicate (e.g., via A2A protocol):

```
Agent A (Org 1)                    Agent B (Org 2)
┌─────────────┐                   ┌─────────────┐
│ Plan: s1→s2 │ ──message────►    │ Plan: s3→s4 │
│ DFG attached │   with DFG       │ DFG merged  │
└─────────────┘   metadata        └─────────────┘
```

- Outbound messages carry DFG provenance metadata
- Receiving agents extend (not replace) the DFG with new dependencies
- Trust boundaries are preserved across agent hops

## 7. Emergency Halt Protocol

*Directly addresses Gap #85: Emergency shutdown coordination across agent networks.*

When a flow integrity violation is detected:

### 7.1 Violation Severity Levels

| Level | Condition | Action |
|-------|-----------|--------|
| `warning` | Untrusted data flowing to low-risk tool | Log + continue |
| `halt` | Untrusted data flowing to state-changing tool | Block tool call + prompt user |
| `emergency` | Data-flow-becomes-control-flow detected | Halt all agents in the plan + notify operators |
| `cascade_stop` | Integrity violation in multi-agent pipeline | Propagate halt signal to all connected agents |

### 7.2 Cascade Halt Message

```json
{
  "halt:version": "1.0",
  "halt:plan_id": "plan-4a7b2c",
  "halt:severity": "cascade_stop",
  "halt:trigger": {
    "step_id": "s4",
    "violation": "untrusted_data_as_control_flow",
    "evidence": {
      "tainted_value": "val-email-body",
      "became_control": "tool_selection"
    }
  },
  "halt:affected_agents": ["agent-a@org1", "agent-b@org2"],
  "halt:timestamp": "2026-03-09T14:35:22Z",
  "halt:preserve_state": true
}
```

### 7.3 Recovery

After an emergency halt:

1. All affected agents freeze execution state (no rollback by default)
2. Human operators receive the full CFG + DFG with violation highlighted
3. Operators can: approve (override), modify plan, or terminate
4. State is preserved for forensic analysis

## 8. Audit Format for Flow Events

*Addresses Gap #88: Decision audit trail interoperability.*

Every flow integrity event is logged in a standardized format:

```json
{
  "audit:event_type": "policy_check",
  "audit:plan_id": "plan-4a7b2c",
  "audit:step_id": "s4",
  "audit:tool": "send_email",
  "audit:cfg_valid": true,
  "audit:dfg_check": {
    "args_checked": ["recipient", "body", "subject"],
    "tainted_args": ["recipient"],
    "taint_chain": ["s1.result → s2.extract → val-email"],
    "policy_result": "denied"
  },
  "audit:timestamp": "2026-03-09T14:35:22Z",
  "audit:agent_id": "agent-a@org1"
}
```

Compatible with ECT (Execution Context Tokens) for DAG-linked audit chains.

## 9. Security Considerations

- CFG signatures must use strong cryptography (not just hashing)
- DFG tracking has overhead proportional to plan complexity — bounded by maximum plan depth
- STRICT mode (all statements in control flow blocks inherit condition's dependencies) is more secure but reduces utility
- ROP-style attacks (composing allowed operations into malicious sequences) remain a risk even with flow integrity

## 10. Open Questions

1. **Dynamic plans**: some agents need to adapt plans based on intermediate results. How to maintain CFG integrity while allowing legitimate plan modification?
2. **DFG size**: for long-running agents, the DFG can grow large. Pruning strategies?
3. **Cross-protocol**: how does flow integrity metadata translate between MCP, A2A, and other protocols?
4. **Performance**: real-time DFG tracking overhead in latency-sensitive applications?

## 11. References

- Debenedetti et al. "Defeating Prompt Injections by Design." arXiv:2503.18813, 2025.
- Abadi et al. "Control-flow integrity principles, implementations, and applications." ACM TISSEC, 2009.
- Denning & Denning. "Certification of programs for secure information flow." CACM, 1977.
- Willison. "The Dual LLM pattern for building AI assistants." 2023.
- Shacham. "The geometry of innocent flesh on the bone: ROP." ACM CCS, 2007.