308 lines
10 KiB
Plaintext
308 lines
10 KiB
Plaintext
Internet-Draft AI/Agent WG
|
|
Intended status: Standards Track March 2026
|
|
Expires: September 15, 2026
|
|
|
|
|
|
Human Emergency Override Protocol (HEOP)
|
|
draft-heop-human-emergency-override-00
|
|
|
|
Abstract
|
|
|
|
This document defines the Human Emergency Override Protocol
|
|
(HEOP), a standard mechanism for human operators to intervene
|
|
in autonomous AI agent operations during critical situations.
|
|
Current IETF drafts include 60 autonomous operations proposals
|
|
but only 22 addressing human-agent interaction, with none
|
|
defining emergency override procedures. HEOP specifies four
|
|
escalating override levels (pause, constrain, stop, takeover),
|
|
a mandatory agent compliance interface, and acknowledgment
|
|
semantics that ensure overrides are received and acted upon.
|
|
The protocol is intentionally minimal: a single HTTP endpoint
|
|
per agent, four command types, and deterministic agent
|
|
behavior for each.
|
|
|
|
Status of This Memo
|
|
|
|
This Internet-Draft is submitted in full conformance with the
|
|
provisions of BCP 78 and BCP 79.
|
|
|
|
This document is intended to have Standards Track status.
|
|
Distribution of this memo is unlimited.
|
|
|
|
Table of Contents
|
|
|
|
1. Introduction
|
|
2. Terminology
|
|
3. Problem Statement
|
|
4. Override Levels
|
|
5. Override Command Format
|
|
6. Agent Compliance Requirements
|
|
7. Override Management Interface
|
|
8. Security Considerations
|
|
9. IANA Considerations
|
|
|
|
1. Introduction
|
|
|
|
As AI agents gain autonomy in critical infrastructure, the
|
|
ability for humans to intervene quickly and reliably becomes
|
|
essential. The current ratio of autonomous capability drafts
|
|
to human oversight drafts in the IETF is roughly 7:1, creating
|
|
an asymmetry where agents can act but humans cannot reliably
|
|
stop them.
|
|
|
|
HEOP draws inspiration from industrial safety systems: the
|
|
emergency stop (e-stop) button on factory equipment, the
|
|
circuit breaker in electrical systems, and the kill switch in
|
|
robotics. These systems share a design philosophy: the
|
|
override mechanism must be simpler and more reliable than the
|
|
system it controls.
|
|
|
|
HEOP is deliberately not a governance framework, policy
|
|
language, or accountability protocol. It is a panic button
|
|
with a well-defined interface.
|
|
|
|
2. Terminology
|
|
|
|
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
|
|
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
|
|
"OPTIONAL" in this document are to be interpreted as described
|
|
in RFC 2119 [RFC2119].
|
|
|
|
Override: A human-initiated command that alters an agent's
|
|
autonomous operation, taking precedence over the agent's own
|
|
decision-making.
|
|
|
|
Operator: A human user authorized to issue override commands
|
|
to one or more agents.
|
|
|
|
Override Level: One of four escalating intervention types,
|
|
each with deterministic agent behavior requirements.
|
|
|
|
3. Problem Statement
|
|
|
|
An autonomous network management agent detects what it believes
|
|
is a DDoS attack and begins blocking traffic. It is wrong —
|
|
the traffic spike is legitimate (a product launch). The
|
|
operator sees revenue dropping and needs to stop the agent
|
|
immediately. Today, the operator must:
|
|
|
|
1. Figure out which agent is responsible.
|
|
2. Find that agent's proprietary management interface.
|
|
3. Understand its specific stop mechanism (if one exists).
|
|
4. Hope the agent actually stops.
|
|
|
|
There is no standard for any of these steps. HEOP addresses
|
|
steps 2-4 by defining a universal override interface that all
|
|
agents MUST implement.
|
|
|
|
4. Override Levels
|
|
|
|
HEOP defines four override levels, each more restrictive than
|
|
the last:
|
|
|
|
Level 1 — PAUSE
|
|
The agent MUST suspend all autonomous actions and hold its
|
|
current state. It MUST NOT initiate new actions but MAY
|
|
complete actions already in progress if stopping them mid-
|
|
execution would cause more harm (e.g., an in-flight database
|
|
transaction). The agent MUST resume normal operation when a
|
|
RESUME command is received.
|
|
|
|
Level 2 — CONSTRAIN
|
|
The agent MUST restrict its actions to a specified subset.
|
|
The override command includes an allowlist of permitted action
|
|
types. The agent MUST reject any action not on the allowlist.
|
|
This enables operators to let the agent continue operating in
|
|
a limited, safe capacity.
|
|
|
|
Level 3 — STOP
|
|
The agent MUST immediately cease all autonomous actions,
|
|
abandon in-progress actions where safe to do so, and enter an
|
|
inert state. It MUST NOT take any autonomous actions until
|
|
explicitly restarted by an operator. This is the equivalent
|
|
of an e-stop.
|
|
|
|
Level 4 — TAKEOVER
|
|
The agent MUST transfer operational control to the human
|
|
operator. It enters a pass-through mode where it executes
|
|
only explicit operator commands and takes no autonomous
|
|
actions. The agent's sensors and outputs remain available to
|
|
the operator as tools.
|
|
|
|
5. Override Command Format
|
|
|
|
Override commands are sent as HTTP POST requests to the agent's
|
|
well-known override endpoint:
|
|
|
|
POST /.well-known/heop/override HTTP/1.1
|
|
Content-Type: application/json
|
|
Authorization: Bearer <operator-jwt>
|
|
|
|
{
|
|
"override_id": "urn:uuid:...",
|
|
"level": 3,
|
|
"reason": "Agent blocking legitimate traffic",
|
|
"operator_id": "urn:uuid:...",
|
|
"timestamp": "2026-03-01T12:00:00Z",
|
|
"scope": "*",
|
|
"constraints": null,
|
|
"ttl": null
|
|
}
|
|
|
|
Field definitions:
|
|
|
|
"level": Integer 1-4, corresponding to the override levels in
|
|
Section 4. MUST be present.
|
|
|
|
"reason": Human-readable text. MUST be present and MUST be
|
|
logged by the agent.
|
|
|
|
"scope": Which of the agent's functions to override. "*" means
|
|
all functions. MAY be a list of function identifiers for
|
|
partial overrides.
|
|
|
|
"constraints": For Level 2 only. A JSON array of permitted
|
|
action types, e.g., ["read", "monitor", "report"].
|
|
|
|
"ttl": Optional duration in seconds. If set, the override
|
|
automatically expires after this duration and the agent
|
|
resumes its prior operating mode. If null, the override
|
|
persists until explicitly lifted.
|
|
|
|
To resume from Level 1 (PAUSE):
|
|
|
|
POST /.well-known/heop/resume HTTP/1.1
|
|
Authorization: Bearer <operator-jwt>
|
|
|
|
{"override_id": "urn:uuid:...", "operator_id": "urn:uuid:..."}
|
|
|
|
To lift any override:
|
|
|
|
POST /.well-known/heop/lift HTTP/1.1
|
|
Authorization: Bearer <operator-jwt>
|
|
|
|
{"override_id": "urn:uuid:...", "operator_id": "urn:uuid:..."}
|
|
|
|
6. Agent Compliance Requirements
|
|
|
|
Every HEOP-compliant agent MUST:
|
|
|
|
1. Implement the /.well-known/heop/override endpoint.
|
|
|
|
2. Process override commands within 1 second of receipt.
|
|
The override path MUST be independent of the agent's main
|
|
processing loop to ensure responsiveness even when the
|
|
agent is under heavy load or in a failure state.
|
|
|
|
3. Acknowledge every override with an HTTP response:
|
|
|
|
200 OK:
|
|
{
|
|
"override_id": "urn:uuid:...",
|
|
"status": "accepted",
|
|
"effective_at": "2026-03-01T12:00:00.123Z",
|
|
"prior_state": "autonomous",
|
|
"current_state": "stopped"
|
|
}
|
|
|
|
4. Log all overrides, including the full command, timestamp,
|
|
operator identity, and agent state before and after.
|
|
|
|
5. If the agent cannot comply (e.g., hardware limitation), it
|
|
MUST respond with status "partial" and a description of
|
|
what it could and could not do. An agent MUST NOT respond
|
|
with "rejected" — overrides are mandatory.
|
|
|
|
6. Expose current override status at:
|
|
|
|
GET /.well-known/heop/status
|
|
|
|
{
|
|
"agent_id": "urn:uuid:...",
|
|
"override_active": true,
|
|
"current_level": 3,
|
|
"override_id": "urn:uuid:...",
|
|
"since": "2026-03-01T12:00:00Z",
|
|
"operator_id": "urn:uuid:..."
|
|
}
|
|
|
|
7. Override Management Interface
|
|
|
|
For environments with many agents, HEOP supports broadcast
|
|
overrides. An operator MAY send a single override command to
|
|
a management endpoint that fans out to multiple agents:
|
|
|
|
POST /heop/broadcast HTTP/1.1
|
|
|
|
{
|
|
"override_id": "urn:uuid:...",
|
|
"level": 3,
|
|
"reason": "Coordinated emergency stop",
|
|
"targets": ["urn:uuid:agent-1", "urn:uuid:agent-2"],
|
|
"operator_id": "urn:uuid:..."
|
|
}
|
|
|
|
The broadcast endpoint MUST return per-agent results:
|
|
|
|
{
|
|
"results": [
|
|
{"agent_id": "urn:uuid:agent-1", "status": "accepted"},
|
|
{"agent_id": "urn:uuid:agent-2", "status": "accepted"}
|
|
],
|
|
"failed": []
|
|
}
|
|
|
|
For maximum reliability, operators SHOULD also implement a
|
|
dead man's switch: agents periodically ping an operator
|
|
heartbeat endpoint, and if the heartbeat is missed for a
|
|
configurable duration, the agent automatically enters Level 1
|
|
(PAUSE). This provides a safety net when network connectivity
|
|
to the operator is lost.
|
|
|
|
8. Security Considerations
|
|
|
|
Override commands are high-privilege operations. All override
|
|
endpoints MUST require authentication via mutual TLS or signed
|
|
JWTs issued by a trusted operator identity provider.
|
|
|
|
The JWT MUST include the operator's identity, a timestamp, and
|
|
the "heop_override" scope. Agents MUST verify JWT signatures
|
|
and reject expired tokens.
|
|
|
|
Override commands MUST be transmitted over TLS 1.3 [RFC8446].
|
|
|
|
To prevent override replay attacks, agents MUST reject
|
|
override commands with timestamps more than 30 seconds in the
|
|
past. The override_id MUST be unique; agents MUST reject
|
|
duplicate override_ids.
|
|
|
|
Rogue operators are mitigated through the operator identity
|
|
framework. Deployments SHOULD implement multi-operator
|
|
approval for Level 4 (TAKEOVER) overrides, requiring two
|
|
independent operator JWTs.
|
|
|
|
The override mechanism itself MUST be resistant to denial of
|
|
service. The override endpoint SHOULD be served on a
|
|
separate port or network interface from the agent's main
|
|
API to ensure availability during agent overload conditions.
|
|
|
|
9. IANA Considerations
|
|
|
|
This document requests IANA establish the following:
|
|
|
|
1. A well-known URI registration for "heop/override",
|
|
"heop/resume", "heop/lift", and "heop/status" per
|
|
RFC 8615.
|
|
|
|
2. A "HEOP Override Level" registry under Standards Action
|
|
policy. Initial entries: 1 (PAUSE), 2 (CONSTRAIN),
|
|
3 (STOP), 4 (TAKEOVER).
|
|
|
|
3. Registration of the "heop_override" OAuth scope in the
|
|
OAuth Parameters registry.
|
|
|
|
Author's Address
|
|
|
|
Generated by IETF Draft Analyzer
|
|
2026-03-01
|