How to validate an AI agent's persistent effects before commit

Do not treat an approved tool call or a successful response as proof that the resulting state is approved. For a consequential transactional operation, specify allowed durable changes independently of the agent, hold the candidate changes, observe effects within a declared boundary, and commit only when the result matches. Save the decision so retries and downstream work cannot turn an unresolved attempt into apparent success.

An allowed action can have an unapproved result

Imagine an agent asked to mark an invoice paid. A reviewer approves the invoice-status update, and the SQL command succeeds. A trigger also inserts an unapproved notification. Checking only the command or invoice row misses the extra write. The EffectMatch paper uses this example to separate call approval from persistent-outcome approval. Authorization still decides whether the action may be attempted; outcome validation decides whether it left only permitted changes. See how to bind an approval to an exact agent action.

Name the three artifacts before execution

EffectMatch models approved and observed outcomes as comparable collections of changes. EXACT means complete observed effects equal approved effects; DIVERGED means a complete observation differs; INDETERMINATE means completeness cannot be established. “Complete” is relative to the certified backend profile and granularity: a write-count comparison does not prove value-level correctness.

Put comparison before commit where the backend permits it

1. Bind approval to this attempt. Record its source, target, relevant state and plan revision, expiry, and single execution occurrence. A changed invoice or reused approval needs another decision.
2. Declare and check the effect boundary. Identify tables, triggers, functions, and dispatch routes that may persist a change. Cover or block each reachable route; an unknown route is not “no effect.” The paper's PostgreSQL prototype certifies a restricted, preregistered profile, not arbitrary database behavior.
3. Hold and inspect the candidate. Execute in a private transaction where possible, collect direct and induced writes, and compare the result with approval at the chosen granularity. PostgreSQL transactions provide an all-or-nothing database unit; observation completeness requires additional machinery.
4. Commit only an exact, current result. If the invoice update also inserted an unapproved notification, roll back the held transaction. If observation is incomplete, persist uncertainty and block dependent work until a named recovery path resolves it.

A transaction makes a group of writes atomic; it does not decide whether the group is authorized. A post-hoc final-state test may expose a defect but cannot prevent an already-completed commit. For that distinct testing decision, see evaluating stateful agent outcomes.

Remote effects need reconciliation, not fictional rollback

A payment, email, or service mutation may be durable before the agent receives its response. No local transaction automatically undoes it. EffectMatch's GitHub study reads back registered issue fields and gates local continuation, but does not attribute every remote effect or reverse the remote change.

Design the tool contract around stable request identity, bounded readback or reconciliation, and a durable unknown or diverged state that blocks dependent work. Microsoft's compensating-transaction guidance describes undoing completed distributed steps as application-specific work that must account for concurrent changes, not a simple snapshot restore.

What the evidence supports

EffectMatch reports preserving clean executions and rejecting tested incorrect commits in controlled comparisons. Its broader reference comparison includes 206 public business tasks; independently specified approvals cover a smaller set of business and PostgreSQL tasks. Ablations examine comparison, observation, execution binding, and durable continuation records. These are mechanism tests, not a production failure rate or proof that every hidden effect can be found.

The PostgreSQL profile and other backend adapters have different control powers. The arXiv v1 results do not establish universal coverage. Keep the boundary and its granularity explicit in any release claim.