Writing ·
How to handle AI agent tool calls when the interface changes
Validate calls against the current authorized tool contract; use bounded retry or argument repair for detectable drift, then recheck intent and safety before execution.
By Youssef Hemimy · agent reliability · tool contracts · MCP
Treat a changed tool list or schema as a release regression. Check a model-generated call against the current authorized contract before execution. A missing name may receive a bounded retry over allowed candidates; invalid arguments may receive one constrained repair. Revalidate the result, and stop when intent or authority is still uncertain.
01
Current contract
Snapshot authorized names, schemas, and version.
02
Validate call
Reject unknown names or malformed arguments before execution.
03
Bound repair
Retry missing names against allowed candidates; repair invalid arguments once.
04
Recheck intent
Revalidate syntax and independently check authority and effect.
A valid repaired call is not automatically the right or authorized call.
What actually drifts?
A sound production tool contract can still be presented alongside stale field examples, renamed parameters, or newly added tools with overlapping descriptions. The MCP Tools specification makes these surfaces concrete: clients discover tool names, descriptions, and input schemas with tools/list; a server may advertise list-change notifications.
The Robust Function Calling preprint studies three semantics-preserving perturbations: stale field-name examples in descriptions, renamed parameters, and expanded candidate sets with distractors. A removed capability, changed permission, or new side effect is not just interface drift; it is a contract or authorization change. In one Qwen3.5-9B DICE-300 ablation, a distractor tool caused most of the measured accuracy loss. That specific one-tool-baseline experiment does not make parameter renames harmless elsewhere.
Put bounded repair behind a versioned contract
- Snapshot the current authorized interface. Record allowed names, descriptions, schemas, and version. A list-change signal prompts refresh, not blanket authorization of newly listed tools.
- Validate before execution. Check the proposed name against that set and its arguments against the current schema. Return a structured mismatch rather than guessing.
- Bound the retry. A missing name may be retried against only visible, authorized candidates. A resolved call with invalid arguments may receive one constrained repair. Revalidate the result. The paper's missing-name path can use a candidate retry and then a repair call, so “one-shot repair” does not always mean one extra model call.
- Fail safely on uncertainty. Valid shape does not prove correct tool choice, values, or authority. Preserve approval and idempotent-call safeguards separately; clarify, review, or use a known-safe fallback if the effect is unclear.
| Detectable surface error | Decision after validation |
|---|---|
| Missing tool name | Retry once over current authorized candidates; stop if unresolved. |
| Invalid argument object | Constrain one repair to the current schema, then revalidate. |
| Valid but wrong tool or value | Schema checks cannot detect intent; use semantic review or safe fallback. |
In the study's main non-oracle Qwen3.5-9B runs, accuracy rose from 0.717 to 0.813 on DICE-300 and from 0.580 to 0.805 on BFCL-200 after repair. No harms were observed on originally correct slices in those runs. These are benchmark outcomes under constructed drift, not production success or safety rates; validator-invisible valid-but-wrong calls and unresolved missing-name or missing-field failures remained.
Test the release decision, not just the syntax
Replay representative workflow calls under old and proposed interfaces. Include renamed fields, similar neighboring tools, missing names, missing required fields, and valid-but-wrong calls. Keep the expected authorized tool and outcome evaluator-side, outside the repair prompt. Compare the base call, an identical retry, and bounded validation-and-repair. Record correct task outcomes, wrong-tool executions, mistaken repairs, unresolved calls, approvals, token cost, and tail latency.
In the paper's two main runs, repair fired on 16% of DICE cases and 32% of BFCL cases. Repair-token overhead was 32.6% and 47.1% relative to drifted-call tokens; a separate BFCL latency rerun estimated about 3.5 seconds added per example. Budget and measure locally instead of adopting those figures as service-level targets. Preserve the contract version, original call, validator report, retry, and final execution decision. For the earlier problem of choosing among authorized tools using prior context, see history-aware MCP routing.
Where the evidence stops
The preprint studies one primary model family on benchmark slices, not a live deployment. A Llama-4-Scout BFCL check recovered few originally correct drift misses because missing names remained unresolved; a GPT-4o-mini check had a small, statistically inconclusive gain. Type narrowing, enum replacement, and multi-tool refactors were not tested. The authors warn that repair might mask malicious candidate additions.
Sources
FAQ
What should happen when an agent's tool schema changes?
Validate against the current authorized schema before execution. Permit one constrained argument repair for a detectable mismatch, revalidate, and use a safe fallback if the intended effect remains uncertain.
Can a valid repaired call be executed automatically?
Not solely because its shape is valid. Check the selected tool, values, authorization, approvals, and side-effect protections independently.
Does the interface-drift study prove this works in production?
No. Its main results come from constructed drift on Qwen3.5-9B benchmark slices. Measure recovery, mistaken repairs, cost, and latency on your own workflow before rollout.
Building something that has to hold up?
We do this work for teams — agent reliability hardening, custom MCP servers, and full-stack AI systems built to survive production.