Writing
Mechanism-level pieces on agent reliability, AgentOps, MCP servers, and the harness that keeps AI systems honest in production.
· agent safety · authorization · AgentOps
For a consequential agent action, a missing policy decision must mean do not perform the action: bound the policy call with a timeout, accept only a validated verdict, and turn a missing, invalid, or failed decision into a durable, recorded deny.
Read →· agent safety · agent workflows · AgentOps
Choose review or autonomous execution for each AI operations workflow based on action boundaries, permissions, recovery, and evidence—not a blanket setting for the entire agent.
Read →· OpenAI · agent evaluation · AgentOps
Treat a moving model alias such as chat-latest as a discovery and evaluation input, not a production pin: gate any change behind a pinned snapshot, a recorded evaluation, a rollout decision, and a rollback path.
Read →· OpenAI · API security · AgentOps
Treat an expiring OpenAI project API key as a planned cutover: inventory consumers, create a scoped replacement before expiry, verify each workload after cutover, and revoke the old key only when the evidence is complete.
Read →· OpenAI · agent workflows · AgentOps
Use the OpenAI Agents API when an OpenAI-managed Codex harness and durable sessions remove mechanics you do not need to operate; keep tool authority, execution-environment choice, application state, and consequential-action policy explicit in your application.
Read →· MCP · gateway · AgentOps
Use the MCP 2026-07-28 Streamable HTTP headers to make a narrow edge classification decision — version, RPC method, and named tool or resource — then let the MCP server reject header/body mismatches, validate the request and token, authorize the operation, and enforce argument- and side-effect-specific controls.
Read →· OpenAI · agent workflows · AgentOps
Use async tool calling only when useful model work can continue while your application owns the slow job — correlate the result by call_id, keep cancellation, compensation, and relevance decisions outside the model, and treat mid-turn steering as queued input, not a rollback of tool work already started.
Read →· MCP · protocol · compatibility
MCP 2026-07-28 removes the initialize handshake for a stateless, per-request protocol — detect a peer's era with the documented probes, keep authorization failures separate from version negotiation, and hold traffic until the still-unreleased revision is verified.
Read →· OpenAI · agent reliability · AgentOps
Branch retry control on the documented failure meaning, not the HTTP status: reduce ramp after a 429 slow_down, use exponential backoff for a temporary 503 server_is_overloaded, and honor Retry-After when either is present.
Read →· agent evaluation · agent safety · AgentOps
Treat an agent evaluation as an untrusted security zone: use a separate identity, default-deny egress and shared-service access, broker only required dependencies, and test escape paths before the run.
Read →· agent memory · agent safety · AgentOps
Keep provenance and confirmation attached through memory consolidation, bind that evidence to action-relevant tool arguments, and require authority appropriate to the action risk.
Read →· agent safety · agent workflows · MCP · AgentOps
Treat each agent handoff as an explicit, bounded delegation: narrow the resource and scope, set expiry and budget, then enforce the envelope outside the model before a consequential tool action.
Read →· agent safety · MCP · cryptographic signing · AgentOps
Keep a private signing key inside hardware, then require identity, scope, semantic, and taint checks before an AI agent's request can reach the signature boundary.
Read →· agent evaluation · agent reliability · AgentOps
Evaluate a state-changing agent by its durable postcondition, allowed execution path, and prohibited side effects—then repeat controlled cases before treating the workflow as dependable.
Read →· voice agents · agent reliability · AgentOps
Keep the live media loop separate from slow background work, retain authoritative session and job state outside it, and make pending, cancellation, recovery, and relevance decisions explicit.
Read →· coding agents · agent workflows · AgentOps
Treat a specification from one coding agent as an input the receiving agent must accept against representative work and required checks; retain the handoff only when its implementation meets the agreed acceptance criteria.
Read →· OpenAI · agent observability · cost controls · AgentOps
Baseline cache reads and writes by model and service tier, then change one request-shape or routing variable at a time; retain a change only when cache evidence and accepted task outcome support it.
Read →· agent memory · agent reliability · AgentOps
When validated evidence supersedes a memory under a stable key, remove the old record from ordinary retrieval, preserve its revocation trail for audit, and expose only active evidence by default.
Read →· agent observability · agent safety · AgentOps
Treat agent telemetry as a governed data-export path: keep content capture deliberate, separate restricted traces from broad health signals, and test where representative sensitive data actually lands before release.
Read →· agent evaluation · agent reliability · AgentOps
Compare a compressed run with a matched full-context baseline, then measure accepted completion, retrieval and execution work, latency, and cost separately before widening the policy.
Read →· agent evaluation · agent reliability · AgentOps
Keep the release decision outside the optimization loop: reserve acceptance evidence, meter the search budget, retain the exact candidate and its evaluation record, then run an independent release check.
Read →· agent memory · agent reliability · AgentOps
Manage reusable agent experience as a feedback-controlled asset: admit it selectively, preserve provenance and later outcome evidence, and retire records whose reuse is associated with poor downstream results.
Read →· OpenAI · cost controls · AgentOps
Assign one documented API key to one workload, then use OpenAI API-key filtering and grouping to inspect provider-recorded usage and cost without mistaking that ledger for full workflow economics.
Read →· agent reliability · observability · AgentOps
Find the earliest failure that matters by reviewing an evidence-backed error lifecycle—not by selecting the first bad-looking step or the final symptom.
Read →· coding agents · agent reliability · AgentOps
Treat a user edit as a plan-invalidating state change: pause writes, preserve and inspect the diff, identify affected assumptions and checks, then re-plan or escalate before continuing.
Read →· MCP · agent reliability · AgentOps
Use interaction history in MCP tool routing only after resolving the current authorized tool set: record the history and candidates behind a choice, retain a stateless fallback, and test whether added state improves accepted outcomes.
Read →· agent safety · agent evaluation · AgentOps
Test instruction hierarchy at the action boundary: declare the protected constraint, inject user and tool-output conflicts, and assert an allowed action, refusal, or escalation before a side effect occurs.
Read →· OpenAI · agent reliability · AgentOps
Treat OpenAI Fast mode as an observable latency tier: request it deliberately, ramp gradually, inspect the returned service tier, and pause expansion when the returned path does not match the request.
Read →· agent workflows · AgentOps · orchestration
Use multiple agents only when a bounded relay preserves the facts the next decision needs; otherwise keep shared context or redesign and test the relay against a single-agent baseline.
Read →· OpenAI · cost controls · AgentOps
Treat an OpenAI hard spend limit as an intentional availability boundary: classify the quota condition, preserve the workflow checkpoint, and explicitly decide whether to restore service or wait.
Read →· agent reliability · AgentOps · agent evaluation
Stop a verify–repair loop when evidence no longer supports another revision: preserve the champion, enforce a budget, and escalate when the verifier or calibration cannot support a reliable choice.
Read →· agent evaluation · AgentOps · agent reliability
Evaluate an AI agent reviewer as a chain: correct detection, critique uptake, accepted repair, and cost. Reviewer precision alone does not prove that the final answer improved.
Read →· MCP · security · agent reliability
Keep a private MCP server non-public while giving an agent a controlled connection path—then retain authorization, tool policy, auditability, and recovery at the server boundary.
Read →· agent workflows · AgentOps · orchestration
Agent loops remain useful for bounded work. When a system needs parallel branches, approval gates, checkpoints, and scoped recovery, make the control flow explicit with an agent workflow graph.
Read →· OpenAI · agent security · identity
Use OpenAI Workload Identity Federation to exchange an external workload identity for a short-lived access token, with a narrow service-account mapping and explicit permissions.
Read →· observability · agent reliability · AgentOps
An agent observability dashboard should show run state, tool behavior, retry pressure, and spend together so operators can explain the workflow before they intervene.
Read →· agent reliability · AI readiness · AgentOps
An AI agent rollout checklist should prove scope, tool safety, recovery, review, telemetry, and cost controls before the workflow runs without a human watching every step.
Read →· AgentOps · incident response · agent reliability
AgentOps incident response starts by freezing side effects, preserving state, classifying the failure, and choosing a recovery path before the agent resumes.
Read →· agent reliability · observability · failure modes
AI agents fail in production for boring harness reasons: vague tool contracts, lost state, bad retries, and missing observability.
Read →· MCP · tools · production
A production MCP server survives when tools, resources, authorization, and logging are explicit and the server owns the audit trail.
Read →· AgentOps · agent reliability · SRE
Treat production AI agents like SRE systems: checkpoints, idempotent writes, retry budgets, observability, and cost guards around the model.
Read →· agent reliability · governance · approvals
Human approval gates work when the harness freezes the exact action surface, preserves the reviewer decision, and invalidates stale approvals before resume.
Read →· agent reliability · recovery · checkpoints
An agent workflow recovery plan reconciles checkpoints, side effects, and approval state before a resumed run is allowed to act again.
Read →· AgentOps · agent reliability · SLOs
Agent reliability SLOs should measure finished work, recovery behavior, escalation, and cost per accepted output rather than model latency alone.
Read →· cost controls · agent reliability · runbooks
A cost runbook classifies the loop, contains the damage, and measures accepted output so routing and retry controls get tuned for the right failure mode.
Read →· agent reliability · governance · controls
A kill switch should pause the workflow, preserve the last safe checkpoint, and hand the decision back to a human before the agent keeps acting.
Read →· agent reliability · tool contracts · idempotency
A production agent tool contract separates intent, validation, execution, and audit so retries stay legible and side effects stay bounded.
Read →· agent reliability · AI readiness · scoping
An AI agent scope audit chooses the first workflow by recoverability: clear ownership, bounded tools, observable progress, and reviewable side effects.
Read →· AgentOps · agent reliability · services
AgentOps hardening services inspect workflow contracts, tool safety, recovery behavior, telemetry, and review boundaries before a production agent runs unattended.
Read →· cost controls · agent reliability · routing
Multi-model routing controls cost by making the harness choose the cheapest reliable lane for each agent step, then measuring accepted output instead of raw call spend.
Read →· agent reliability · idempotency · MCP
Make tool calls idempotent by separating intent, side effect, and acknowledgement — so a retry never double-charges, double-sends, or double-writes.
Read →· AgentOps · agent reliability · production AI
AgentOps is the discipline of making AI agents reliable in production — checkpoints, idempotent tool calls, retry budgets, cost guards, and observability. The mechanism, the metrics, and a hardening checklist.
Read →