Writing ·
How to protect AI agent telemetry from sensitive prompts and tool data in 2026
Treat agent telemetry as a governed data-export path: keep content capture deliberate, separate restricted traces from broad health signals, and test where representative sensitive data actually lands before release.
By Youssef Hemimy · agent observability · agent safety · AgentOps
Protect agent telemetry by treating it as a governed data-export path, not harmless exhaust: choose the prompt, response, tool-input, and tool-output fields needed for diagnosis; keep content capture off unless justified; isolate restricted traces; and test the actual export path before release. The decision is not whether to observe an agent. It is how to retain enough evidence to explain a run without turning every observability destination into a transcript store.
- 2
- telemetry paths to govern: broad health and restricted diagnostics
- 4
- exception controls: purpose, scope, lifetime, and review
- 1
- export-path test before a configuration reaches real traffic
Start with the diagnostic question, not the fields available
An instrumentation library can expose more data than an operator needs for a release or incident decision. Name the question a signal must answer, then keep the least content-bearing evidence that answers it. If losing a payload field would not prevent a decision, do not collect it by default.
| Diagnostic question | Prefer this evidence | Do not make this the default |
|---|---|---|
| Did the run meet its service objective? | Outcome, elapsed time, deadline status, retry count | Full prompt and response bodies |
| Which tool operation failed? | Tool name, operation class, status, latency, error class | Unbounded tool arguments and results |
| Did a release change behavior? | Release, policy, model, workflow, and task-class identifiers | A copied conversation in release metadata |
| Can a responder reconstruct a disputed action? | A scoped restricted trace or approved audit artifact | Universal access to raw payloads |
This is AgentOps Hardening: make the observability boundary part of the harness instead of accepting whatever an integration happens to emit.
Classify the data before it crosses an export boundary
Do not stop at an SDK toggle. Content can move through the agent, a chat client, an instrumented tool, an exporter, a collector, a backend, an alert, and an incident attachment. Microsoft's documentation is a concrete reason to trace the whole path: when sensitive data is enabled, chat context captured by both a client and an agent can appear in both spans.
Instrumented run
Agent, client, and tool events
An event may carry ordinary run signals or content from a prompt, response, tool argument, or result.
Field policy
Classify before export
Keep the least content-bearing evidence that answers the diagnostic question. Treat raw content as an explicit exception.
Broad health view
Operational signals
Outcome, latency, retry pressure, tool status, release, and policy identity support ordinary operating decisions.
Restricted diagnostic path
Approved content evidence
A bounded diagnostic need, named audience, expiry, and review control access to approved raw content.
Release test
Send representative safe canary values through the configured path. An unexpected copy in a backend, alert, ticket, or attachment is a failed boundary—not harmless telemetry exhaust.
| Field class | Default handling |
|---|---|
| Operational identifiers | Make broadly queryable when workload access permits it. |
| Outcome signals | Use for broad health views with controlled retention. |
| Content-derived summaries | Allow only with an explicit diagnostic purpose. |
| Raw content | Keep restricted and exception-based, not routine dashboard data. |
Separate broad health signals from restricted diagnostic evidence
Ordinary operations usually need outcome, latency, retry pressure, tool status, resource use, release identity, and policy identity—not a raw transcript. When a review genuinely needs content, give it a smaller audience, shorter retention under the team's policy, and an incident or review context.
OpenTelemetry publishes GenAI semantic conventions, including agent-span conventions. Use that documented vocabulary for the operational fields compared across runs, then make content-bearing attributes a separate governance decision. A semantic convention does not itself decide authorization, retention, or data classification.
Make richer capture a time-bounded exception
Before enabling content-rich diagnostics, record four controls:
- Purpose: the failure mode or release question ordinary signals cannot answer.
- Scope: the workflow, task class, fields, environment, and people who may access it.
- Lifetime: when the capture expires or becomes inaccessible under the retention policy.
- Review: who confirms it answered the question and restores the low-content policy.
Test the export path with representative sensitive data
Configuration review cannot demonstrate where data lands. In a non-production test, place distinct safe canary values in a prompt and a tool argument, then run the same instrumentation and export path you intend to release. Inspect the agent and client spans, exporter and collector logs or failure queues, backend search and dashboards, alerts, tickets, incident attachments, and any restricted diagnostic store.
Record where each value appeared, which access boundary applied, and whether the result matched the field register. If a value appears in an unexpected destination, narrow capture or repair the route before real traffic. This is an engineering test procedure, not evidence of regulatory compliance.
Use a compact release gate
| Gate | Evidence to record |
|---|---|
| Field purpose | Every captured field has a diagnostic question and owner. |
| Classification | Raw content, content-derived fields, and operational signals are distinguished. |
| Access path | Broad health and restricted diagnostics have defined audiences. |
| Export-path test | Representative canary values were checked in every intended destination. |
| Exception control | Any rich capture has scope, expiry, and reviewer. |
Tie that record to the release and policy version, then repeat it when a new tool, exporter, or agent surface can copy context. Start with anagent observability dashboard for normal run health, define acceptable outcomes and retries with agent reliability SLOs, and carry the same explicit boundary to custom MCP servers and AgentOps incident response.
Sources
FAQ
Should we turn off agent observability to avoid sensitive-data exposure?
No. Keep the operational signals needed to understand outcome, latency, retries, tool status, and release behavior. Govern content-bearing capture separately instead of making every trace a transcript store.
Why can agent telemetry duplicate prompts and responses?
More than one instrumented component can capture the same chat context. Microsoft specifically warns that, when sensitive data is enabled, chat-client and agent spans can both contain prompts and responses they capture.
Are OpenTelemetry semantic conventions a data-protection policy?
No. They provide a common telemetry vocabulary, including GenAI and agent-span conventions. They do not determine which fields a team may capture, where those fields go, who may access them, or how long they are retained.
When is raw prompt or tool data justified in a trace?
Only when a defined diagnostic or review question cannot be answered from ordinary operational signals. Scope the fields and workflow, give the capture an expiry and reviewer, and test its real export path before it handles real data.
Building something that has to hold up?
We do this work for teams — agent reliability hardening, custom MCP servers, and full-stack AI systems built to survive production.