How to forecast an AI agent run's remaining token budget

Revise an AI agent run budget while the agent works, rather than treating its first estimate as a promise. Record provider-reported input and output tokens after each model call. Forecast both the remaining calls and the input context they will carry, update the range after tools return, and test a proposed pause rule in shadow mode before it can interrupt real work. This operating method is informed by TokenCast's forecasting formulation; its benchmark savings are not a production guarantee.

A token allowance is not a forecast

One failed verification can create several more calls. Each may process much of the prior context again, even if its new answer is short. TokenCast separates a segment's call count, input-length change, and residual consumption; composition carries context growth into later calls. OpenAI's conversation-state guidance says previous input tokens in a response chain using previous_response_id are billed as input tokens. Exact accounting and price still depend on provider and caching behavior.

TokenCast frames an updated total as confirmed usage plus predicted remaining consumption. A cap enforces a boundary; it cannot say whether this particular run is likely to finish within it.

Build the first forecast, then update it

Keep a ledger for each run: task family, model and harness version, call sequence, provider-recorded input and output usage, tool outcomes, current input length, and terminal outcome. Reconcile preflight counts with actual usage. Anthropic's token-counting guidance calls its count an estimate and points to the Messages API usage object for actual accounting. Character counts are not a substitute for billed tokens.

Estimate a distribution of remaining consumption from comparable completed traces. Include likely remaining call count and likely input length at those calls. A simple history-based range is an implementable baseline, but it is not TokenCast's learned segment model or its measured result. Update after a failed tool call, successful verification, retry, or new large document. Save the forecast made at the time alongside eventual actual tokens so calibration can be measured without hindsight.

Let the forecast inform a decision, not conceal it

These thresholds are design choices, not validated TokenCast thresholds. A controller that saves tokens by stopping recoverable runs may worsen the workflow. Bonfire's agent cost runbook covers spend already outrunning output; the observability dashboard helps expose the run-level evidence needed here.

Read the paper's numbers within their limits

Across four task suites and six agent models, TokenCast reported an average 14.5% mean-absolute-error reduction against its strongest comparator over 96 benchmark/model/prediction-point combinations. At Task Start, its average was 2.2% worse; gains grew after execution evidence arrived. In one SWE-bench Verified GPT-5.4 setup, every-call updates took a reported 32.8 ms mean cumulative prediction time per run. Neither result is a portable accuracy or overhead guarantee.

The reported 21.3% token saving used offline budget-control replay on 288 recorded GPT-5.4 runs from 144 SWE-bench Verified tasks, averaged over seven budgets at matched fixed-budget trace completion. Trace completion meant reaching the recorded terminal state, not resolving the task. It does not establish production savings or show that interrupted runs could not have recovered. The authors leave active runtime budget-control integration to future work.

Trial the policy on your own accepted outcomes

Choose one task family. Compare the new forecast with a history-median baseline on held-out runs: remaining-token error at each checkpoint, interval coverage, provider-recorded consumption, and accepted completion. Shadow a proposed pause or stop rule without interrupting runs; inspect every successful run it would have stopped. Widen enforcement only when saved consumption outweighs false stops under your acceptance criteria. If another model lane becomes appropriate, treat multi-model routing as a separate decision. A token forecast is not a dollar forecast: account for prices, caching, tools, and non-model costs separately.