Where the Risk Goes: Error Allocation in LLM Agents under Conformal Risk Control

Conformal risk control can calibrate a commit-or-abstain rule so that an LLM agent's marginal probability of committing to a wrong answer stays below a chosen level. This guarantee does not control where those errors occur: the same overall risk can be distributed very differently across trajectories of different lengths. That distribution depends on the trajectory score the rule uses, while common summaries can obscure where uncertainty occurs or how uncertainty across steps combines. We introduce trajectory spatial contrast (TSC), a family of scores that treats the sequence of per-step uncertainty values as an ordered multivariate object. Each path is normalized to a fixed grid and scored by contrasting its spatial ranks against reference trajectories from correct and incorrect episodes. On synthetic examples designed to isolate trajectory structure, TSC separates failure patterns that no symmetric summary distinguishes and no additive score separates perfectly. Because every score is fitted on a separate reference split, conformal risk control gives TSC and competing scores the same marginal wrong-commit guarantee. Evaluating twelve scores across three benchmarks and three model families, we show that accumulating scores can concentrate almost all permitted wrong commits on the shortest trajectories, making the rule sensitive to shifts in the trajectory-length distribution. By contrast, the TSC family places errors most evenly across length groups in most benchmark–model pairs, at a modest cost in commit rate: it commits most in a minority of pairs and trails the best alternative by a few percentage points on average elsewhere.