1. For stochastic tools, "closer to the reference" is not a valid identity criterion, and the positive-margin assumption silently excludes the natural attack rather than merely the hard cases. Take a tool that returns 1 with probability 0.9 and 0 with probability 0.1 on a class of inputs, with the reference recorded from that same tool. For scalar outputs the distance reduces to inequality, so the expected gold-versus-reference distance is μgold=2(0.9)(0.1)=0.18, while a replacement that always returns 1 has μrepl=Pr[ref=0]=0.1<0.18. The replacement has changed the tool's output distribution, yet the nearest-distance rule prefers it, and prefers it more reliably as m grows; it also clears an absolute threshold calibrated as τhi=μgold+ϵ. This is not a counterexample to Theorem 1 — it violates γ>0 — and that is precisely the problem: the assumption that makes the theorem work rules out distribution-contracting replacements, which is how one would actually attack a stochastic service. The paper supports stochastic tools and notes positive self-distance, but its noise experiments only add perturbation to otherwise-correct responses; there is no substitute that returns the modal answer, reduces output entropy or shifts the conditional distribution. Three things need separating and the paper merges them: a single response being close, the conditional output distribution agreeing, and the implementation or functional semantics agreeing.
The calibration transfer has a matching gap. The theorem needs the threshold related to the true μgold, while deployment uses s^ from a few admission runs and sets τ=s^+ϵ. Three distinct uncertainties are in play — probe sampling, the tool's own response randomness, and estimation error in the reference self-distance — and the headline confidence radius addresses the first. Appendix S's finite-population correction makes this concrete: reading the entire bank drives the sampling radius to zero, which is sound for a fixed per-probe score table as Proposition 3 states, but for an online stochastic tool it only removes uncertainty about which inputs were drawn, not about what those inputs will return next time. If s^ and a later d^ are two identically distributed estimates from the same undrifted tool, then rejecting on d^>s^ with zero radius has rejection probability near 1/2, not 0.05. The authors concede that under a complete bank a future gold distance slightly above the admission estimate is rejected, and the held-out interval is short. For a paper whose contribution is a calibrated protocol, this is the core of the deployment story.
A third strand points the same way: the operative security property is bank coverage, which the paper does not characterize beyond type-aware enumeration. Figure 3(a)'s failures are diagnostic — the determinant probe invents a tensor name that never existed, so gold and substitute both return "name does not exist" and sit at distance zero over the whole bank, while the agent calls the tool on a tensor that exists; a batch-conversion probe never covers the modified category. In the real-server survey, 10 of the 35 tools classed deterministic return errors on every probe. Schema-valid is not semantically valid, and a stable error is not a fingerprint.
2. What the experiments establish does not require the proposed machinery, and the adversarial evaluation excludes the threat that matters. On clean cached cases the structural distance scores 0.952 at k≥2 — the same as exact equality on complete responses — and an ablation that reads only top-level type and serialization length "already scores 0.927". Turning the canonicalizer off "raises every comparison of complete responses to 0.974", because several substitutes mutate only the fields the canonicalizer deletes: it drops twelve volatile keys including id, cursor, etag and date at any depth and rounds floats to six decimals. Those are noise in some tools and business semantics in others — {"id": "account_A", "value": 100} and {"id": "account_B", "value": 100} canonicalize to the same thing, which needs no adversary who defeats the distance, only one who works upstream of it. The structural distance has a second scope problem that Appendix N shows: a small critical change inside a large response is diluted, so margin falls with output size. The distance therefore measures the fraction of structure that differs, not the security impact of the difference. Taken together with cost — exact equality uses roughly one fifteenth of the probe calls of the full-bank structural configuration in the end-to-end study (8,412 against 126,428), with 8 of 16 tainted tasks against 9 of 16 — the experiments do not establish that the proposed comparator is the part doing the work, and the paper has already moved its claim from "best distance" to "protocol". That makes the unanswered question sharper: against history-response regression testing with exact comparison and a standard threshold, what is ProbeID's irreplaceable part?
The security evaluation has a structural limitation the authors state: in the adaptive studies the gold remains among the candidates at distance zero, so with a non-negative distance no substitute can be strictly nearer, "certification is excluded by construction", and attacks can only convert identification into abstention. Zero misidentifications over 14,080 runs therefore measures the abstention implementation, not robustness to a general adversary. The deployment-relevant case is gold absent — the original endpoint replaced, with no zero-distance gold to shield the decision and a behavior-matching substitute facing only the single-candidate threshold. Relatedly the adversary must fix its mimicry region before seeing the round's sample; an attacker who observes probes and then adapts is outside the model, while the agent integration draws whole banks at every check, which accelerates probe exposure. And the availability cost is real: the white-box attack pushes abstention to 0.90 at k=8. Finally, the per-check δ is not an end-to-end task-level failure probability; repeated checks need an error budget, and a union bound grows with the number of checks.
Author Response to Reviewer Jsqm
Thank you for the review. Admission checks cannot see a later behavior-only change, and version pins miss changes under unchanged definitions. Over eleven weeks, 7/59 pinned tools changed responses (Appendix O). In the agent study, 15/16 tasks receive tampered values ungated and 9/16 gated in each of three repeats, despite 1.000 task success (Table P.1). This shows incomplete runtime protection, without establishing how often production verification should run.
1. Comparison with autonomous CLI agents. That comparison is missing. The 7B and 32B behavioral LLM judges evaluate supplied responses rather than autonomously designing and executing tests (Appendix H). ProbeID supplies a trusted-reference protocol, abstention and explicit error-bound conditions, together with evaluation of the values actually delivered to an agent. We claim neither general test-generation capability nor superiority over autonomous coding agents. Such agents could complement the protocol by constructing valid state preconditions.
2. Verification frequency. An admission check cannot detect a later endpoint or backing-data change. Repeated runtime checks address replacement in the agent study, but do not show that every call needs a full-bank check. Reusing a verdict requires an explicit freshness window and state policy. We have not evaluated that cost-risk trade-off (Section 6).
3. Execution cost. Probe calls execute tools and must be counted. The 3.527 ms CPU decision time excludes them (Appendix H.1). Table P.2 gives the following single-run rug-pull configurations on the same 16 tasks.
Recording and calibrating the banks adds 152,348 calls. Equality used a budget chosen after earlier outcomes and followed different trajectories, so 8/16 versus 9/16 does not establish superior protection. Full-bank checks took 0.66 s on local held-out tools (B=512) versus 4.2 min on a rate-limited weather server (B=96, Appendix R). The sequential gate's eight-probe average concerns full value drift in bank studies, not an end-to-end agent run. Its protection, refused calls and latency have not been evaluated jointly.
4. Legitimate changes and recovery. A refusal calls for revalidation and does not establish malice. In the held-out study, the original structural gate refuses all 16 tools whose answers change under reworded errors and added fields on 5/7 affected local tools (Table R.2). The finite-population gate refuses added fields on all 7 at the full bank (Appendix S). No refusal occurs across the benign agent arm's 269 structural verdicts or 278 equality verdicts, but static runs do not estimate false alarms under routine production updates. Finite-bank resampling likewise does not certify future stochastic responses.
As stated in the revised Ethics statement, safe recovery requires authorized inspection and a newly trusted reference for an accepted update. Automatically adopting a rejected response would defeat the clean-admission assumption. Recovery latency and its effect on downstream availability have not been evaluated.
5. Ethics. You flagged this submission for ethics review. Probing can expose sensitive responses or change state. The census used whitelisted read-only tools on an isolated host (Appendix O), while write and authentication experiments used controlled, reset sandboxes (Appendix M). Real-world response changes are not classified as malicious. The protocol does not authorize third-party production probing or bypassing access controls. Its risks include misplaced admission trust, service load, false refusals and missed drift. Supported protection concerns only behavioral changes reached by the probes.