- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies decision-relevant evidence reuse in adaptive brain–AI interfaces under changes in user intent and recording conditions. The central observation is that evidence collected before a change should not necessarily be discarded in its entirety. The authors formalize this problem using a finite latent-state model with separate goal and recording variables. Based on this framework, they introduce Decision-Aware Reuse and Probing (DARP), which combines belief transport, cost-aware diagnostic queries, and posterior-based action selection. The updated DARP results indicate reduced diagnostic time and improved throughput, although the updated implementation is not accompanied by the run-level records necessary to reproduce or statistically assess these improvements.
Critical strengths
-
The distinction between goal changes and recording changes is well motivated. Existing adaptive interfaces can conflate changes in the user's desired action with changes in the observation mechanism, potentially retaining evidence that no longer supports the correct decision. The proposed formulation makes this distinction explicit and connects evidence preservation to the current decision.
-
The paper develops a coherent analysis of action identifiability and additional diagnostic cost.
Critical weaknesses
-
Reading/understanding the results is quite confusing. The result numbers come from an “updated DARP” row “supplied by the study team” (Table 1), with no description of what changed. The reproducible DARP-KL run trails Goal-IG on coverage, error, outdated actions and deadline completion, in all four stress settings, and along the threshold sweep.
-
The regime where reuse should help is never tested. Theorem 3 predicts gains only when recalibration is costly and confirmation is not cheap. The simulator tests only cheap or absent confirmation, with no sweep over the two costs against Goal-IG. Even without confirmation, DARP-KL trails Goal-IG on both coverage and error.
-
The theory seems incremental and doesn’t cover the tested algorithm. The paper specializes Chernoff-style active hypothesis testing to action-changing alternatives. For fixed retained evidence, reuse gives no asymptotic gain in the paper.
Questions for authors
-
Can the authors provide the updated DARP implementation, event-level records, and paired evaluation against Goal-IG under identical agents, random seeds, query budgets, and stopping criteria? In particular, what changed between the archived DARP-KL and updated DARP implementations
-
Can the authors provide risk–coverage–diagnostic cost curves for DARP, Goal-IG, and global resetting, preferably with paired confidence intervals? The current comparisons use different attained operating points, making it difficult to determine whether DARP actually provides a better efficiency–reliability tradeoff.