- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether reading interaction history separately for each candidate object improves early target prediction in virtual reality. The proposed approach, CALM-VR, uses candidate-specific queries to retrieve relevant evidence from gaze, head, and controller histories and predict the user’s next target. Experiments on target acquisition, cluttered-shelf search, and sequential assembly show an overall predictive benefit, with gains varying across tasks.
This paper should be rejected because (1) the core design’s benefits remain insufficiently characterized, (2) key experimental details are missing, (3) the claimed utility–risk trade-off is inadequately supported, and (4) the presentation is inconsistent and difficult to follow.
Critical strengths
The paper offers a useful modeling insight for early VR target prediction: candidate information can improve target scoring from the retrieval of relevant sensor history. CALM-VR implements this idea by allowing each candidate to attend separately to multimodal histories. Controlled comparisons support its incremental benefit in unseen participants, with further evidence from held-out layouts and task flows.
Critical weaknesses
The incremental benefit of candidate-specific queries is not sufficiently characterized. Table 2 reports confidence intervals and corrected tests for the overall query benefit, but mixes this comparison with external baselines, temperature scaling, and other differences. This makes it difficult to distinguish the full–broadcast query effect from other configuration effects or assess its uncertainty within each task. The robustness and runtime analyses (Tables S17 and S20) omit the broadcast-query control, leaving unclear whether the advantage persists under sensor degradation and how much additional computation it requires. Figures 1–2 also do not clearly connect candidate information, history reading, and the broadcast-query control to the method description. Together, these gaps limit the paper's central contribution: a controlled empirical account of this design choice.
The selective policy does not yet establish practical utility. Table 4 reports 96.9% accuracy on triggered events, but the policy triggers on only 6.5% of eligible opportunities (74/1,131; Table S10). This only shows high conditional accuracy at low coverage while not establish better overall assistance. The reduction in estimated waiting-time was relatively small to no assistance condition, and there lacks clear definitions of preparation time, prefetch timing, and the cost of incorrect prefetches, leaving its calculation difficult to assess. Wrong/min metric also lacks a clear link between elapsed time and eligible opportunities.
The figures and overall organization make the argument difficult to follow. Figure 1 does not clearly show candidate-specific history reading, while Figure 2 leaves the correspondence between its notation and §3.1 unclear. Table 2, Table 3, and Figures 3a–b repeat the overall gains, and the configuration differences and task-specific limitations needed to interpret them are scattered across the main text and appendices.
Questions for authors
Why use APS to trigger prefetching? Appendix F.3 reports lower selective risk for confidence ranking at matched action rates.
What benefit does temperature scaling provide? Unchanged Top-1 accuracy in Table 2 does not establish whether calibration improves. Appendix F.3 also reports higher selective risk for scaled APS than raw APS.
How were the recovery results in Figures 3c–d evaluated?