- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PERCEPT-VR++, a framework for predicting perceptual risk before visual updates in virtual reality. It combines multimodal temporal modeling (M2P-CF), probability calibration, and a risk-aware scheduler (Safe-UARS) to determine whether visual updates should be executed, delayed, attenuated, or split. The authors introduce VR-Mask++, containing 8,301 valid events from 450 participants, and conduct an independent 55-participant closed-loop evaluation. The results demonstrate improved perceptual-risk prediction and reduced false-safe rates through update-conditioned risk re-estimation.
Critical strengths
The paper addresses an important problem in perceptual rendering by incorporating pre-update behavioral signals and pending-update attributes into risk prediction. The newly collected VR-Mask++ dataset and independent closed-loop user study provide substantial empirical support.
Critical weaknesses
1.The evaluated predictors do not consistently use equivalent input representations or model capacities. For example, the temporal Transformer baseline lacks an explicit update-attribute vector, whereas M2P-CF incorporates these attributes through a dedicated branch. This makes it difficult to attribute performance improvements specifically to the proposed modeling approach rather than differences in input information or architecture.
2.Although re-querying improves closed-loop outcomes, the paper does not directly establish whether predicted probabilities remain well calibrated for attenuated or split updates. Since modifying update attributes changes the prediction context, additional action-conditioned calibration and reliability analyses are needed to validate the risk estimates used for scheduling.
3.The experiments use a single VR hardware platform, while the independent closed-loop study covers only three representative scenes. Furthermore, the reported inference latency is measured on a high-end workstation and excludes rendering and motion-to-photon delays. Cross-device evaluations and end-to-end headset measurements would strengthen the practical relevance of the framework.
Questions for authors
1.What are the fundamental methodological innovations of PERCEPT-VR++ beyond integrating existing temporal modeling, calibration, and rule-based scheduling techniques?
2.Given the marginal improvement over M2P-core, can the authors provide stronger statistical evidence demonstrating the necessity of the full architecture and auxiliary objectives?
3.What are the actual computational savings and visual-quality trade-offs achieved by Safe-UARS, considering update attenuation, splitting, and additional scheduling delays?
6 comments
For reliability, mean predicted versus observed detection rates are 7.657% versus 6.646% for Attenuate and 8.205% versus 5.952% for Split. The corresponding prediction-minus-observation gaps are +1.011 percentage points [−1.468, +3.428] and +2.253 [−2.684, +6.379]. The common bins [0, 0.10) and [0.10, 1] contain 222/94 Attenuate events with 5/16 detections and 58/26 Split events with 1/4 detections. In Attenuate's lower bin, the mean prediction is 4.671% versus 2.252% observed detection, a gap of +2.419 percentage points [0.418, 4.015]. Table S54 gives finer-bin sensitivity and participant counts. Action-specific estimates use 316 Attenuate events from 60 participants and 84 Split events from 46 participants; all 60 participants remain in the common bootstrap sampling frame. Tables S52–S53 and Figure S4 give action-level scores and bin-wise estimates.
The revision thus provides direct action-conditioned probability evidence: re-query improves prediction for both types of displayed modification, and their reliability is quantified with explicit support and uncertainty.
Revision locations. Section 5.5 (pp. 9–10); Appendix A.5.1, Table S38 (pp. 33–35); Appendix A.5.3, Table S51 (p. 43); Appendix A.5.4, Tables S52, S53, S54 (pp. 44–46).
R3.3 — Scene coverage and measured system performance
Concern. The single VR platform, three-scene independent study, and workstation-only inference timing limit practical relevance; cross-device and end-to-end headset measurements would strengthen practical relevance.
Response. The added evaluation combines 60 new participants, six scenes, and system measurements on the Quest Pro–RTX 4090 platform. The six scenes include two from each of three task families: controlled detection, natural search, and reading/UI. Every participant completes C0, C1, and R1, yielding 3,600 candidate requests per condition. This complements the original 55-participant, three-scene study with a separate cohort and broader interactive tasks.
The system evaluation measures rendering, CPU execution, display-deadline completion, and request delivery over 36,000 participant-seconds per condition. For C1/R1, rendering GPU mean/P95/P99 is 7.92/9.60/10.60 ms and 7.88/9.50/10.40 ms per frame; CPU main-thread time averages 3.9 ms in both conditions. Missed display-deadline slots are 0.45%/0.42%, measured against 3,240,000 runtime-recorded target slots per condition. PC GPU activity includes rendering, ensemble inference, related transfers, and runtime/compositor work.
Request latency is measured from request generation to the first updated frame's actual software submission, incorporating computation, scheduling, and submission waiting. C1 and R1 both have a measured median/P95 of 124.44/204.44 ms. Under this matched timing, re-query reduces FSR from 5.123% to 3.168% while increasing full completions from 660 to 680 and undetected full completions from 631 to 655.
These measurements connect the perceptual benefit to interactive operation on the evaluated system: broader task and scene coverage, fewer detected applications, more complete deliveries, and measured rendering and scheduling costs. The revision reports model-query, frame-level, and request-level timing with their respective endpoints and denominators, making the deployment evidence directly interpretable.
Revision locations. Section 5.5, Tables 6–7 (pp. 9–10); Appendix A.7, Table S59 (p. 48); Appendix A.5.1, Tables S46, S47 (pp. 33–42).
R3.4 — Methodological contribution beyond existing components
Concern. What is the fundamental methodological innovation beyond combining temporal modeling, calibration, and rule-based scheduling?
Response. The methodological contribution is to formulate perceptual risk as a property of the pre-update state–proposed-update pair, and to use that formulation when the scheduler changes the update. For a fixed behavioral history H, attenuation or splitting changes the query from to . Re-query evaluates the modified proposal with the same frozen model and calibrator, so admission uses a prediction for the update actually proposed for display.
PERCEPT-VR++ conditions on behavioral history and update attributes to determine when and in what form an update is applied. Update-conditioned temporal aggregation and the wide branch implement this dependency; matched-event regularization connects comparable events, and calibration connects prediction to frozen admission rules. This complements spatial fidelity allocation and discrete blink/saccade windows with update-specific risk.
The revised evidence tests this connection at the information, decision, and prediction levels. Removing update attributes changes AUROC from 0.831 to 0.793, establishing their predictive contribution. With a shared frozen predictor and common computation/submission barriers, re-query reduces FSR from 5.123% to 3.168%. A further fixed-opportunity control disables autonomous Delay and retains the reduction, from 5.256% to 3.415% (difference −1.842 percentage points, 95% CI [−3.082, −0.616]; Holm-adjusted p = 0.02175). On C1's 400 actually displayed modifications, shadow re-query reduces Brier error from 0.062619 to 0.054445, a 13.05% reduction, using the same history and observed outcome for each paired prediction.
These results establish the value of update-conditioned re-query: more accurate prediction of displayed modifications and improved scheduling decisions.
Revision locations. Sections 1–3 (pp. 1–5); Section 5.2/Table 3 (p. 8); Section 5.5 (p. 9); Appendix A.1 (p. 15); Appendix A.5.2 (p. 42); Appendix A.5.3 (p. 43).
R3.5 — Full versus Core and auxiliary-objective necessity
Concern. Can stronger statistical evidence establish the necessity of the full architecture and auxiliary objectives despite the small gain over M2P-core?
Response. Full improves AUROC and fixed-coverage low-risk selection, with consistent gains across five paired runs. AUROC is 0.831 versus Core's 0.826, a paired difference of +0.005 (95% participant-cluster CI [+0.0007, +0.0096]). Full reduces mean FS@20 from 0.07377 to 0.05574, a reduction of 1.803 percentage points [0.500, 3.110]. Core/Full AUPRC is 0.739/0.731. AUPRC summarizes the precision–recall curve, whereas FS@20 evaluates observed detection among the lowest-risk 20% at fixed coverage. These results support the full design for fixed-coverage low-risk selection.
The objective comparisons retain the Full architecture. Each run selects 244 of the same 1,219 test events. Positive excess FS@20 favors Full:
| Comparator | Mean FS@20 | Excess over Full, pp [95% cluster CI] |
|---|---|---|
| Core | 0.07377 | 1.803 [0.500, 3.110] |
| Without ranking | 0.08197 | 2.623 [1.100, 4.250] |
| Without grouped contrast | 0.07295 | 1.721 [0.350, 3.100] |
| Without invariance | 0.07705 | 2.131 [0.600, 3.670] |
| BCE-only | 0.09098 | 3.525 [1.800, 5.300] |
Full records 13, 14, 13, 14, and 14 detections, giving FS@20 0.055738 ± 0.002245 (mean ± sample SD). It has lower FS@20 than every comparator in all five paired runs (Table S20).
The intervals use 20,000 paired participant-cluster bootstrap resamples shared across variants and runs. Each run reselects its lowest-risk 20% using its frozen calibrator; paired differences are then averaged across runs. AUROC uses the mean of five raw member probabilities. The intervals describe participant-sampling uncertainty; run-wise SDs describe training variability (Appendix A.3.5).
Core uses final GRU states and abruptness supervision; Full adds update-conditioned attention and the complete objective. Single-objective ablations retain abruptness; BCE-only removes all auxiliary losses. These comparisons establish reproducible gains from the complete configuration and the reported objectives in low-risk selection.
Revision locations. Sections 5.1–5.3, Tables 1 and 3–4 (pp. 7–8); Section 4, FS@20 definition (pp. 6–7); Appendix A.3.1, Table S13 (p. 22); Appendix A.3.5, Tables S19, S20 (pp. 25–26).
R3.6 — Actual computation, delivered updates, quality, and delays
Concern. What computational savings and quality trade-offs result after attenuation, splitting, and scheduling delay?
Response. The added 60-participant experiment measures actual delivery, image quality, computation, and delay over 3,600 requests and 36,000 participant-seconds per condition. C0 uses the original current-score path; C1 performs the modification query as a shadow computation with a common submission barrier; R1 uses the recomputed outputs with that same barrier.
| Outcome | C0 | C1 | R1 |
|---|---|---|---|
| Fully completed original requests | 650 | 660 | 680 |
| Undetected full completions | 617 | 631 | 655 |
| LOD / resolution PSNR (dB) | 35.6 / 34.9 | 35.6 / 35.0 | 35.5 / 34.9 |
| Mean rendering GPU time (ms/frame) | 7.90 | 7.92 | 7.88 |
| PC GPU activity (s) | 25,700 | 25,760 | 25,640 |
| Request-to-submission median / P95 (ms) | 96 / 230 | 124.44 / 204.44 | 124.44 / 204.44 |
Attenuation is credited with its actual terminal change. A Split counts as fully complete when its original target is reached within the deadline and type-specific tolerance. Relative to C1, R1 delivers 20 more complete requests and 24 more undetected full completions. Removed triangles increase from 8.25 to 8.50 million, and removed resolution pixels from 61.1472 to 63.8496 million; Table S43 reports all five update types separately.
Response to Reviewer i8ik
Response to Reviewer i8ik
PERCEPT-VR++, Submission 46014
Thank you for recognizing pre-update behavioral signals and pending-update attributes. The empirical foundation combines a 450-participant, nine-scene dataset, an independent 55-participant policy and experience study, and a separate 60-participant, six-scene follow-up. The revision adds matched-input comparisons, controlled mechanism tests, action-specific reliability, and actual delivery, quality, computation, and latency measurements. We address the three weaknesses and three questions below; concern summaries are paraphrases.
R3.1 — Equivalent inputs and model capacities
Concern. Unequal input representations and capacities, particularly the temporal Transformer's missing update-attribute vector, prevent attributing the prediction gains to the proposed modeling approach.
Response. The added comparison directly evaluates prediction with shared input information, comparable capacities, and matched model-selection budgets. Transformer+U and ModernTCN+U receive the same 500-ms eye, head, and context sequences, valid-step masks, preprocessing, missing-data handling, and update/static scene attributes as M2P. Both baselines explicitly encode the pending update in a static branch fused with the temporal representation; M2P incorporates it through its event query and wide branch.
All four models use the same participant-disjoint training/validation/test split of 5,828/1,254/1,219 events. Each searches 16 configurations, selects by validation AUROC, and trains five final members with the same seeds. They share AdamW, learning rate 3×10⁻⁴, batch size 128, a 120-epoch maximum, and early-stopping patience 20. Single-member parameter counts are 0.72M for Transformer+U, 0.78M for ModernTCN+U, and 0.82M for both M2P variants. Tables S17–S18 provide the selected architectures and results.
The comparison separates shared supervision from the complete training objective. With class-weighted BCE, Transformer+U, ModernTCN+U, and M2P BCE-only achieve AUROC 0.811, 0.802, and 0.816, respectively. Holding the M2P architecture fixed, Full adds grouped contrastive, invariance, ranking, and abruptness objectives and reaches 0.831. Each AUROC is computed on the same held-out events from the mean probabilities of five members.
M2P retains a higher observed AUROC with shared information and comparable selection resources. Shared-BCE comparisons evaluate encoding and fusion; Full versus BCE-only evaluates the combined training objectives.
Revision locations. Abstract (p. 1); Section 5.1 and Table 2 (pp. 7–8); Appendix A.3.4, Tables S17, S18 (pp. 24–25).
R3.2 — Calibration after attenuation and splitting
Concern. Improved closed-loop outcomes do not establish that attenuated or split updates have reliable, action-conditioned probability estimates.
Response. We now evaluate modified-plan probabilities directly for Attenuate and Split separately, using actual displayed events, reliability bins, and participant-cluster confidence intervals. The prediction target is the complete proposed plan, represented by action class, step masks, planned timing, and cumulative states. Split has one label for detection of any stage or transition. The follow-up model uses 960 training plans from 32 participants and 240 calibration plans from eight disjoint participants, balanced across Execute, Attenuate, and Split. Model, calibrator, and thresholds are frozen before evaluation on the separate 60-participant cohort.
The analysis includes all 400 C1 modifications: 316 Attenuate with 21 detections, and 84 Split with five detections. C1 selects these events with the original outputs; modified-plan probabilities are shadow predictions. Original and modified-plan probabilities therefore share the displayed event, history cutoff, and observed label. Overall Brier decreases from 0.062619 to 0.054445, and NLL from 0.251220 to 0.200493.
The improvement holds for both actions. Attenuate Brier decreases from 0.063289 to 0.055052 (paired difference −0.008237; 95% CI [−0.013031, −0.003976]), and Split Brier from 0.060098 to 0.052163 (−0.007936; [−0.018927, −0.000347]). NLL decreases from 0.250861 to 0.201219 for Attenuate and from 0.252569 to 0.197763 for Split. These new action-level intervals use 20,000 resamples of all 60 participants, retaining each participant's events and recomputing pooled-event metrics.