acceptodds
Under review as a conference paper at ICLR 2027

PERCEPT-VR++: Learning Pre-Update Perceptual Risk for Risk-Aware Visual Update Scheduling in VR

Abstract

A perceptual-risk score for a pending visual update may no longer describe the update after a scheduler changes its attributes. We study pre-update risk prediction conditioned jointly on multimodal behavioral history and the proposed update. PERCEPT-VR++ instantiates this formulation with M2P-CF, using eye dynamics, head motion, scene context, and update attributes. We construct VR-Mask++ through nine independently administered scene studies with 450 participants, collecting 9,000 candidate trials and retaining 8,301 valid multimodal events. Held-out AUROC increases from 0.766 for the strongest evaluated non-M2P baselines to 0.831 for M2P-CF; beta calibration reduces its ECE from 0.109 to 0.043. An independent closed-loop study recruits 55 new participants and records 7,425 candidate policy-event observations. False-safe rate decreases from 0.105 for calibrated prediction-only to 0.059 for current-score Safe-UARS and 0.030 for re-query Safe-UARS, while applied-update coverage rises from 0.231 to 0.265 and 0.280. Current-score and re-query share all frozen policy parameters, differing only in whether risk is re-estimated after update attributes change. Re-query yields a participant-paired FSR difference of −0.029 (95% CI [−0.053,−0.005]); both adjacent-policy paired intervals exclude zero, with consistent ordering across scene families. These results support re-estimating risk for the update actually proposed for application, rather than retaining the score of its original specification.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human-Computer Interaction, Image & Video Synthesis, Datasets & Benchmarks for Vision, Virtual Reality

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_i8ikOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PERCEPT-VR++, a framework for predicting perceptual risk before visual updates in virtual reality. It combines multimodal temporal modeling (M2P-CF), probability calibration, and a risk-aware scheduler (Safe-UARS) to determine whether visual updates should be executed, delayed, attenuated, or split. The authors introduce VR-Mask++, containing 8,301 valid events from 450 participants, and conduct an independent 55-participant closed-loop evaluation. The results demonstrate improved perceptual-risk prediction and reduced false-safe rates through update-conditioned risk re-estimation.

Critical strengths

The paper addresses an important problem in perceptual rendering by incorporating pre-update behavioral signals and pending-update attributes into risk prediction. The newly collected VR-Mask++ dataset and independent closed-loop user study provide substantial empirical support.

Critical weaknesses

1.The evaluated predictors do not consistently use equivalent input representations or model capacities. For example, the temporal Transformer baseline lacks an explicit update-attribute vector, whereas M2P-CF incorporates these attributes through a dedicated branch. This makes it difficult to attribute performance improvements specifically to the proposed modeling approach rather than differences in input information or architecture.

2.Although re-querying improves closed-loop outcomes, the paper does not directly establish whether predicted probabilities remain well calibrated for attenuated or split updates. Since modifying update attributes changes the prediction context, additional action-conditioned calibration and reliability analyses are needed to validate the risk estimates used for scheduling.

3.The experiments use a single VR hardware platform, while the independent closed-loop study covers only three representative scenes. Furthermore, the reported inference latency is measured on a high-end workstation and excludes rendering and motion-to-photon delays. Cross-device evaluations and end-to-end headset measurements would strengthen the practical relevance of the framework.

Questions for authors

1.What are the fundamental methodological innovations of PERCEPT-VR++ beyond integrating existing temporal modeling, calibration, and rule-based scheduling techniques?

2.Given the marginal improvement over M2P-core, can the authors provide stronger statistical evidence demonstrating the necessity of the full architecture and auxiliary objectives?

3.What are the actual computational savings and visual-quality trade-offs achieved by Safe-UARS, considering update attenuation, splitting, and additional scheduling delays?

6 comments
AuthorsOpenReview ↗

Response to Reviewer i8ik

Response to Reviewer i8ik

PERCEPT-VR++, Submission 46014

Thank you for recognizing pre-update behavioral signals and pending-update attributes. The empirical foundation combines a 450-participant, nine-scene dataset, an independent 55-participant policy and experience study, and a separate 60-participant, six-scene follow-up. The revision adds matched-input comparisons, controlled mechanism tests, action-specific reliability, and actual delivery, quality, computation, and latency measurements. We address the three weaknesses and three questions below; concern summaries are paraphrases.

R3.1 — Equivalent inputs and model capacities

Concern. Unequal input representations and capacities, particularly the temporal Transformer's missing update-attribute vector, prevent attributing the prediction gains to the proposed modeling approach.

Response. The added comparison directly evaluates prediction with shared input information, comparable capacities, and matched model-selection budgets. Transformer+U and ModernTCN+U receive the same 500-ms eye, head, and context sequences, valid-step masks, preprocessing, missing-data handling, and update/static scene attributes as M2P. Both baselines explicitly encode the pending update in a static branch fused with the temporal representation; M2P incorporates it through its event query and wide branch.

All four models use the same participant-disjoint training/validation/test split of 5,828/1,254/1,219 events. Each searches 16 configurations, selects by validation AUROC, and trains five final members with the same seeds. They share AdamW, learning rate 3×10⁻⁴, batch size 128, a 120-epoch maximum, and early-stopping patience 20. Single-member parameter counts are 0.72M for Transformer+U, 0.78M for ModernTCN+U, and 0.82M for both M2P variants. Tables S17–S18 provide the selected architectures and results.

The comparison separates shared supervision from the complete training objective. With class-weighted BCE, Transformer+U, ModernTCN+U, and M2P BCE-only achieve AUROC 0.811, 0.802, and 0.816, respectively. Holding the M2P architecture fixed, Full adds grouped contrastive, invariance, ranking, and abruptness objectives and reaches 0.831. Each AUROC is computed on the same held-out events from the mean probabilities of five members.

M2P retains a higher observed AUROC with shared information and comparable selection resources. Shared-BCE comparisons evaluate encoding and fusion; Full versus BCE-only evaluates the combined training objectives.

Revision locations. Abstract (p. 1); Section 5.1 and Table 2 (pp. 7–8); Appendix A.3.4, Tables S17, S18 (pp. 24–25).

R3.2 — Calibration after attenuation and splitting

Concern. Improved closed-loop outcomes do not establish that attenuated or split updates have reliable, action-conditioned probability estimates.

Response. We now evaluate modified-plan probabilities directly for Attenuate and Split separately, using actual displayed events, reliability bins, and participant-cluster confidence intervals. The prediction target is the complete proposed plan, represented by action class, step masks, planned timing, and cumulative states. Split has one label for detection of any stage or transition. The follow-up model uses 960 training plans from 32 participants and 240 calibration plans from eight disjoint participants, balanced across Execute, Attenuate, and Split. Model, calibrator, and thresholds are frozen before evaluation on the separate 60-participant cohort.

The analysis includes all 400 C1 modifications: 316 Attenuate with 21 detections, and 84 Split with five detections. C1 selects these events with the original outputs; modified-plan probabilities are shadow predictions. Original and modified-plan probabilities therefore share the displayed event, history cutoff, and observed label. Overall Brier decreases from 0.062619 to 0.054445, and NLL from 0.251220 to 0.200493.

The improvement holds for both actions. Attenuate Brier decreases from 0.063289 to 0.055052 (paired difference −0.008237; 95% CI [−0.013031, −0.003976]), and Split Brier from 0.060098 to 0.052163 (−0.007936; [−0.018927, −0.000347]). NLL decreases from 0.250861 to 0.201219 for Attenuate and from 0.252569 to 0.197763 for Split. These new action-level intervals use 20,000 resamples of all 60 participants, retaining each participant's events and recomputing pooled-event metrics.

For reliability, mean predicted versus observed detection rates are 7.657% versus 6.646% for Attenuate and 8.205% versus 5.952% for Split. The corresponding prediction-minus-observation gaps are +1.011 percentage points [−1.468, +3.428] and +2.253 [−2.684, +6.379]. The common bins [0, 0.10) and [0.10, 1] contain 222/94 Attenuate events with 5/16 detections and 58/26 Split events with 1/4 detections. In Attenuate's lower bin, the mean prediction is 4.671% versus 2.252% observed detection, a gap of +2.419 percentage points [0.418, 4.015]. Table S54 gives finer-bin sensitivity and participant counts. Action-specific estimates use 316 Attenuate events from 60 participants and 84 Split events from 46 participants; all 60 participants remain in the common bootstrap sampling frame. Tables S52–S53 and Figure S4 give action-level scores and bin-wise estimates.

The revision thus provides direct action-conditioned probability evidence: re-query improves prediction for both types of displayed modification, and their reliability is quantified with explicit support and uncertainty.

Revision locations. Section 5.5 (pp. 9–10); Appendix A.5.1, Table S38 (pp. 33–35); Appendix A.5.3, Table S51 (p. 43); Appendix A.5.4, Tables S52, S53, S54 (pp. 44–46).

R3.3 — Scene coverage and measured system performance

Concern. The single VR platform, three-scene independent study, and workstation-only inference timing limit practical relevance; cross-device and end-to-end headset measurements would strengthen practical relevance.

Response. The added evaluation combines 60 new participants, six scenes, and system measurements on the Quest Pro–RTX 4090 platform. The six scenes include two from each of three task families: controlled detection, natural search, and reading/UI. Every participant completes C0, C1, and R1, yielding 3,600 candidate requests per condition. This complements the original 55-participant, three-scene study with a separate cohort and broader interactive tasks.

The system evaluation measures rendering, CPU execution, display-deadline completion, and request delivery over 36,000 participant-seconds per condition. For C1/R1, rendering GPU mean/P95/P99 is 7.92/9.60/10.60 ms and 7.88/9.50/10.40 ms per frame; CPU main-thread time averages 3.9 ms in both conditions. Missed display-deadline slots are 0.45%/0.42%, measured against 3,240,000 runtime-recorded target slots per condition. PC GPU activity includes rendering, ensemble inference, related transfers, and runtime/compositor work.

Request latency is measured from request generation to the first updated frame's actual software submission, incorporating computation, scheduling, and submission waiting. C1 and R1 both have a measured median/P95 of 124.44/204.44 ms. Under this matched timing, re-query reduces FSR from 5.123% to 3.168% while increasing full completions from 660 to 680 and undetected full completions from 631 to 655.

These measurements connect the perceptual benefit to interactive operation on the evaluated system: broader task and scene coverage, fewer detected applications, more complete deliveries, and measured rendering and scheduling costs. The revision reports model-query, frame-level, and request-level timing with their respective endpoints and denominators, making the deployment evidence directly interpretable.

Revision locations. Section 5.5, Tables 6–7 (pp. 9–10); Appendix A.7, Table S59 (p. 48); Appendix A.5.1, Tables S46, S47 (pp. 33–42).

R3.4 — Methodological contribution beyond existing components

Concern. What is the fundamental methodological innovation beyond combining temporal modeling, calibration, and rule-based scheduling?

Response. The methodological contribution is to formulate perceptual risk as a property of the pre-update state–proposed-update pair, and to use that formulation when the scheduler changes the update. For a fixed behavioral history H, attenuation or splitting changes the query from r(H,U)r(H,U) to r(H,U′)r(H,U^{\prime}). Re-query evaluates the modified proposal with the same frozen model and calibrator, so admission uses a prediction for the update actually proposed for display.

PERCEPT-VR++ conditions on behavioral history and update attributes to determine when and in what form an update is applied. Update-conditioned temporal aggregation and the wide branch implement this dependency; matched-event regularization connects comparable events, and calibration connects prediction to frozen admission rules. This complements spatial fidelity allocation and discrete blink/saccade windows with update-specific risk.

The revised evidence tests this connection at the information, decision, and prediction levels. Removing update attributes changes AUROC from 0.831 to 0.793, establishing their predictive contribution. With a shared frozen predictor and common computation/submission barriers, re-query reduces FSR from 5.123% to 3.168%. A further fixed-opportunity control disables autonomous Delay and retains the reduction, from 5.256% to 3.415% (difference −1.842 percentage points, 95% CI [−3.082, −0.616]; Holm-adjusted p = 0.02175). On C1's 400 actually displayed modifications, shadow re-query reduces Brier error from 0.062619 to 0.054445, a 13.05% reduction, using the same history and observed outcome for each paired prediction.

These results establish the value of update-conditioned re-query: more accurate prediction of displayed modifications and improved scheduling decisions.

Revision locations. Sections 1–3 (pp. 1–5); Section 5.2/Table 3 (p. 8); Section 5.5 (p. 9); Appendix A.1 (p. 15); Appendix A.5.2 (p. 42); Appendix A.5.3 (p. 43).

R3.5 — Full versus Core and auxiliary-objective necessity

Concern. Can stronger statistical evidence establish the necessity of the full architecture and auxiliary objectives despite the small gain over M2P-core?

Response. Full improves AUROC and fixed-coverage low-risk selection, with consistent gains across five paired runs. AUROC is 0.831 versus Core's 0.826, a paired difference of +0.005 (95% participant-cluster CI [+0.0007, +0.0096]). Full reduces mean FS@20 from 0.07377 to 0.05574, a reduction of 1.803 percentage points [0.500, 3.110]. Core/Full AUPRC is 0.739/0.731. AUPRC summarizes the precision–recall curve, whereas FS@20 evaluates observed detection among the lowest-risk 20% at fixed coverage. These results support the full design for fixed-coverage low-risk selection.

The objective comparisons retain the Full architecture. Each run selects 244 of the same 1,219 test events. Positive excess FS@20 favors Full:

ComparatorMean FS@20Excess over Full, pp [95% cluster CI]
Core0.073771.803 [0.500, 3.110]
Without ranking0.081972.623 [1.100, 4.250]
Without grouped contrast0.072951.721 [0.350, 3.100]
Without invariance0.077052.131 [0.600, 3.670]
BCE-only0.090983.525 [1.800, 5.300]

Full records 13, 14, 13, 14, and 14 detections, giving FS@20 0.055738 ± 0.002245 (mean ± sample SD). It has lower FS@20 than every comparator in all five paired runs (Table S20).

The intervals use 20,000 paired participant-cluster bootstrap resamples shared across variants and runs. Each run reselects its lowest-risk 20% using its frozen calibrator; paired differences are then averaged across runs. AUROC uses the mean of five raw member probabilities. The intervals describe participant-sampling uncertainty; run-wise SDs describe training variability (Appendix A.3.5).

Core uses final GRU states and abruptness supervision; Full adds update-conditioned attention and the complete objective. Single-objective ablations retain abruptness; BCE-only removes all auxiliary losses. These comparisons establish reproducible gains from the complete configuration and the reported objectives in low-risk selection.

Revision locations. Sections 5.1–5.3, Tables 1 and 3–4 (pp. 7–8); Section 4, FS@20 definition (pp. 6–7); Appendix A.3.1, Table S13 (p. 22); Appendix A.3.5, Tables S19, S20 (pp. 25–26).

R3.6 — Actual computation, delivered updates, quality, and delays

Concern. What computational savings and quality trade-offs result after attenuation, splitting, and scheduling delay?

Response. The added 60-participant experiment measures actual delivery, image quality, computation, and delay over 3,600 requests and 36,000 participant-seconds per condition. C0 uses the original current-score path; C1 performs the modification query as a shadow computation with a common submission barrier; R1 uses the recomputed outputs with that same barrier.

OutcomeC0C1R1
Fully completed original requests650660680
Undetected full completions617631655
LOD / resolution PSNR (dB)35.6 / 34.935.6 / 35.035.5 / 34.9
Mean rendering GPU time (ms/frame)7.907.927.88
PC GPU activity (s)25,70025,76025,640
Request-to-submission median / P95 (ms)96 / 230124.44 / 204.44124.44 / 204.44

Attenuation is credited with its actual terminal change. A Split counts as fully complete when its original target is reached within the deadline and type-specific tolerance. Relative to C1, R1 delivers 20 more complete requests and 24 more undetected full completions. Removed triangles increase from 8.25 to 8.50 million, and removed resolution pixels from 61.1472 to 63.8496 million; Table S43 reports all five update types separately.

Reviewer_aewvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper asks whether visual updates in VR can be scheduled so that users are less likely to notice them. The authors propose PERCEPT-VR++ to estimate whether a pending update will be noticed and Safe-UARS to decide whether the update should be applied immediately, delayed, attenuated, or split.

Critical strengths
  1. The evaluation goes beyond offline replay. The main dataset contains observations from 450 participants, and the scheduler is then tested with 55 different participants in a separate closed-loop study. This second experiment matters because replay alone cannot show how scheduling decisions affect the interaction as it unfolds.
Critical weaknesses
  1. The re-query comparison is confounded by latency. Re-query adds about 9–12 ms and applies updates later than current-score. At the millisecond scale studied here, that delay alone could make updates less noticeable. The reported FSR drop from 0.059 to 0.030 therefore does not isolate the effect of rescoring. A latency-matched control is needed.
  2. The scheduler's utility is underspecified. The paper does not define attenuation strength, split behavior, thresholds, or fallback rules, while ER and SUR count Execute, Attenuate, and Split as equivalent successes. Lower FSR and higher ER may therefore come from weaker or fragmented updates rather than more useful work.
Questions for authors
  1. Timing control: Were current-score and re-query updates applied at the same time? If not, please add a latency-matched control. Does the FSR improvement remain under this comparison?
4 comments
AuthorsOpenReview ↗

Response to Reviewer aewv

Response to Reviewer aewv

Revision response — PERCEPT-VR++, Submission 46014

Thank you for the questions about timing and useful delivery. The study program combines a 450-participant dataset, independent 55-participant policy and experience evaluation, and a separate 60-participant, six-scene follow-up. The follow-up contributes 10,800 common-barrier candidates and 14,400 additional fixed-opportunity candidates, paired scoring of displayed modifications, and measurements of full-target completion, quality, and system cost. These complementary experiments connect the re-query mechanism to actual delivery.

R2.1 — Does the re-query benefit survive controls for computation and execution opportunity?

Concern (paraphrase of the weakness and question). The additional query introduces computation and waiting, so the lower FSR may reflect later application rather than rescoring. Were the original conditions applied at the same time, and does the benefit survive a latency-matched control?

Response. The independent 55-participant study evaluates complete policies and user experience; the separate 60-participant follow-up tests the re-query mechanism with controlled timing. The FSR improvement persists after matching computational waiting and after disabling autonomous Delay; rescoring also improves prediction on identical displayed modifications. Each analysis addresses a distinct part of the timing concern.

1. Common computation and submission barrier. The study includes six scenes and 3,600 candidates per condition, using a shared frozen plan model. C0 retains the original current-score rule. C1 and R1 both execute the modified-update query: C1 logs its output but retains the original probability, conservative risk score, and uncertainty gate; R1 uses the recomputed tuple. Both use original scores for Execute. Within each opportunity, original and modified queries share the same 500-ms history cutoff. C1/R1 also share the 0/80/160-ms opportunity schedule, four-frame submission barrier, and missed-barrier rule. Both retain autonomous Delay, so this control targets computational waiting; their opportunity distributions are reported separately. Measured median/P95 request-to-first-software-submission latencies are 124.44/204.44 ms in both conditions. Independent two-query profiling finds 995/1,000 runs ready before the barrier per condition; late outputs cannot trigger automatic late submission.

With extra computation and its submission barrier matched, FSR falls from 50/976 = 5.123% to 32/1,010 = 3.168%, a difference of −1.955 percentage points (95% CI [−3.70, −0.25]). Applications increase from 976 to 1,010 and full-target completions from 660 to 680.

2. A single fixed opportunity with no autonomous Delay. To remove policy-selected waiting, the same 60 participants completed C-fixed and R-fixed, with 7,200 additional candidates per condition. Every request has only the third opportunity, starting nominally at 160 ms, and cannot move to a later opportunity. The first-submission barrier is f2+4 (nominally 204.44 ms); a committed Split ends at f2+6 (nominally 226.67 ms), before the 300-ms deadline. Actual timestamps determine eligibility and completion. Failed gates, timeouts, missed barriers, or infeasible deadlines terminate the request without retry. A Split whose first stage appeared remains applied but incomplete if its second stage fails.

FSR falls from 82/1,560 = 5.256% to 56/1,640 = 3.415%, a difference of −1.842 percentage points (95% CI [−3.082, −0.616]; Holm-adjusted p = 0.02175). FSR remains lower with re-query under the fixed-opportunity protocol. Applied/full-completion counts are 1,560/1,100 versus 1,640/1,140, respectively. This experiment is analyzed separately from the common-barrier study. Both FSR intervals use 10,000 paired participant-cluster bootstrap resamples. Fixed-opportunity p values use 99,999 two-sided participant-level condition exchanges and Holm correction across FSR, ER, SUR, FCR, and undetected full-completion rate.

3. Paired prediction validation on identical displayed events. We compare old and new calibrated probabilities on all 400 C1 modifications actually shown: 316 Attenuate and 84 Split events, with 26 detections. The old probability describes the original request; the new probability describes its modification. Each pair uses the same history cutoff, frozen model/calibrator, and actual detection label; Split has one whole-episode label. C1 selects events using old outputs, while new outputs remain shadow records. Holding the displayed event and outcome fixed tests whether rescoring improves prediction of the modification.

Brier, the primary mechanism metric, decreases from 0.062619 to 0.054445 (difference −0.008174, 95% CI [−0.012525, −0.004278], p = 0.00010); supplementary NLL decreases from 0.251220 to 0.200493. The Brier interval uses 10,000 participant-cluster bootstrap resamples. The two-sided test jointly flips each participant's paired error differences across 99,999 Monte Carlo draws; nine are at least as extreme as observed, giving p = (9+1)/(99,999+1) = 0.00010. The lower prediction error on identical events supports the rescoring mechanism alongside the controlled policy results.

Revision locations. Abstract and Introduction (pp. 1–2); Sections 4 and 5.5/Table 6 (pp. 6, 9); Appendix A.5.1, Tables S35, S39, S42 (pp. 33–38); Appendix A.5.2 (p. 42); Appendix A.5.3 (p. 43). The Conclusion (p. 10) and Reproducibility Statement (p. 11) reflect these controls.

R2.2 — What is delivered, at what quality and cost?

Concern (paraphrase). Attenuation strength, split behavior, thresholds, and fallback rules are underspecified. ER and SUR count all three application actions, so their improvement might reflect weaker or fragmented changes rather than useful work.

Response. Lower FSR accompanies more completed original requests and greater delivered change. We now distinguish application coverage, partial delivery, and original-target completion, and report quality and measured cost alongside these outcomes.

1. Explicit actions, completion, and fallback. The common-barrier experiment contains 720 candidates per update family and condition. Half permit partial endpoints; half require full endpoints and prohibit attenuation. Both categories are rejectable. Mandatory requests bypass perceptual suppression and return to the application for a semantically valid full update.

UpdateOriginal targetAttenuated endpointTwo-step Split
LOD (triangles)100,000 → 50,00075,00075,000 → 50,000
Blur sigma (px)0 → 2.01.21.0 → 2.0
Resolution (px)1000² → 800²880²900² → 800²
Michelson contrast0.40 → 0.600.520.50 → 0.60
Position (mm)0 → +10+6+5 → +10

Continuous attenuation advances 60% of the parameter change; LOD uses the intermediate asset. Split applies one nonrecursive plan at frames f and f+2. Completion requires the original target within 300 ms and the type-specific tolerance. Each request counts once. An interrupted Split remains applied but incomplete, with attained delivery and detection retained; Attenuate ends at its partial target.

The frozen execute/modified-action thresholds are 0.15/0.22, disagreement cutoff 0.08, and minimum attenuation gain proxy 0.05. Opportunities at nominal offsets 0/80/160 ms allow at most two Delay transitions and one modified query per opportunity. Rejection leads to Delay or termination. At the final opportunity, full-endpoint-only requests may use a legal Split meeting the deadline and gate. No admissible application gives Not applied; its detection label remains missing.

2. Whole-plan supervision and independent calibration. A 33-field descriptor augments the proposed-update attributes. One ensemble query predicts whether any stage is detected, without substep queries. An additional offline collection from disjoint subsets of the original 450 participants provided 960 training plans from 32 participants and 240 calibration plans from eight participants, with 30 plans per person and equal counts of Execute, Attenuate, and Split. These new events are separate from the original 9,000 candidates/8,301 valid events. Neither group overlaps the 60-person evaluation cohort. The shared model, calibrator, and thresholds were frozen before closed-loop evaluation.

3. Original-target delivery and quality. All Execute/Split requests in this common-barrier experiment reached their full targets. C1→R1 increases complete requests from 660 to 680 and undetected complete requests from 631 to 655. Dividing by all 3,600 candidates, FCR is 18.333%→18.889%, and undetected full completion is 17.528%→18.194%. ER counts applications, SUR counts undetected applications, and FCR requires completion of the original target.

Delivered change increases in every update family, in its own physical units: removed triangles rise from 8.25 to 8.50 million and removed pixels from 61,147,200 to 63,849,600; Table S43 also reports blur, contrast, and displacement. These descriptive increases show that the lower FSR accompanies more original-target completions and actual delivery.

C1/R1 LOD PSNR is 35.6/35.5 dB. Quality compares submitted eye images with matched high-quality rerenders in original-request ROIs expanded by 16 pixels, using a fixed [0,1000) ms window with 90 slots. Valid eyes, slots, requests, and participants are averaged hierarchically; PSNR converts the resulting MSE to dB. SSIM/LPIPS use full-image spatial maps before ROI reduction. Table S44 reports quality and task outcomes descriptively.

All six LOD/resolution condition-by-type groups retain 720/720 requests, with 99.9228%–99.9537% of their 64,800 slots having at least one valid eye. Appendix A.5.1 now gives eye availability and mutually exclusive missingness reasons. For C1–LOD, 240 invalid eye-slots comprise 140 invisible-ROI, 80 missing-capture, and 20 reference/metric failures. Its 320 repeated-frame slots (0.494%) and 280 runtime-reprojection slots (0.432%) are recorded separately; they need not be mutually exclusive and are not automatically missing. Actual software builds, including scikit-image 0.24.0 and LPIPS 0.1.4, are specified.

4. Measured cost and frame accounting. Across 36,000 participant-seconds per condition, C1/R1 make 10,080/9,904 five-member ensemble queries, averaging 12.8 ms each. Rendering GPU means are 7.92/7.88 ms/frame; included PC GPU activity is 25,760/25,640 s. Tables S45–S46 distinguish query wall-clock products, frame times, and ETW execution-interval unions. Concurrent engines count once; offline evaluation and the headset GPU are excluded.

Per-frame GPU time unions frame-attributed rendering intervals; CPU main-thread time counts execution and excludes waiting. Statistics pool valid complete application frames within each condition. C1 has 3,225,480 valid complete frames, after excluding 120 boundary frames from 3,225,600 identified frames, with no other incomplete timing records. Missed slots require a new frame that is not submission/GPU/synchronization-ready by its runtime deadline. C1/R1 miss 14,580/13,608 of 3,240,000 valid display slots (0.45%/0.42%). Appendix A.5.1 documents these definitions, denominators, and condition-specific counts.

Revision locations.

Main text: Section 4 (pp. 5–7) specifies follow-up training/calibration, action contracts, and application versus full-completion metrics. Section 5.5 and Tables 6–7 (pp. 9–10) report complete and undetected-complete requests, actual delivery, quality coverage, and measured runtime. The Reproducibility Statement (p. 11) points to the sample, quality, software, and frame-accounting details.

Actions and supervision: Appendix A.5.1, Tables S36, S37, S38 (pp. 33–35) defines action contracts, thresholds, whole-plan supervision, and the independent 960/240 training/calibration plans.

Delivery and quality: Appendix A.5.1, Tables S40, S43, S44 (pp. 33–40) reports completions, physical delivery, quality/task outcomes, and the adjacent coverage, missingness, and software-build details.

Cost and timing: Appendix A.5.1, Tables S45, S46, S47 (pp. 33–42) gives query accounting, frame/slot ledgers, GPU/CPU statistics, display deadlines, and request-to-submission latency.

AuthorsOpenReview ↗

Rescoring reduces the detected fraction of applied VR updates at a fixed execution opportunity

Thank you for asking whether waiting could explain the reduction in detections and how much of the requested rendering work actually reached the display.

1. Were the updates applied at the same time? The 55-participant comparison allowed each policy to choose when to act, so execution times could differ. Median/P95 added scheduling delays were 94/226 ms for current-score and 103/238 ms for re-query. Participants detected 39/656 and 21/693 applied requests, respectively. We therefore tested timing directly in a 60-participant, six-scene study (Table 5, p. 8, and Section 5.5, p. 9).

Common submission barrier. Both policies computed the modified-plan score when proposing a modification, using the same 500-ms history cutoff and four-frame submission barrier. C1 logged the new score and chose actions using the request score. R1 acted on the new score. Both could still delay. Median/P95 request-to-first-software-submission times were 124.44/204.44 ms in both conditions. The fraction of applied requests that participants detected fell from 50/976 (5.123%) to 32/1,010 (3.168%, paired difference −1.955 percentage points, 95% CI −3.70 to −0.25). Table 6 (p. 9) and Tables S39–S42 and S47 (pp. 37–38, 43).

Single execution opportunity. Each of 7,200 requests per condition had one decision opportunity at a nominal 160 ms, with no autonomous delay or retry. Detections fell from 82/1,560 applications to 56/1,640 (5.256% to 3.415%, paired difference −1.842 percentage points, 95% CI −3.082 to −0.616, Holm-adjusted p = 0.02175). For the same 400 modifications displayed under C1, rescoring also lowered Brier error from 0.062619 to 0.054445. A simple correction using the request score, action, and planned magnitude yielded 0.061682. The fixed opportunity tests the decision. The paired predictions hold the displayed event and human response constant. Section 5.5 (p. 9), Appendices A.5.2–A.5.3, Tables S48–S51 (pp. 43–45), and Appendix A.8, Table S60 (p. 50).

2. What does each action deliver? Continuous attenuation applies 60% of the requested parameter change. LOD attenuation uses an intermediate asset. A split commits two stages two frames apart. In the common-barrier study, half the requests allowed partial delivery and half required the full target if applied. Execute and modified-plan risk thresholds were 0.15 and 0.22, the uncertainty cutoff was 0.08, and attenuation required a gain proxy of at least 0.05. A rejected plan could wait for the next eligible opportunity. At the last opportunity, a full-target request could use a legal split that passed the risk gate and could meet the deadline. Execution ratio counts applied requests and successful update ratio counts undetected applications, each divided by all candidates. A split counts once even if interrupted. Full completion requires reaching the requested target within tolerance and 300 ms. The whole-plan predictor was trained on 960 plans and calibrated on 240 from disjoint groups of 32 and eight participants, independent of the evaluation cohort. Section 4 (pp. 5–6) and Appendix A.5.1, Tables S36–S38 (pp. 34–36).

Delivered work, quality, and cost. Among 3,600 candidates per condition, C1 and R1 completed 660 and 680 full targets, respectively. Of these, 631 and 655 were completed undetected. Re-query increased actual reductions from 8.25 to 8.50 million triangles and from 61.15 to 63.85 million pixels. LOD PSNR was 35.6/35.5 dB for C1/R1 over fixed 1,000-ms windows. Across equal observation periods, the policies made 10,080/9,904 queries and averaged 7.92/7.88 ms of rendering GPU time per frame. Tables 6–7 (pp. 9–10) and S40–S46 (pp. 38–42) report completions, all five update families, image quality, task outcomes, and measured costs.

Contribution. A request to reduce resolution from 1000 × 1000 to 800 × 800 can stop at 880 × 880 or pass through 900 × 900 on the way to the full target. Even at the same decision time, the viewer sees different changes. PERCEPT-VR++ predicts the risk of the plan that will reach the display. VR-Mask++ makes that question measurable by linking the preceding 500 ms of eye and head motion, scene context, update attributes, and a detection report for 8,301 updates from 450 participants across nine scenes. The data span controlled detection, natural search, and reading or interface tasks. We plan to publicly release the complete anonymized processed dataset after publication within the approved consent and data-sharing terms (Sections 1 and 4, pp. 1–2 and 5–6, and Appendix A.7, p. 49).

We believe these revisions address all the questions you raised, and we would be grateful if you could reconsider your rating in light of the experimental results.

Reviewer_9W1FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a method to estimate the pre-update risk for VR systems. The problem is interesting and the method proposed seems to achieve reasonable outcome. There are also fundamental problems that need to be further addressed.

Critical strengths
  1. this paper analyze a problem that uses the data driven methods to improve the VR experience with collected feedback from the use.
  2. the paper presents a model trained to estimate the human perception experience from the device.
Critical weaknesses
  1. predicting future Gazing and egocentric data flow for later usage in render in common. I am wondering the specific novelty involved in the design and implementation.
  2. More analysis on the accuracy and stability of the collected dataset is important. it's not obvious if the dataset can effectively reflect the problem. Changes happen consistently in VR environment. how can the dataset distinguish valid changes vs unexpected changes?
  3. what is the speed of the inference? will the inference speed impact the usability of the system?
  4. it's not clear how the accuracy translates to the experience enhancement in the paper.
  5. comparison with more ML based SOTA methods are needed
Questions for authors

More analysis on the accuracy and stability of the collected dataset is important. it's not obvious if the dataset can effectively reflect the problem. Changes happen consistently in VR environment. how can the dataset distinguish valid changes vs unexpected changes?

I am interested to know in data collection and experiments, how do you differentiate between necessary updates vs unnecessary updates. also what if necessary updates cause discomfort etc. How will these cases be handled?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9W1F

PERCEPT-VR++, Submission 46014

Thank you for the questions about contribution, data, runtime, experience, comparisons, and necessary updates. The empirical contribution spans 450 participants for nine-scene data collection, 55 independent participants for policy and experience evaluation, and 60 new participants for six-scene mechanism and system validation. The revision adds repeat measurements, an event-attribution audit, matched baselines, and application-routing evidence. We address each concern and identify the revisions below; concern summaries are paraphrases.

R1.1 — Novelty beyond gaze prediction

Concern. Predicting gaze and egocentric signals for rendering is established; what is novel in the design and implementation?

Response. The contribution is predicting perceptual risk for a specific proposed update, then recomputing that risk when the scheduler changes the update. Gaze prediction estimates future viewing behavior. Our output is the probability of noticing a particular update, conditioned jointly on pre-update eye/head dynamics, scene context, and update attributes. Two updates can therefore receive different risks under the same behavioral history. Attenuation or splitting changes the object of the prediction, motivating a new query before committing the modified plan.

The implementation makes this dependence explicit: update attributes enter both the temporal-attention query and the wide branch, and the scheduler queries the proposed modified attributes using the same pre-update history cutoff. Three tests examine the information, prediction, and decision components of this contribution:

  • Removing update attributes changes AUROC from 0.831 to 0.793.
  • On the same 400 displayed modified events, rescoring reduces Brier error from 0.062619 to 0.054445 (difference −0.008174; 95% CI [−0.012525, −0.004278]; participant-cluster p = 0.00010).
  • With execution opportunity fixed and autonomous Delay disabled, re-query reduces FSR from 5.256% to 3.415% (difference −1.842 percentage points; 95% CI [−3.082, −0.616]; Holm-adjusted p = 0.02175).

These results connect the update-conditioned formulation to its implementation and controlled online benefit. We have reorganized the contribution and related-work discussion around this connection.

Revision locations. Sections 1–3 (pp. 1–5); Table 3 (p. 8); Section 5.5 (pp. 9–10); Appendix A.1 (p. 15); Appendix A.5.2 (p. 42); Appendix A.5.3 (p. 43).

R1.2 — Dataset accuracy, stability, and unexpected changes

Concern. Does the dataset reliably represent the problem, and how are intended updates distinguished from unexpected changes in a continuously changing VR environment?

Response. We added repeat measurements with 45 new participants and a structured audit of all 8,301 original update events, followed by evaluation on an audit-filtered test subset with the original models frozen. These analyses separately test label repeatability, concurrent-change attribution, and the stability of the prediction results.

The original collection contains 9,000 candidates from 450 participants across nine scenes: 8,301 executed updates and 699 no-update catch trials. Catch trials are a separate trial type. Each target update has a request/object identity, attributes, intended terminal state, actual timing, and feedback record. The participant-disjoint training/validation/test split remains 5,828/1,254/1,219, and model inputs retain only pre-update information.

Added evidenceDesign and accountingKey result
Label repeatability45 new participants, five per scene; 10 legal templates per person, repeated in two sessions; 450 paired labels370/450 agree (82.22%); Cohen's κ = 0.60; positive/negative agreement 73.33%/86.67%
Concurrent-change auditAll 8,301 events screened; 2,359 manually reviewed, including all 1,351 additional-change/anomaly events, all 108 insufficient-evidence events, and 900 sampled events6,842 without extra discrete changes; 1,131 with co-occurrence but no predefined confounding evidence; 220 with confound indicators; 108 unassessable
Frozen-model sensitivityExclude 220 with confound indicators and 108 unassessable records; retain 7,973 overall, including 1,170 of the original 1,219 test eventsFull AUROC 0.831→0.834; FS@20 0.056→0.055; the update-attribute AUROC advantage is 0.038→0.037

Repeated templates preserve scene/task, object/region, update type and magnitude, trigger configuration, feedback question, and platform. The actual paired-event interval has median 48.3 hours and IQR 47.9–48.9 hours. The 900 repeated-update events and 180 additional catch trials are recorded separately from the original dataset. The paired-label counts are 260 both undetected, 110 both detected, and 40 in each discordant direction.

The audit uses actual observation/feedback windows and request, object, region, and plan-stage identifiers. Normal continuous scene motion remains part of the scene context. Among the 1,351 additional discrete changes/anomalies, 1,104 are expected background events, 183 unexpected updates, and 64 visible technical anomalies. The 1,131 co-occurrences without predefined confounding evidence were outside the feedback-covered window or had no visible effect in the presented image. The 900-event manual sample identified no additional missed unexpected updates or visible technical anomalies.

Sensitivity evaluation keeps each original checkpoint, calibrator, F1 threshold, and metric procedure fixed, without retraining or refitting. On the retained test subset, Full/w/o-update-attributes AUROC is 0.834/0.797, and FS@20 is 0.055/0.086. FS@20 retains its five-run calculation, selecting the lowest-risk 234 of 1,170 events per run instead of 244 of 1,219. Thus, the main result and the contribution of update attributes persist after filtering potential attribution problems.

The original catch diagnostic remains 39/699 positive reports (5.579%; 95% CI [3.929%, 7.303%]). Separately, all 53 accounting checks pass for the 55-participant closed-loop study: 1,921 applied candidates have detection labels, and 5,504 unapplied candidates retain missing labels. The revision now reports catch responses, repeated-label agreement, attribution, and accounting as distinct evidence.

Revision locations. Section 4 (pp. 5–7); Appendix A.2.1 (p. 15); Appendix A.2.2 (p. 17); Appendix A.2.3 (p. 19); Appendix A.2.4, Tables S9, S10, S11, S12 (pp. 20–22); Appendix A.4.2, Table S32 (p. 30).

R1.3 — Inference speed and usability

Concern. How fast is inference, and does its latency affect usability?

Response. The measured system uses event-triggered inference, scheduled update submission, and separately measured rendering. The original prediction/calibration/scheduling pipeline averages 12.6 ms/event on an RTX 4090 / Intel i9-13900K workstation: FP32, one event per input batch, a vectorized five-member ensemble, 100 warm-ups, and 1,000 synchronized measurements. This is model-side timing.

The follow-up whole-plan model averages 12.8 ms per complete K=5 logical query. Its complete ledger includes initial, post-Delay, and modified/shadow queries: C0/C1/R1 make 9,600/10,080/9,904 queries over 36,000 participant-seconds per condition. R1 therefore averages 0.2751 queries per participant-second.

For C1/R1, window-ready-to-decision means are 26.4/26.5 ms, with P99 33.0/33.1 ms. Both are ready before the common four-frame submission barrier in 995/1,000 profiling records; formal operation skips a missed opportunity. Including policy-selected waiting, measured request-to-first-software-submission latency has median 124.44 ms and P95 204.44 ms in both conditions.

Under this workload, C1/R1 rendering GPU means are 7.92/7.88 ms/frame, CPU main-thread means 3.9/3.9 ms/frame, and missed display-deadline rates 0.45%/0.42%, each over 3,240,000 target slots. Query frequency, submission latency, and frame/deadline measurements together characterize usability for the tested PCVR scheduler. The updated timing definitions distinguish query wall-clock duration from per-frame rendering and measured software submission.

Revision locations. Section 5.5/Table 7 (p. 10); Appendix A.7, Table S59 (p. 48); Appendix A.5.1, Tables S39, S45, S46, S47 (pp. 33–42).

R1.4 — Prediction and participant experience

Concern. How do prediction and detection improvements translate into better experience?

Response. The original 55-participant study provides paired subjective evidence: current-score reduces distraction by 0.619 points and increases comfort by 0.456 points relative to prediction-only. Both use 1–7 scales; lower distraction and higher comfort are better. We added participant-bootstrap 95% intervals and two-sided paired tests with Holm adjustment across the four contrasts.

OutcomeContrastMean difference [95% CI]Holm-adjusted p
DistractionCurrent-score − prediction-only−0.619 [−0.759, −0.475]8.47 × 10⁻¹¹
ComfortCurrent-score − prediction-only+0.456 [+0.337, +0.577]4.20 × 10⁻⁹
DistractionRe-query − current-score−0.054 [−0.196, +0.087]0.457
ComfortRe-query − current-score+0.105 [−0.014, +0.222]0.178

The corresponding pooled-FSR differences are −0.0454 [−0.0770, −0.0172] for current-score versus prediction-only and −0.0291 [−0.0527, −0.0052] for re-query versus current-score, using 10,000 paired participant-bootstrap replicates. These policy-level comparisons connect fewer detected updates with improved reported experience for current-score. Re-query's additional rating differences are not significant after adjustment; its demonstrated additional benefit is lower FSR. The follow-up further reports original-target completion, visual quality, task correctness, and rendering cost alongside these subjective outcomes.

Revision locations. Section 5.5 (p. 10); Appendix A.4.1, Tables S27, S28, S31 (pp. 26–30); Appendix A.4.2, Table S33 (p. 30); Appendix A.5.1, Tables S43, S44 (pp. 33–40).

R1.5 — Coverage of machine-learning comparisons

Concern. Comparisons with more machine-learning SOTA methods are needed.

Response. We added Transformer+U and ModernTCN+U with the same input information, data split, and model-selection budget, alongside M2P BCE-only and M2P-Full. All four receive the same 500-ms eye/head/context sequences, valid-step masks, update/static attributes, preprocessing, and participant-disjoint 5,828/1,254/1,219 split. This extends the existing linear, tree-ensemble, MLP, feature-token, and gaze-based comparisons with controlled temporal baselines.

ModelTest AUROCParameters/member (M)Searched configurations
Transformer+U0.8110.7216
ModernTCN+U0.8020.7816
M2P BCE-only0.8160.8216
M2P-Full0.8310.8216

Every model uses validation AUROC for selection, a maximum of 120 epochs, early-stopping patience 20, batch size 128, and five final members with seeds 11/22/33/44/55. Ensemble probabilities are averaged before evaluating AUROC on the same 1,219 test events. Beta calibration is fitted on validation predictions; FS@20 retains separate per-run calibration and selection before averaging.

The comparison has two complementary levels. Transformer+U, ModernTCN+U, and M2P BCE-only share class-weighted BCE supervision, assessing their encoding/fusion designs with matched information. M2P-Full adds the CF, invariance, ranking, and abruptness objectives to the same M2P architecture. M2P-Full reaches AUROC 0.831 under this common evaluation protocol, compared with 0.811/0.802 for the added temporal baselines and 0.816 for BCE-only. Appendix A.3.4 reports configurations and metric definitions, including ModernTCN+U's beta-calibrated Brier 0.148 and FS@20 0.080.

Revision locations. Abstract (p. 1); Section 4 (pp. 5–7); Section 5.1/Table 2 (pp. 7–8); Appendix A.3.1 (p. 22); Appendix A.3.4, Tables S17, S18 (pp. 24–25).

R1.6 — Necessary updates and discomfort

Concern. How are necessary and unnecessary updates differentiated, and what happens when a necessary update causes discomfort?

Response. Necessity is specified by application semantics, and mandatory requests bypass perceptual rejection. We added retrospective request-class annotation and a separate ledger of 360 mandatory requests from the same 60 participants' common-barrier sessions. The 8,301 original update events are semantically omissible; the 699 catch trials receive a not-applicable necessity label. The 7,425 original closed-loop, 10,800 common-barrier, and 14,400 fixed-opportunity candidates permit rejection. Within the common-barrier pool, 5,400 permit partial endpoints and 5,400 require full endpoints if applied. Requiring a full endpoint and requiring execution are recorded separately.

The 360 mandatory requests are outside the 10,800-candidate pool. All occurred outside each participant's 600-s effective measurement period for the corresponding condition, without overlapping the main candidates' 500-ms input, 1000-ms quality, or feedback windows. Their associated pauses, reassessments, and follow-up also occurred outside that measurement period.

There are 120 immediate requests where an obsolete target/instruction conflicts with the current task state, and 240 deadline-bound requests needed to synchronize visible state after a confirmed interaction. The application submits the complete original target at the first semantically legal, presentable frame or within the required deadline. For deadline-bound requests, completion requires tfull−trequest≤300 mst_{\mathrm{full}} - t_{\mathrm{request}} \le 300\ \mathrm{ms}; tfullt_{\mathrm{full}} is the timestamp of the actual software-submission frame attaining the original target and type-specific tolerance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Request for Further Reviewer Engagement Before the Discussion Deadline

Dear Area Chair,

We sincerely apologize for reaching out once again, and we greatly appreciate your continued time and efforts in coordinating the review process.

We submitted our detailed rebuttal on November 13, 2026, carefully addressing the reviewers' comments and concerns. We are very grateful that one reviewer has now kindly responded and engaged in the discussion. However, we have not yet received follow-up responses from the remaining reviewers.

With less than 24 hours remaining before the discussion period closes, we sincerely hope to have the opportunity to discuss any outstanding concerns with the other reviewers as well.

We fully understand that reviewers have busy schedules and deeply appreciate their contributions. If possible, we would be extremely grateful if you could kindly encourage the remaining reviewers to revisit our rebuttal and share any further feedback or questions before the deadline.

We remain fully available to provide prompt clarifications and would greatly value any additional constructive discussion.

Thank you very much for your patience, understanding, and continued support. We sincerely appreciate your assistance.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.