- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces Deep-ARViM, a perceptual visibility benchmark and learned metric for optical see-through AR. The dataset contains 67,984 pairwise visibility judgments and 21,027 visibility-matching responses. Visibility is modeled as a learned feature distance between a background image and its AR composite, using pretrained vision backbones adapted through channel weighting, LoRA, or full fine-tuning. Models trained on pairwise choices are evaluated on those choices, subjective visibility ratings, and brightness-matching responses without matching-task supervision. A separate 17-participant optical study tests transfer from simulated compositing to real additive light.
Critical strengths
-
The paper addresses a useful perceptual problem with a substantial human-judgment dataset. Combining pairwise choices with visibility matching provides complementary evidence about relative visibility and the brightness needed to achieve comparable visibility. This gives the benchmark value beyond a single prediction task.
-
The evaluation connects binary supervision to other perceptual judgments. Models trained only on pairwise choices improve brightness-matching predictions, and the web-trained DINOv2-LoRA model reaches 79.1% consistency-weighted accuracy in the optical study, compared with 68.6% for the learned low-level baseline (Appendix D). These results support the usefulness of the learned score across the tested protocols.
Critical weaknesses
-
Generalization to unseen content is not sufficiently established. Appendix C.2 describes an 80/20 split over stimulus pairs and selects checkpoints using validation performance, but does not clarify whether foreground images or background scenes are shared across splits. Pair-level separation alone does not establish content-level separation, and no separate final test set is described. This matters for a metric intended to handle arbitrary foreground–background combinations.
-
The modeling contribution mainly adapts existing perceptual-distance learning and pretrained backbones, so much of the paper’s value rests on the benchmark and transfer evidence. That evidence remains limited to mostly static stimuli with centered, fixed-size foregrounds, plus a small optical study. It supports a metric for the tested conditions, while broader claims about general AR visibility and practical display adaptation remain less established.
Questions for authors
-
Will the dataset (images, metadata, human annotations) and code be released, including licensing details for Ego4D/Aria backgrounds and Gemini-generated foregrounds?
-
Can the authors clarify overlap across the training and validation splits at the levels of foreground images, background scenes, and source videos? Are results available on a content-disjoint test set that was not used for hyperparameter or checkpoint selection?
-
Were the stimuli used for visibility matching and the optical study excluded from 2AFC training? Please distinguish transfer to new judgments from transfer to unseen content, and provide uncertainty estimates for the transfer results by resampling stimuli or participants.