- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents PaGE for third-person gaze target estimation. It combines scene–head interaction modules, spatially aligned positional encoding, staged backbone fine-tuning, and feature distillation on unlabeled images. The resulting models improve performance on three established gaze benchmarks, transfer to additional evaluation sets, and provide smaller students with substantially reduced computation.
Critical strengths
-
The empirical improvements are substantial and cover multiple datasets and model sizes. The zero-shot results in Tab. 4 provide useful evidence beyond the datasets used for supervised training.
-
The paper systematically develops its architecture and training recipe. Tab. 1 and the additional architectural and distillation ablations make the implementation choices unusually transparent.
-
The lightweight students and the inference measurements increase practical value. The discussion of heatmap / coordinate bias is also useful.
Critical weaknesses
-
The headline claim of human-level performance needs a narrower interpretation. On GazeFollow, agreement with a multi-annotator consensus is different from measuring gaze against independently observed eye-tracking ground truth. In addition, VAT and ChildPlay localization errors remain above the reported human baselines.
-
The contribution of individual architectural changes is less certain than the overall system improvement. Please clarify which validation data governed design selection and provide uncertainty estimates for the central SIM versus token-concatenation comparison.
-
Practical deployment is supported by desktop GPU/CPU profiling, but the broader robot and consumer-device claim remains less established.
Questions for authors
-
How were the human baselines constructed, and how comparable are their inputs and annotation protocols to the model evaluations? Can the claims explicitly distinguish consensus prediction from physical gaze accuracy?
-
Does the 16-ms GPU timing include head detection and preprocessing, and how does cost change with the number of queried people?
4 comments
Response to Reviewer GpW4 (2/3)
Q2 Deployment & Profiling: The 16-ms GPU timing includes head detection but not preprocessing. We ran the experiment on the RTX 4090 again to include preprocessing. Below are additional results with different number of heads.
| Person Per Frame | FPS on RTX 4090 |
|---|---|
| 1 | 46.23 |
| 2 | 43.63 |
| 4 | 41.52 |
| 8 | 36.75 |
For profiling on robots and mobile devices, we deployed PaGE ViT-S on an Nvidia Jetson AGX Orin 64GB, a common edge AI and robotics platform using the TensorRT backend, and an iPhone 17 Pro Max using the Core ML backend to gauge the model's edge performance. All results here use the standard 512*512 scenes and 256*256 heads, and include preprocessing, head detection, and PaGE inference.
| Person Per Frame | FPS on Jetson AGX Orin |
|---|---|
| 1 | 16.45 |
| 2 | 15.02 |
| 4 | 11.65 |
| 8 | 8.01 |
| Person Per Frame | FPS on iPhone |
|---|---|
| 1 | 31.55 |
| 2 | 24.57 |
| 4 | 17.03 |
| 8 | 9.96 |
Response to Reviewer GpW4 (3/3)
W2 Validation-set Results: During the development of PaGE, we used a held-out validation set that contained 10% of GazeFollow train set images chosen randomly, using the remaining 90% as the train set. Since GazeFollow provides 1 annotation per image for the train set instead of several annotations per image for the test set, there is no longer a distinction between Avg L2 and Min L2, and the results below are not directly comparable to GazeFollow test set results. Here are the results (mean$\pm$std over 3 random seeds) for key head branch design choices from our development logs.
| Step | AUC | L2 |
|---|---|---|
| Gaze-LLE w/ DINOv3 | 0.9627$\pm$0.0002 | 0.1152$\pm$0.0004 |
| Token-concat Head Branch | 0.9648$\pm$0.0003 | 0.1108$\pm$0.0005 |
| 3 Layers 5 Layers | 0.9652$\pm$0.0002 | 0.1086$\pm$0.0004 |
| DINO Feature Dropout | 0.9658$\pm$0.0001 | 0.1085$\pm$0.0006 |
| SIM Head Branch | 0.9664$\pm$0.0001 | 0.1074$\pm$0.0002 |
Reply to authors
Thanks for your additional information, my concerns are solved. I'll keep my original score.
Response to Reviewer GpW4 (1/3)
Thanks for your support. We hope the results below address your concerns.
Q1 Human Baseline: The human baselines are the official results reported in the dataset papers. These numbers have since been cited extensively in gaze target estimation literature. The human baseline evaluation protocols for the three datasets are summarized below.
L2 error, the localization metric, is directly comparable between the human baselines and our model. For the In/Out AP comparison, the model outputs a probability in the range [0, 1], while humans provide a binary classification (either 0 or 1). While its suitability is debatable, it has been used extensively in previous work and we inherit this setup. The comparability of AUC is less certain since humans annotate a point, not a heatmap, although many previous paper still include it in the comparison. We also agree that the human results are inter-annotator agreement metrics, not physical gaze accuracy computed with actual ground truth. Taking these factors into consideration, we will modify our main claim from "human-level performance" to "matching reported human inter-annotator gaze localization metrics (L2) on GazeFollow while significantly closing the gap on VAT and ChildPlay".
References
[1] Recasens, Adria, et al. "Where are they looking?." Advances in neural information processing systems 28 (2015).
[2] Chong, Eunji, et al. "Detecting attended visual targets in video." 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020.
[3] Tafasca, Samy, Anshul Gupta, and Jean-Marc Odobez. "Childplay: A new benchmark for understanding children’s gaze behaviour." 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023.