acceptodds
Under review as a conference paper at ICLR 2027

GazeGGT: Geometry-Grounded Transformer for Uncalibrated Multi-View Gaze Estimation

Abstract

Robust 3D gaze estimation remains challenging because monocular appearance cues are inherently under-constrained: eye visibility, head pose, and self-occlusion can make the same gaze direction appear substantially different across views. Multi-view observations offer a natural remedy, but existing multi-view gaze estimators typically depend on fixed camera layouts, calibrated cameras, or test-time adaptation, limiting their applicability in flexible real-world environments. We study the underexplored problem of uncalibrated multi-view gaze estimation, where multiple synchronized face observations are available but camera intrinsics, extrinsics, and view configurations are unknown at inference. We propose GazeGGT, a geometry-grounded transformer for unconstrained, calibration-free multi-view gaze estimation. GazeGGT encodes an arbitrary view number of face crops into per-view features and employs alternating multi-view attention to fuse information across views, followed by separate decoders for per-view 3D gaze directions and camera poses. Unlike prior multi-view methods, GazeGGT requires no camera parameters and naturally supports a variable number of input views, enabling flexible camera configurations. We further introduce a Geometry-Grounded Supervision Paradigm that jointly supervises gaze and camera pose predictions, providing the geometric grounding necessary for accurate calibration-free gaze estimation. Experiments show that GazeGGT achieves competitive performance in both single-view and dual-view settings and consistently benefits from increasing numbers of input views. Ablation studies further demonstrate the importance of the Geometry-Grounded Supervision Paradigm for calibration-free estimation. Our results establish a practical and extensible approach to gaze estimation in unconstrained multi-camera environments.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
gaze estimation, eye tracking, multi view, transformer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VY9fOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GazeGGT, which solves a meaningful 3D gaze estimation problem: uncalibrated multi-view gaze estimation from arbitrary number of synchronized face crops. GazeGGT takes multiple face images as input and learns view-specific feature for each image, which are fused through alternating multi-view attention. The author further utilize a Geometry-Grounded Supervision Paradigm that jointly supervises gaze and camera pose predictions by constraining the relative rotations among different views. Then separate decoders estimate 3D gaze directions and camera poses for each view. Two multi-view datasets ETH-XGaze and GazeGene are evaluated through in-domain and cross-domain experiments.

Critical strengths
  1. The perspective of "arbitrary uncalibrated views" for 3D gaze estimation is interesting and this paper makes a good foundation and baseline in this domain.
  2. Comprehensive experimental results are provided across five gaze datasets and achieves improvement in most single view datasets and multi-view datasets. The cross-dataset evaluation validates the effectiveness of multi-view inference.
Critical weaknesses
  1. Although I like the idea of unconstrained, calibration-free multi-view gaze estimation, the Geometry-Grounded Supervision Paradigm is not novel to me. Similar relative rotation constraints have been proposed in [1], which is not cited as a related work in this paper. The central idea of rotation consistency in [1] is constraining a invariant gaze direction in the same global coordinate system between image pairs under different camera poses. Correspondingly, the ablation study in Table 6 shows that GazeGGT without Geometry-Grounded Supervision term only performs slightly worse than the full model on both two datasets. This raises the question of the contribution of GGSP theoretically and experimentally.

[1] Bao, Y., Liu, Y., Wang, H. and Lu, F., 2022, June. Generalizing gaze estimation with rotation consistency. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 4197-4206). IEEE.

  1. Although the paper is targeting at unconstrained multi-view gaze estimation from uncalibrated images, but the relative camera angles of ETH-XGaze dataset are fixed (which means ETH-XGaze use pre-defined 18 camera orientations, I'm not sure about if GazeGene randomize camera angle for each subject). Based on this limitation, the model is trained and evaluated within the same camera pose distribution. It is unclear how the model performs in real-world multi-view uncalibrated images which are out of the pose distribution of ETH-XGaze and GazeGene. Since the author also mentioned that at test time, view order is not shuffled. The model might already have the prior knowledge of the pose distribution in the test set.
Questions for authors

In Table 6, the random initialized model perform much worse than Pi3X initialized model. At the same time, the DINOv2 initialization performs slightly worse than the geometry-aware model in both N =1 and N = 9 cases. The results in Table 6 raises three essential questions:

  1. The architecture of the GazeGGT model largely builds upon the Visual Geometry Grounded Transformer and the Pi3X model. What is the technical novelty of GazeGGT beyond adapting these two models to 3D gaze estimation task?

  2. How much does GazeGGT rely on pret-rained geometry models like Pi3X? Since Pi3X already have abilities for geometry-aware representation learning. What will be the gaze detection performances using frozen Pi3X weights?

  3. The improvements after introducing GGSP is subtle, the contribution of GGSP is unclear to me.

In Section 4.3, please fix the text “Appendix Table ??”

2 comments

Dear Reviewer VY9f,

Thank you for your careful review and constructive feedback. And we address each concern as follows.

Q1. Contribution beyond applying Pi3X to gaze. Thank you for recognising our conceptual contribution: uncalibrated, arbitrary-view 3D gaze estimation, which provides a foundation and a baseline for this setting.

Technically, we contribute three things:

  1. Design choices for multi-view fusion. Setting camera pose prediction aside, we first established what fusion accuracy requires: whether to add an extra gaze token (no), which training priors matter, whether the Pi3X backbone can be frozen, and how long to train. We will organise this reasoning and the experiments in the revised paper.
  2. GGSP for calibration-free inference. We then designed GGSP so that the model can predict both gaze and camera rotation accurately. Without it, uncalibrated multi-view accuracy degrades sharply (Q3).
  3. Broad evidence. Single-view and multi-view, in-domain and cross-domain, single-dataset and joint training. This includes cross-rig transfer between ETH-XGaze and GazeGene, which tests generalisation to unseen camera poses (W2).

Q2. Reliance on Pi3X; frozen Pi3X. Without Pi3X's pretraining on large-scale view combinations data, cross-domain transfer does not hold (initialisation ablation, Table 5 in the paper). Freezing the backbone and training only the gaze head is not sufficient either. Ref-Avg (°) error on ETH-XGaze:

N1249
Frozen Pi3X6.736.846.594.75
GazeGGT (full)3.903.492.882.51

The frozen model also improves with N, but its error stays far above the full model.

Q3. Contribution of GGSP. GGSP is what makes calibration-free use possible: at inference the model relies on the camera rotations it predicts itself. The relevant metric is Ref-Avg. With GGSP, the model predicts gaze and rotation well together, so even cross-domain, uncalibrated Ref-Avg error falls below single-view at every N. Without GGSP it does not. Ref-Avg (°), trained on ETH-XGaze, tested on GazeGene:

N1249
GazeGGT (full)11.3810.5810.1510.43
w/o GGSP11.1311.5111.0513.44

Without GGSP the error is above the single-view value at N=2 and N=9, and reaches 13.44° at N=9. In-domain (ETH-XGaze, 3 training runs each), Ref-Avg at N=9 is 2.47 ± 0.03 (full) vs 4.24 ± 0.14 (w/o GGSP).

W1. GGSP vs. [1] (Bao et al., CVPR 2022). [1] uses known camera rotations during training to make single-view predictions rotation-consistent. In GazeGGT the relative rotation is a prediction target of the camera head, estimated from images alone at test time, which is what removes calibration at inference. We will cite and discuss [1] in Related Work.

W2. Does ETH-XGaze ↔ GazeGene transfer show uncalibrated performance? The two datasets define cameras very differently. ETH-XGaze is a real-world capture with a fixed 18-camera rig. GazeGene is a synthetic dataset rendered from 9 cameras per frame. Measured from the released calibration and labels:

ETH-XGazeGazeGene
Cameras per frame189
Neighbouring cameras13–33° apart15–30° apart
Largest camera separationabout 125°about 75°

How much do the camera configurations overlap? Take the GazeGene-trained model tested on ETH-XGaze. For each camera combination in ETH-XGaze, we find the closest combination in the GazeGene rig, in terms of the relative rotations between its cameras. Share of ETH-XGaze combinations with a GazeGene match within the given error:

Views per combinationMatch within 3°Match within 5°
22.0%9.8%
30.0%0.4%

Even for pairs, only 2.0% of ETH-XGaze combinations have a GazeGene counterpart within 3°, and for three views the overlap is essentially zero: ETH-XGaze has wide baselines up to 125° that the 75° GazeGene rig does not cover. So GazeGene → ETH-XGaze is evaluated on camera configurations unseen in training, without any calibration. Ref-Avg (°), random camera subsets:

Train → TestN=1N=2N=9
ETH-XGaze → GazeGene11.3810.259.53
GazeGene → ETH-XGaze9.197.987.01

Error decreases with N in both directions.

Minor errors. Thank you for pointing out the unresolved references. We will fix it and proofread the full text in the revised version.

AuthorsOpenReview ↗

Follow-up response

Dear Reviewer,

Thank you again for your careful review. As the discussion period ends today, we would like to briefly follow up on our response. Please let us know if you have any further questions; we would be happy to address them before the deadline. If our responses have resolved your concerns, we would be grateful if you would consider reflecting this in your final assessment.

Thank you and best wishes!

Reviewer_H1xxOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GazeGGT (geometry-grounded transformer) for the task of multi-view gaze estimation. Specifically, they target uncalibrated/calibration-free gaze estimation, where the camera intrinsics, extrinsics and configurations are not known at inference time. This is in contrast to many previous methods that do not generalize to new camera layouts or require calibrated cameras. The authors enable this in 2 ways: usage of the Pi3X architecture (for simultaneous camera parameter estimation and per-camera gaze estimation) with geometrically grounded constraints and by training on a mix of 2 multi-view datasets (ETH-XGaze, GazeGene) and 3 single-view datasets (MPIIFaceGaze, EyeDiap, Gaze360). The results show that the proposed model, when trained on “SV+MV+joint-trained” performs favorably compared to UniGaze-H, while showing improved gaze errors with more camera views, both when using ground-truth camera poses and predicted camera poses. In relation to existing dual/two-view gaze estimation methods, GazeGGT performs respectably.

Critical strengths

The paper is written well, with good motivations and a clear statement of the benefits of GazeGGT. The experimental results are strong, with each table showing both within- and cross-dataset performances at a varying number of cameras. The strong single-view results in Table 2 may indicate that the proposed GazeGGT could be used in both scenarios, making it generally applicable.

Critical weaknesses

The paper proposes a novel task and proves their performance quite well. However, method-wise it is a Pi3X model that has been lightly tuned using rather standard losses. What might the authors say about this? Can we consider the method/model to be novel?

Secondly, in which practical real-world scenarios do the authors expected there to be multi-view camera setups where gaze-tracking needs to be done… but the cameras are not calibrated? I have a hard time imagining that very specific situation. Furthermore, even if we accept that such scenarios may be abundantly available/possible, does GazeGGT allow for accurate metric-space gaze rays (with 3D rotation and 3D translation) to be defined?

Questions for authors

The “Relative-rotation supervision” described in line 232~235 only discuss rotations, however, the ETH-XGaze cameras do contain relative translations, and due to the difference in how the head-coordinate-system and the gaze ray is defined, translation may have a role, strictly speaking. Also, how is the predicted camera translation used?

Can the GazeGGT framework be expanded to include weakly-supervised learning, where some frames (or people) may not have gaze labels?

In the abstract, it’s mentioned that camera intrinsics can be unknown at inference time. Is this the case? The focal length, in particular, can be important for determining the exact 6-D gaze ray in real-world coordinates.

Miscellaneous:

  • Many citations are done in citet style, when they should be done in citep style.
  • There’s a broken table reference in line 370
3 comments

Dear Reviewer H1xx,

Thank you for your thoughtful review and helpful suggestions. We address each concern below.

W1. Contribution beyond applying Pi3X to gaze. Thank you for recognising our motivation and conceptual contribution: uncalibrated, arbitrary-view 3D gaze estimation with a single model. Technically, we contribute three things:

  1. Design choices for multi-view fusion. Setting camera pose prediction aside, we first established what fusion accuracy requires: whether to add an extra gaze token (no), which training priors matter, whether the Pi3X backbone can be frozen, and how long to train. We will organise this reasoning and the experiments in the revised paper.
  2. GGSP for calibration-free inference. We then designed GGSP so that the model predicts both gaze and camera rotation accurately. Without it, uncalibrated multi-view accuracy (Ref-Avg) degrades sharply:
  3. Broad evidence. Single-view and multi-view, in-domain and cross-domain, single-dataset and joint training, including cross-rig transfer between ETH-XGaze and GazeGene, which tests generalisation to unseen camera poses.

W2a. Practical scenarios. Multi-camera gaze estimation is promising in smart meetings and in-vehicle interaction. Calibration-free use matters for easy deployment, and for camera shifts or user adjustments in practice. Our method can also take calibrated camera poses for cross-view fusion, which gives a small gain.

W2b. Metric-space gaze rays. Not supported. We focus on jointly predicting gaze directions and camera rotations, which supports multi-view fusion of gaze directions. Metric-space gaze rays are a good direction for future work.

Q1. Relative translations. We neither supervise nor use relative translations at inference, for two reasons. (1) We follow the common 3D gaze estimation task, which estimates a direction vector, not an origin, so camera rotation is enough. (2) Estimating translation from face crops is sensitive to intrinsics, so it may need joint prediction or intrinsics as input. ETH-XGaze provides translations and intrinsics, but its diversity may not be enough. A preliminary variant with translation supervision (1 run, ETH-XGaze, N=9) recovers translation direction (error 8.9° → 1.1°) but not scale (ratio to ground truth ≈0.008), which is consistent with this view. We see this as a good future direction.

Q2. Weakly-supervised learning. Yes. Given multi-view consistency, future work can explore weakly-supervised learning.

Q3. Intrinsics. Intrinsics and relative translations are not our focus and we do not handle them explicitly. Intrinsics are not inputs: the model sees face crops, so what matters is the intrinsics of the crop. These differ strongly between ETH-XGaze and GazeGene:

ETH-XGazeGazeGene
Crop focal length (px)≈960 (normalised virtual camera)11.8k–14.8k (varies by frame)
Field of view of the crop≈26°≈2°

Since the cross-domain experiments still hold, differing intrinsics do not change the conclusion that multi-view fusion beats single-view (Ref-Avg, °, random camera subsets):

Train → TestN=1N=2N=9
ETH-XGaze → GazeGene11.3810.259.53
GazeGene → ETH-XGaze9.197.987.01

As stated above, we define the task as 3D gaze direction estimation without origin. 6-D gaze ray estimation, which needs intrinsics and relative translations, is an important future direction.

Minor errors. Thank you for pointing out the citation format and unresolved references. We will fix them and proofread the full text in the revised version.

Dear authors,

Thank you for taking the time to follow-up with my comments.

I accept the explanation pertaining to the effort put into making Pi3X work in the multi-view gaze estimation setting, and that practical scenarios corresponding to the setting explained in this paper, may exist (though I'd imagine, rarely).

I do not agree that metric-space gaze rays nor relative transformations are out-of-scope.

  • Example from an adjacent topic: scale-free SfM has its utility in that the goal can be completely different to metric-scale SfM. If scale is necessary, one can determine the correct scale in post-processing, provided that a known size object exists and can be reconstructed.
  • Typical remote gaze estimation settings do not consider the origin, because they assume that the so-called "data normalization" procedure will determine the 3-DoF gaze origin, when necessary.
  • Multi-view gaze estimation, if defined as predicting 3-DoF directions only (as in this paper), has no utility. There is no motivation I can think of, for which this specific formulation applies, or solves an existing problem. There is no meaningful way of defining "scale/translation-free multi-view gaze estimation".

Nevertheless, I see the value of this work within its defined problem space. It is evaluated and presented well, and I am happy to retain my rating.

We are at the end of the official discussion period, so this is not a request, but an optional question to the authors: what is w/o GGSP? GGSP consists of 4 different loss functions. If all 4 are disabled (which w/o GGSP implies), then how would the model predict gaze at all?

Dear Reviewer H1xx,

Thank you for your follow-up, and we really appreciate your retaining your rating. We address the two points you raised below.

Metric-space gaze rays. (1) Our work extends typical remote gaze estimation settings. We keep their preprocessing and the extra components used in practice, so GazeGGT can be applied in the same scenarios. Adding cameras improves head pose coverage and gaze accuracy there. (2) We agree with your analysis and find it very instructive. Metric-space multi-view gaze estimation is an important goal for this line of work, since it could remove many of these extra components and the errors they accumulate. (3) We will share more of the evidence we have in the paper. As discussed in our earlier reply, we found that intrinsics are important for metric-space capability. In addition, GazeGGT's translation and depth predictions are qualitatively preserved after training: in point-cloud visualisations they look similar to the original Pi3X. So spatial structure up to an unknown scale is still reconstructed by the model, and we think metric-space methods can build on our approach.

What is "w/o GGSP"? It is training with the vanilla per-view gaze loss only. We will state this explicitly in the paper. For the effect of GGSP, please see our reply to Reviewer VY9f (Q3).

Reviewer_FRviOpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

GazeGGT introduces a multi-view gaze estimation framework that operates without camera calibration at test time. By combining information from synchronized face images and jointly estimating relative camera orientations, the method effectively exploits complementary viewpoints. On the ETH-XGaze benchmark, the ETH-trained model improves performance from 3.90° using a single view to 2.51° using nine views, while relying solely on predicted camera poses.

Critical strengths
  • The paper addresses a practically relevant problem. A single model can accommodate varying numbers of camera views while eliminating the need for camera calibration parameters during inference, enhancing flexibility and deployability in real-world settings.
  • The benefits of multi-view integration are clearly demonstrated. Performance consistently improves as additional views are incorporated, and these gains remain evident in cross-dataset transfer experiments between ETH-XGaze and GazeGene, suggesting that the approach effectively leverages complementary viewpoint information.
  • The geometric supervision component is shown to be important through ablation studies. Removing this supervision substantially degrades performance, increasing the nine-view reference-frame error on ETH-XGaze from 2.51° to 4.36°. This result provides strong evidence that geometric constraints contribute meaningfully to the model's effectiveness.
  • The experimental analysis extends beyond accuracy metrics. The paper reports model size, inference latency, and memory consumption, offering a more comprehensive assessment of practical deployment considerations. It also explicitly acknowledges a key limitation of the approach, namely the requirement for synchronized images of the same subject across multiple cameras.
Critical weaknesses
  • The comparison with prior multi-view approaches is based on a very limited evaluation set. Results are reported for only two test subjects, making it difficult to determine whether the observed performance trends generalize more broadly. Moreover, UVAGaze Dual-S achieves superior performance on both evaluated subject pairings, although this comes at the cost of requiring separately adapted models.

  • The claimed flexibility with respect to camera configurations would benefit from more extensive validation. The view-count experiments are conducted using fixed groups of consecutive cameras, leaving open the question of how the method performs under more diverse camera selections. Evaluations with randomly sampled camera subsets, as well as reporting variability across repeated runs, would provide a more comprehensive assessment of robustness.

  • The manuscript appears to be incomplete in its current form. The paper references appendix material containing implementation details and robustness analyses, but no appendix is included in the provided PDF. In addition, Page 7 contains an unresolved “Table ??” suggesting that the manuscript would benefit from thorough proofreading.

Questions for authors

Please address the weakness points.

2 comments

Dear Reviewer FRvi,

Thank you for your careful review and helpful suggestions. And we address each concern as follows.

W1a. Two test subjects. This comes from a dataset overlap. Rot-MVGaze and UVAGaze were trained with their official configurations on their own split, and only two subjects (subject0104 and subject0106) are held out by both their split and ours. These are the only subjects on which all methods can be compared fairly, so we restricted the table to them. We will state this limitation next to the table in the revised paper.

W1b. UVAGaze Dual-S is stronger. Yes, at N=2 we are slightly behind it (2.94° vs 3.03° consecutive, 2.51° vs 2.79° large-baseline). In return, GazeGGT needs no retraining or per-pair adaptation, needs no camera calibration, and accepts any number of input views, which Dual-S cannot.

W2a. Camera selection. The original view-count tables use fixed consecutive groups. We now evaluate random camera subsets with random view order: for each sample and each N we draw camera subsets uniformly without replacement, and repeat with 5 draw seeds.

NETH-XGaze → ETH-XGazeETH-XGaze → GazeGeneGazeGene → GazeGeneGazeGene → ETH-XGaze
13.9011.383.869.19
23.081 ± 0.01110.254 ± 0.0322.901 ± 0.0097.975 ± 0.019
42.687 ± 0.0079.707 ± 0.0082.461 ± 0.0137.311 ± 0.014
62.578 ± 0.0069.586 ± 0.0212.346 ± 0.0057.132 ± 0.016
92.504 ± 0.0059.525 ± 0.0092.265 ± 0.0087.006 ± 0.023

(Reference-frame error with predicted poses, random camera subsets and view order, mean ± std over 5 draw seeds. Columns are train → test; the first two use the model trained on ETH-XGaze, the last two the model trained on GazeGene. N=1 is the single-view result.)

The error decreases monotonically with N for every subset draw, and the standard deviation across draws is at most 0.03°. The N=9 value matches the consecutive result (2.504° vs 2.511°), so the gain is not an artefact of adjacent cameras. Cross-dataset shows the same monotone trend.

W2b. Training variance. We retrained the ETH-XGaze model with two additional seeds (3 runs in total, original included):

NPaper (original run, °)Mean ± std over 3 training runs, re-trained (°)Std over 5 camera-subset draws, fixed model (°)
13.903.90 ± 0.04—
23.493.48 ± 0.020.011
42.882.83 ± 0.030.007
92.512.47 ± 0.030.005

(Ref-Avg error with predicted poses, ETH-XGaze, 16 held-out subjects. The paper column is the original run and is one of the 3 training runs; the last column is the std of random camera subsets/view orders for the original model.) The training-run spread (≤0.04°) is small next to the gain from adding views (1.4° from N=1 to N=9).

W3a. Supplementary material. The supplementary material was submitted as a separate PDF in the supplementary materials section of OpenReview, rather than being included in the main paper.

W3b. Minor errors. Thank you for pointing out the unresolved references. We will correct them and carefully proofread the entire manuscript in the revised version.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer FRvi,

Thank you again for your careful review. As the discussion period ends soon, we would like to briefly follow up on our response:

  1. Fair comparison: the two subjects are the only ones held out by all methods' splits; we will state this explicitly in the paper.
  2. Robustness: with random camera subsets and view orders (5 seeds), error drops monotonically from 3.90° (N=1) to 2.50° (N=9), with the same trend cross-dataset. Variance across subset draws (≤0.03°) and across 3 training runs (≤0.04°) is negligible next to this 1.4° gain.
  3. Completeness: the appendix is in the separately submitted supplementary PDF, and all unresolved references will be fixed.

Please let us know if you have any further questions. We would be happy to address them before the deadline. If our responses have resolved your concerns, we would be grateful if you would consider reflecting this in your final assessment.

Best regards The Authors

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your time and consideration of our work. We have provided responses to the questions and concerns raised in the reviews, and would be happy to clarify anything further if needed.

As the discussion period is approaching its end, we just wanted to kindly check whether there are any remaining questions or concerns that we could address.

Thank you again for your valuable feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.