acceptodds
Under review as a conference paper at ICLR 2027

Learning to Gaze Beyond the Face: Unsupervised Transfer from Cropped-Face Experts to Full-Frame Driving Scenes

Abstract

Large-scale gaze estimation datasets are predominantly constructed from cropped face images, enabling accurate gaze prediction but limiting direct deployment in real-world driving scenarios, where cameras capture full-frame images containing complex backgrounds and contextual information. This discrepancy creates a fundamental challenge: how can gaze knowledge learned from cropped faces be transferred to full-frame driving scenes when target-domain gaze annotations are unavailable? In this work, we present an unsupervised framework for cropped-face-to-full-frame gaze transfer. Rather than forcing full-frame representations to directly mimic cropped-face features, which can suppress useful contextual information and cause degenerate predictions, we retain a pretrained cropped-face gaze expert and use it to provide privileged gaze knowledge during training. A full-frame model is subsequently optimized with self-supervised constraints that encourage gaze-relevant representation learning while preserving the intrinsic variation of gaze predictions. In particular, an anti-collapse constraint is introduced to maintain the batch-level gaze distribution and prevent the model from converging to nearly constant predictions. This design allows the full-frame model to learn gaze-sensitive representations without requiring target-domain gaze labels or explicit feature-level alignment between cropped faces and full frames. Experiments on in-vehicle gaze estimation demonstrate that the proposed approach substantially improves over direct application of the cropped-face expert and achieves stable gaze estimation from raw full-frame inputs.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Gaze estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_u3EmOpenReview ↗
Rating
1
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper retains a pretrained cropped-face gaze expert as a source of privileged gaze knowledge and trains a full-frame model without explicit feature alignment.

Critical strengths

The paper addresses a useful deployment setting: removing face localization and cropping at inference without requiring target-domain gaze annotations. Transferring knowledge in gaze-output space avoids potentially inappropriate intermediate feature alignment between cropped-face and full-frame inputs.

Critical weaknesses

lack of novety. Experiments are not persuasive.

Questions for authors
  1. experiments setup are not clear to me, such as 'Source-only Expert', 'Teacher Pseudo Supervision'
  2. The methodological novelty needs sharper positioning.
5 comments
AuthorsOpenReview ↗

Response to Reviewer u3Em

We thank the reviewer for the constructive feedback. We further clarify the experimental settings, refine the positioning of our methodological contributions, and extend the evaluation to an independent target domain.

1. Clarification of experimental settings

We clarify the two baselines that were insufficiently described in the original manuscript.

Source-only Expert: This baseline directly evaluates the pretrained gaze expert on target-domain full-frame images without target-domain adaptation. It measures direct source-to-target transfer under the discrepancy between the teacher's cropped-face training input and full-frame evaluation input. Its LBW error is 23.43°.

Teacher Pseudo Supervision: The frozen teacher generates gaze pseudo-labels from cropped-face images of unlabeled target-domain samples. A student is then trained using corresponding full-frame images and pseudo-label regression only, without the additional relational, sensitivity, consistency, and anti-collapse objectives. Its LBW error is 17.66°.

The teacher itself obtains 16.92° when evaluated on cropped-face images. This differs from the 23.43° full-frame Source-only Expert result because the input conditions are different.

2. Methodological contribution

Our contribution is a task-specific framework for transferring gaze knowledge from a cropped-face expert to a full-frame student under asymmetric input conditions, without target-domain gaze annotations or face cropping at inference.

The framework transfers complementary knowledge through three levels: confidence-aware sample-level prediction guidance, relational transfer that preserves pairwise gaze geometry, and sensitivity transfer that captures teacher-response changes under appearance degradation. Full-frame consistency and anti-collapse regularization further constrain student learning.

The LBW ablation results demonstrate the contribution of this coordinated design:

ConfigurationError (°)
Pseudo + Relation13.74
Pseudo + Sensitivity16.03
Pseudo + Relation + Sensitivity12.61
Pseudo + Relation + Consistency + Anti-collapse11.83
Full Framework11.25

These results demonstrate the contributions of the individual combinations and their joint optimization. The complete framework achieves 11.25°, transferring structured gaze knowledge across different input representations without explicitly aligning intermediate teacher and student features.

3. Additional cross-dataset evaluation

We introduce EyeDiap as an independent target domain with different camera positions and a smaller facial area proportion in full-frame images. HD RGB full frames are used as student inputs, while corresponding cropped faces provide privileged observations to the frozen teacher during training.

We adopt four-fold subject-independent cross-validation, using three folds for unlabeled knowledge transfer and one fold for evaluation. The same LBW-established hyperparameter configuration is applied to all EyeDiap folds without dataset-specific adjustment.

MethodLBWEyeDiap
Source-only Expert23.4325.60
Teacher Pseudo Supervision17.6619.25
PAT15.8417.10
G²D16.2116.55
MonoTAKD15.4717.22
SKD14.7616.81
Ours11.2513.48

Our method achieves 13.48° on EyeDiap, outperforming the strongest transfer baseline, G²D (16.55°), by 3.07°, corresponding to an 18.55% relative error reduction. The method maintains the lowest error across both datasets despite changes in camera positions and facial area proportions. The results also demonstrate the transferability of the LBW-established configuration while retaining full-frame-only inference.

Thank you for your response. According to that, I have two more questions.

  1. How to ensure the comparison using PAT、G²D、MonoTAKD、SKD, are fair?
  2. When you distillize the student using teacher network, one import thing need to be clarify, which is the coordinates registration. The input of the teacher network is the croped face while the input of the student network is the full frame. Firstly, the coordinates registration will introduce errors. Secondly, the EyeDiap and LBW have different settings, the registrations are probably different.
AuthorsOpenReview ↗

Response to Reviewer u3Em

Thank you for raising these important questions.

1. Fairness of baseline comparisons.

PAT, G²D, MonoTAKD, and SKD are knowledge transfer strategies rather than specific backbone architectures, with different requirements for teacher-student compatibility. To ensure a controlled comparison, we adopt ResNet18 as the common visual backbone for all student models and use the same frozen pretrained gaze expert as the teacher. All methods are trained using identical datasets and data splits, on the same server, with the same number of epochs. We retain the default configurations of each method, with necessary adaptations to the crop-to-frame gaze estimation setting. This allows us to evaluate their respective knowledge transfer strategies under consistent backbone and training conditions.

2. Coordinate correspondence between teacher and student.

We clarify that our framework does not require explicit spatial registration between the cropped face and the full-frame image. The cropped face is extracted from the original frame using facial landmarks, while the student directly processes the corresponding full frame. Both observations are associated with the same gaze annotation, and the teacher and student predict gaze in the same yaw-pitch representation. Therefore, knowledge transfer is performed in the gaze space rather than through pixel-level or feature-level spatial alignment.

Furthermore, the datasets considered in this work provide gaze annotations based on their respective image coordinate systems. We directly use the provided annotations without introducing additional coordinate transformations between datasets. Consequently, no cross-dataset coordinate registration is required for the proposed knowledge transfer framework.

Reviewer_Q6jVOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies an important practical mismatch in gaze estimation: strong gaze estimators are commonly trained on cropped facial images, whereas an in-vehicle system naturally observes full frames. The authors formulate this as unsupervised knowledge transfer from a pretrained cropped-face gaze expert to a full-frame gaze model. Facial crops serve as privileged information during training but are removed at inference. The framework contains following objectives:

  1. confidence-aware pseudo-label transfer based on the stability of teacher predictions under appearance perturbations;
  2. relational knowledge transfer preserving pairwise gaze geometry;
  3. gaze-sensitive response transfer based on changes under image degradation;
  4. consistency between perturbed full-frame views; and
  5. covariance matching as an anti-collapse regularizer

Experiments are performed in LBW dataset and found to be outperforming than various KD methods

Critical strengths
  1. The problem statement is genuine and practically relevant.
  2. Strong emprical improvement over baselines
  3. Well written
  4. intense ablation studies.
Critical weaknesses
  1. As the authors pointed out, most of the evidence ultimately centers on LBW as the target driving dataset. In LBW dataset the camera has been placed directly facing the driver, however, in real world scenarios, the placement of camera varies, moreover the field of view will be much larger than LBW dataset, where significant portion of image (`25%) is the face. Hence, the learning process is easier. Lack of breadth of evaluation is the significant weakness of the paper.

  2. Component wise there is very little novelty proposed in the paper. As pseudo-label/distillation losses, relational KD, perturbation consistency, sensitivity matching, and prediction-statistic regularization are already established components in the literature.

Questions for authors
  1. How were hyperparameters and the final configuration selected without target-domain gaze annotations.
  2. Could the authors evaluate the approach with a substantially different target-camera/domain configuration? This would strengthen the stated generality beyond LBW.
  3. There is some inconsistency in Table1 and Table 2 LBW-N results. Why this difference any different protocol used?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Q6jV

We thank the reviewer for recognizing the practical relevance, empirical improvements, and extensive ablation studies of our work. We further clarify the cross-domain evaluation, methodological contributions, and hyperparameter selection protocol.

1. Generalization beyond LBW

We extend the evaluation to EyeDiap, an independent target domain with different camera positions and image acquisition conditions. Compared with LBW, EyeDiap has a smaller facial area proportion in full-frame images. We use HD RGB full frames as student inputs and corresponding cropped faces as privileged observations for the frozen teacher during training.

We adopt four-fold subject-independent cross-validation, using three folds for unlabeled knowledge transfer and the remaining fold for evaluation. Subjects are mutually exclusive across folds, and each subject is evaluated once. Target gaze annotations are excluded from training and model selection.

MethodLBWEyeDiap
Source-only Expert23.4325.60
Teacher Pseudo Supervision17.6619.25
PAT15.8417.10
G²D16.2116.55
MonoTAKD15.4717.22
SKD14.7616.81
Ours11.2513.48

All values represent mean angular errors in degrees.

Our method achieves 13.48° on EyeDiap, outperforming the strongest transfer baseline, G²D (16.55°), by 3.07°, corresponding to an 18.55% relative error reduction. The baseline ranking changes across datasets, whereas our method maintains the lowest error. This supports the effectiveness of the proposed framework under different camera configurations and smaller facial area proportions.

2. Methodological contribution

Our contribution is a task-specific knowledge transfer framework for an asymmetric observation setting, where a cropped-face gaze expert supervises a full-frame gaze predictor. Rather than introducing an isolated replacement for an individual distillation loss, we coordinate complementary forms of gaze knowledge transfer.

Confidence-aware prediction transfer provides sample-level supervision according to teacher prediction stability. Relational transfer preserves pairwise gaze geometry, while gaze-sensitive response transfer captures changes in teacher predictions under appearance degradation. Full-frame consistency and anti-collapse regularization further constrain student learning.

The LBW ablation results illustrate the contributions of this design:

ConfigurationError (°)
Pseudo + Relation13.74
Pseudo + Sensitivity16.03
Pseudo + Relation + Sensitivity12.61
Pseudo + Relation + Consistency + Anti-collapse11.83
Full Framework11.25

These configurations demonstrate the contributions of different transfer objectives. Relational and sensitivity transfer provide complementary gaze information, while consistency and anti-collapse regularization further constrain full-frame student learning. The complete framework achieves 11.25°. The central design is to transfer structured gaze knowledge across different input representations without explicit intermediate feature alignment.

3. Hyperparameter selection and cross-domain configuration

The complete configuration established on LBW is directly transferred to EyeDiap without dataset-specific adjustment. The loss coefficients are λ_pseudo = 0.5, λ_rel = 0.5, λ_sens = 0.5, λ_con = 0.1, and λ_ac = 1. We use batch size 256, initial learning rate 10⁻⁵, 50 training epochs, and a learning-rate decay factor of 0.1 every 25 epochs. The teacher remains frozen.

For model selection, we use a predefined training schedule and the final-epoch checkpoint. The same configuration and selection strategy are applied to all four EyeDiap folds. No EyeDiap-specific hyperparameter adjustment or target-label-based checkpoint selection is performed.

The resulting EyeDiap performance demonstrates that the LBW-established configuration transfers effectively to an independent domain without dataset-specific tuning.

4. Clarification of LBW-N results

The correct nighttime angular error of our method on LBW-N is 13.73°, rather than 14.13°. This value is consistent with Tables 2 and 7. The corresponding improvement over SKD is 5.19°.

Dear Reviewer Q6jV,

We hope you are doing well. We would like to kindly follow up on our previous response to your questions. We would greatly appreciate it if you could review our clarifications and let us know if you have any further concerns or questions.

Thank you again for your time and consideration.

Best regards, The Authors

Dear Reviewer,

We apologize for following up again. As the discussion deadline is approaching, we would like to kindly check whether you have had a chance to review our responses. We sincerely hope to address any remaining concerns and clarify your questions as thoroughly as possible within the limited time available.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_p2vQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a knowledge distillation framework that transfers supervision from a pretrained face-based gaze estimation model to predict driver gaze directly from full-face images. The approach requires neither gaze labels from the target driving dataset during training nor face cropping at test time. On the LBW benchmark, it achieves a gaze estimation error of 11.25°, compared with 14.76° for the strongest transfer-learning baseline.

Critical strengths
  • The paper addresses a practical and relevant problem. Leveraging existing gaze estimation models without requiring the collection of additional gaze annotations for driving datasets is highly valuable, as acquiring accurate gaze labels is costly, time-consuming, and difficult to scale. The proposed knowledge transfer framework therefore has clear practical significance for real-world deployment.

  • The empirical gains over competing methods are substantial. The proposed approach consistently outperforms the compared transfer-learning baselines by a notable margin, achieving lower gaze estimation errors across multiple evaluation settings. Importantly, these improvements are maintained under both daytime and nighttime driving conditions, demonstrating robustness to challenging illumination variations commonly encountered in real-world driving scenarios.

  • The experimental evaluation is reasonably comprehensive. The authors validate the method using three different student backbones and teacher models trained on two distinct source datasets. This diversity of architectures and source domains strengthens confidence that the reported improvements are not tied to a specific network design or teacher configuration, but instead reflect the general effectiveness of the proposed distillation strategy. The ablation studies provide useful insights into the contributions of individual components. By systematically evaluating different loss terms, the paper clarifies the role of each training objective in the overall performance. In particular, the relational knowledge transfer component yields a clear and consistent improvement, providing convincing evidence that preserving inter-sample relationships is an important factor in the success of the proposed framework.

Critical weaknesses
  • The evaluation is limited to a single target dataset. All experiments are conducted using LBW as the target driving dataset, making it difficult to assess the method's generalizability. It remains unclear whether the reported gains would transfer to datasets collected with different camera setups, viewpoints, driver populations, or driving environments.

  • Several implementation details required for reproducibility are insufficiently specified. For example, the weight assigned to the anti-collapse loss is not reported, and the descriptions of the subject-level train/test split and model selection procedure lack sufficient detail. Providing these details would improve the reproducibility and transparency of the work.

  • Some reported results appear inconsistent and need clarification. The teacher model achieves an error of 23.43° in Tables 1 and 2, but 16.92° in Table 4, without a clear explanation for the discrepancy. Similarly, the proposed method reports a nighttime error of 14.13° in Table 1 and 13.73° in Table 2. The paper should explicitly explain the experimental settings underlying these differences to avoid confusion.

  • The reliability of the proposed confidence estimation strategy is not directly validated. The confidence score is based on the assumption that teacher predictions that remain stable under input perturbations are more trustworthy. However, this assumption is not empirically examined. In principle, a teacher model could produce consistently incorrect predictions across multiple perturbed versions of the same image, leading to a high confidence score despite poor prediction quality. Additional analysis correlating confidence scores with actual prediction accuracy would strengthen the justification for this design choice.

Questions for authors

Please address the weakness points highlighted.

3 comments
AuthorsOpenReview ↗

Response to Reviewer p2vQ

We thank the reviewer for the constructive comments and the positive assessment of the practical relevance, empirical improvements, and experimental evaluation of our framework. We have extended the cross-dataset evaluation, clarified the implementation details, and further examined the reliability of the proposed confidence-aware knowledge transfer.

1. Cross-dataset generalization

To examine generalization beyond LBW, we extend the evaluation to EyeDiap, an independent target domain with different camera positions and image acquisition conditions. Compared with LBW, EyeDiap has a smaller facial area proportion in full-frame images, requiring the student to extract gaze-related information from a larger surrounding context.

We use high-definition RGB (HD RGB) full-frame images as student inputs and corresponding cropped faces as privileged observations for the frozen teacher during training. At inference, the student directly predicts gaze from full-frame images.

We adopt four-fold subject-independent cross-validation. In each fold, three subsets are used for unlabeled knowledge transfer and the remaining subset is held out for evaluation. Subjects are mutually exclusive between training and evaluation, and each subject is evaluated once. Target-domain gaze annotations are used exclusively for final evaluation. All compared transfer methods follow identical folds.

MethodLBWLBW-LLBW-NEyeDiap
Supervised9.1211.0612.2710.16
Source-only Expert23.4326.1529.1825.60
Teacher Pseudo Supervision17.6620.4122.9319.25
PAT15.8418.2720.3817.10
G²D16.2118.8621.1716.55
MonoTAKD15.4717.9120.1117.22
SKD14.7616.8318.9216.81
Ours11.2513.5813.7313.48

All values represent mean angular errors in degrees.

On EyeDiap, our method achieves 13.48°, outperforming the strongest transfer baseline, G²D (16.55°), by 3.07°, corresponding to an 18.55% relative error reduction. The ranking of competing methods changes across datasets, while our method maintains the lowest error. This demonstrates the effectiveness of transferring privileged gaze knowledge under different observation conditions and smaller facial area proportions.

2. Implementation details and model selection

The loss coefficients are λ_pseudo = 0.5, λ_rel = 0.5, λ_sens = 0.5, λ_con = 0.1, and λ_ac = 1. The student is trained for 50 epochs with batch size 256 and initial learning rate 10⁻⁵, decayed by a factor of 0.1 every 25 epochs. The teacher remains frozen.

For LBW, training and test sets are partitioned at the subject level, ensuring no subject overlap. We adopt a fixed training schedule and use the final-epoch checkpoint.

Importantly, the complete configuration established on LBW is directly transferred to all EyeDiap folds without dataset-specific modification. No EyeDiap gaze annotations are used for hyperparameter adjustment or checkpoint selection.

3. Clarification of numerical results

We correct the LBW-N angular error of our method in Table 1 from 14.13° to 13.73°, consistent with Tables 2 and 7. Accordingly, the improvement over SKD on LBW-N is 5.19°, rather than 4.79°.

We also clarify the different input conditions underlying the teacher results:

Evaluation settingInputError
Teacher (Table 4)Cropped face16.92°
Source-only ExpertFull frame23.43°
Teacher Pseudo SupervisionFull-frame student17.66°
OursFull-frame student11.25°

The 16.92° result corresponds to the frozen ETH-XGaze teacher evaluated on cropped-face images, whereas 23.43° corresponds to direct evaluation on full-frame images. These results represent different input conditions.

4. Validation of confidence estimation

We examine the relationship between prediction stability and teacher reliability on the labeled LBW evaluation set. The Spearman correlation between confidence scores and teacher angular errors is ρ = -0.46. Samples are divided into four equal-sized confidence groups.

AnalysisConfigurationError (°)
Teacher confidenceQ1 (Lowest)23.40
Teacher confidenceQ218.20
Teacher confidenceQ314.70
Teacher confidenceQ4 (Highest)11.38
Student weightingUniform11.73
Student weightingConfidence-aware11.25

Teacher errors decrease consistently with increasing confidence. Confidence-aware weighting also reduces student error by 0.48° (4.09%) compared with uniform weighting. These results support prediction stability as relative reliability information for weighting transferred supervision, rather than as a calibrated probability of correctness. Ground-truth gaze labels are used exclusively for post-training evaluation.

Dear Reviewer p2vQ,

We hope you are doing well. We would like to kindly follow up on our previous response to your questions. We would greatly appreciate it if you could review our clarifications and let us know if you have any further concerns or questions.

Thank you again for your time and consideration.

Best regards, The Authors

Dear Reviewer,

We apologize for following up again. As the discussion deadline is approaching, we would like to kindly check whether you have had a chance to review our responses. We sincerely hope to address any remaining concerns and clarify your questions as thoroughly as possible within the limited time available.

Thank you again for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.