acceptodds
Withdrawn from ICLR 2027

SPECTRA-Gaze: Gaze Target Estimation via Head-Conditioned Spectral Modulation and Reliability-Aware Heatmap Refinement

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Gaze target estimation combines head cues and scene information to determine where a person is looking. When a scene contains several plausible targets, the same head pose may not distinguish between them, and distractors may receive strong heatmap responses. Existing methods mainly improve head–scene interaction in the spatial domain, but do not explicitly address ambiguity both before and after heatmap decoding. We view gaze ambiguity as two linked problems: weak target evidence before decoding and unreliable candidate peaks afterward. We propose SPECTRA-Gaze to address both through Head-Conditioned Spectral State Interaction (HSSI) and Reliability-Constrained Nonlocal State Interaction (RCNSI). When nearby plausible targets are difficult to distinguish from head pose alone, scene cues around them become important. The DCT separately represents broad and fine spatial variation in scene features. HSSI forms adaptive frequency bands from scene and head descriptors, allowing head cues to modulate scene details that may help distinguish nearby plausible targets. Interactions among band descriptors then guide spectral processing before decoding. RCNSI uses candidate reliability to weight candidate features alongside pooled scene context and refine the coarse heatmap. A spatial gate combines local heatmap dispersion with reliable candidate support to control the location and magnitude of logit corrections. Both modules use a partially shared dual-stream encoder to reduce parameter redundancy. Extensive experiments on the GazeFollow and VideoAttentionTarget datasets demonstrate that our method consistently outperforms state-of-the-art methods while using 18.05% fewer parameters.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Gaze target estimation, Spectral modulation, Nonlocal reasoning, Candidate reliability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_gTX8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SPECTRA-Gaze, a gaze target estimation (GTE) method. SPECTRA-Gaze consists of an HSSI module which alters scene features based on “head information”. An RCNSI module considers “reliability” to refine the final heatmap prediction. The proposed method is evaluated on GazeFollow and VideoAttentionTarget.

Critical strengths

None

Critical weaknesses

The quantitative results are weak. Not only do the authors not evaluate on the commonly used ChildPlay dataset, they conveniently omit PaGE ViT-H+ (which is reported in the original PaGE paper and those reported numbers would out-perform SPECTRA-Gaze. No explanation is provided for either choice. Furthermore, neither the perf. Improvements shown in Table 1 in comparison to PaGE ViT-B nor the ablation in Table 2 show significant improvements in the reported metrics.

The proposed HSSI and RCNSI modules are highly complex, insufficiently motivated, and not very intuitive. For instance, there is no clear reason for HSSI to operate in the frequency domain (using DCT), nor is the benefit of using the frequency domain shown via ablation study (there’s only a HSSI on/off). RCNSI is similarly contrived, non-intuitive, and inconsequential. In fact, we can see that barely any of the metrics that improve in Table 2 (ablation) do so with sufficient increase, while the VAT AP metric actually worsens compared to the base model. I may even guess that such differences can come around simply by training with different random seeds.

Questions for authors

Your network architecture design seems to be heavily influenced by PaGE. What are the most important novel contributions made by SPECTRA-Gaze?

What are the ChildPlay numbers like for Tables 1 and 2? What about GOO-Real? Please note that both Gaze-LLE and PaGE papers provide results for ChildPlay and GOO-Real.

Reviewer_EztYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SPECTRA-Gaze, which estimates gaze targets from a scene image and the bounding box of a specified person’s head. Built on a DINOv3 dual-stream encoder with partially shared parameters, the method uses HSSI for head-conditioned frequency-domain feature modulation and RCNSI for heatmap refinement based on candidate reliability and scene context. The model is jointly trained on GazeFollow and VideoAttentionTarget. The paper reports improvements across all six metrics over a locally reproduced, non-distilled PaGE ViT-B baseline, along with an 18.05% reduction in parameter count.

Critical strengths
  1. The matched comparison with PaGE, results across multiple random seeds, and paired statistical analysis on GazeFollow collectively support the reported improvements in localization performance. The model improves localization while reducing the parameter count, and the paper also reports a decline in AP, transparently presenting the trade-offs across metrics.
  2. The method addresses gaze ambiguity through feature modulation before decoding and heatmap refinement after decoding. The design is well motivated, with clearly defined roles for the two modules. Module ablations show that HSSI and RCNSI each improve localization, with their combination achieving the lowest localization error, supporting their complementary roles.
Critical weaknesses
  1. The independent contributions of the two core mechanisms have not been sufficiently established. HSSI lacks comparisons with a spatial adapter and a frequency-domain variant without head conditioning under matched training conditions, and the fixed-weight interventions are insufficient to establish the specific advantages of its design. Table 6 supports the overall effectiveness of RCNSI, but the ablations do not disentangle the contributions of the reliability mechanism, additional model capacity, and auxiliary losses. Moreover, reliability-based ranking provides only modest gains at K=8, while the analysis at K=100 is a standalone probe that does not affect the final heatmap, offering limited evidence for the mechanism’s contribution in practice.
  2. The matched comparison with non-distilled PaGE is valuable, but it is insufficient to support the state-of-the-art claim. The cited distilled PaGE model reports lower localization error, and the paper should restrict its performance claims to the settings actually evaluated.
Questions for authors
  1. Does HSSI retain its advantage over a spatial adapter and a frequency-domain variant without head conditioning when all variants are trained under matched conditions?
  2. With the refinement backbone and auxiliary supervision held constant, does the learned reliability prior improve final heatmap performance over a constant prior or a prior based on coarse heatmap scores?
  3. How does performance change when the full RCNSI module is retained but the auxiliary reliability loss is removed?
  4. Please clarify the settings under which the reported performance advantages hold, and explain how the PaGE reproduction corresponds to the publicly released model in terms of the number of interaction layers and positional encoding.
Reviewer_y3AVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the gaze target estimation task based on pretrained general visual model. In this work, they propose two modules, HSSI and RCNSI, which mitigate gaze ambiguity before and after heatmap decoding.

Critical strengths

The paper is well structured and clearly written. The appendix provides detailed implementation details and results across multiple random seeds, which supports good reproducibility.

Critical weaknesses

The novelty of this work is limited. This paper expands upon the recent work PaGE, using the same main framework and training strategy. The improvements brought by the two core modules are relatively small (relative to Base, GF Avg L2 only decreased by 0.0039, while VAT AP decreased by 0.0050), and their effectiveness and mechanism require further investigation. In addition, the comparison with PaGE is incomplete.

Questions for authors

My main concerns are as follows:

  1. Limited innovation compared to PaGE. The main framework and training strategy of this paper are largely inherited from PaGE (Ye et al., 2026), with the actual new contributions concentrated in the HSSI and RCNSI modules. However, compared to Base (the compact PaGE variant), the two modules only reduce GF Avg L2 by 0.0039 and VAT L2 by 0.0042, while VAT AP decreases by 0.0050 (Table 2).
  2. The core modules lack specific validation. This paper attributes gaze ambiguity to two problems: insufficient target cues in the scene features before decoding, and the possibility that the maximum response of the coarse heatmap after decoding may fall on obstructions. Two modules, HSSI and RCNSI, are proposed to address these issues. However, existing experiments mainly report overall performance metrics, lacking direct validation of the two problems and the effectiveness of the corresponding modules. For example, the authors could report the proportion of coarse heatmaps whose peak responses fall on distractors, as well as the proportions of incorrect predictions corrected and correct predictions disrupted by RCNSI refinement.
  3. Existing analytical experiments are insufficient to support the mechanisms of modules. In Table 5, head-descriptor mismatch increases GF Avg L2 by only 0.0003. This makes it difficult to conclude that HSSI effectively uses the head information of the queried person. In Table 6, RCNSI refinement reduces GF Avg L2 by only 0.00035, so the correction it provides is very limited.
  4. The comparison with PaGE is incomplete. The results in Table 1 labeled "PaGE ViT-B (Ye et al., 2026)" differ from the ViT-B results reported in the original PaGE paper, and the differences are not explained. Furthermore, the distilled version of ViT-B, ViT-L, and ViT-H+ in the original PaGE paper outperformed this paper in all six metrics. It is recommended that the authors supplement these results in Table 1 and analyze the reasons for the gap.
  5. The significance of the reduced parameter count is unclear. This paper claims an 18.05% reduction in parameters compared to PaGE, but this reduction comes from the simplification of the PaGE architecture (e.g., reducing the interaction layer from 5 to 3 layers), without providing a reason or ablation. PaGE has already demonstrated that 5 layers of interaction are superior to 3 layers after adding the head branch Furthermore, the shared layers are still executed separately for the two branches. This reduces the parameter count but not the computational cost. The paper also does not report inference time. Thus, the practical benefit of reducing the parameter count is limited.
  6. Suggested evaluation on a stronger baseline. PaGE has released an open-source distilled ViT-B model that outperforms the baseline used in this paper. The authors could integrate HSSI and RCNSI into this model to further evaluate whether these modules improve performance on a stronger baseline.
  7. Suggested evaluation on more datasets. PaGE has been evaluated on ChildPlay and GOO-Real in addition to GazeFollow and VAT.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.