acceptodds
Under review as a conference paper at ICLR 2027

-EGO: Proxies as mediators for Poly-modal-Induced Egocentric Video Representation Learning

Abstract

Egocentric RGB video provides an inherently limited view of human actions, often obscuring body motion and surrounding scene context. Privileged information from complementary modalities, exocentric views, and foundation models can provide richer supervision during training, but is often unavailable or impractical at inference. To this end, we introduce -EGO, a poly-modal-induced egocentric RGB encoder learned through PROXYDIST, a multi-teacher distillation framework that uses proxies as mediators to consolidate knowledge from nine heterogeneous teachers spanning viewpoints, modalities, and foundation model representations. PROXYDIST learns representation-specific proxies that translate heterogeneous knowledge into a homogeneous egocentric space rather than directly distilling from incompatible teacher feature spaces. Then, the proxies are merged through a learned convex initialization, followed by Selective Proxy Distillation (SPD), which selectively distills from the most reliable proxies for each training sample. The resultant -EGO achieves state-of-the-art performance across action recognition, video retrieval, and action segmentation on three challenging ego-exo benchmarks, outperforming direct multi-teacher distillation while requiring only egocentric RGB video and no additional computational overhead at inference.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Egocentric Video Understanding, Action Recognition, Video Representation Learning, Ego-Exo Learning, Knowledge Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WMisOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose pi-Ego, a poly-modal-induced Ego encoder learned through a multi-teacher distillation framework (i.e., ProxyDist). It learns from representation-specific proxies that translate heterogeneous knowledge into a unified Ego representation space to mitigate representation gaps. Then, Selective Proxy Distillation (SPD), which selectively distills from the most reliable proxies for each training sample. The proposed method achieves SOTA performance on multiple downstream tasks on three commonly adopted Ego datasets, which further demonstrates its effectiveness.

Critical strengths

1/ The paper is well-motivated and clearly identifies two key challenges of multi-teacher distillation in egocentric video representation learning: representational gaps and conflicting gradients.

2/ The paper is well-written and easy to follow.

3/ The proposed method achieves strong performance across multiple mainstream egocentric benchmarks and different downstream tasks. Extensive experiments further demonstrate the effectiveness of the proposed method.

4/ The proposed method is efficient at inference, as it only requires egocentric RGB videos without relying on additional data, such as exocentric videos or other modalities.

Critical weaknesses

1/ My primary concern is whether the proposed method is specifically designed for Ego scenes and tasks. Although the method achieves promising performance, it remains unclear which technical designs are tailored to the unique challenges of egocentric videos. The proposed framework appears to be a general multi-teacher distillation approach rather than a method specifically designed to address egocentric challenges, such as limited field of view and occlusion.

2/ The authors should further clarify the definition of "poly-modal". For example, egocentric and exocentric RGB videos represent different viewpoints rather than different modalities. A clearer distinction among these concepts would help justify the use of the term "poly-modal".

3/ The proposed framework relies on multiple modalities and viewpoints during training. Although several Ego-Exo datasets have been introduced, not all of them provide the complete set of modalities required by the teacher models. It remains unclear how the proposed method handles missing modalities during training. It would be helpful if the authors could provide further discussion or analysis on this issue.

4/ Have the authors considered the EgoMe dataset? It would be helpful if the authors could conduct experiments on EgoMe to further evaluate the generalizability of the proposed method.

Overall, despite the remaining concerns, the paper is well-motivated, clearly written, and demonstrates strong experimental results. Therefore, my initial rating is 3: Weak Accept. I will consider adjusting my rating according to the authors’ rebuttal and other reviewers’ comments.

Questions for authors

See weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer WMis [Part 1/2]

W1/Q1. Motivation and Design for Egocentric Video Understanding

We thank the reviewer for raising this concern. Our design is motivated by the partial observations inherent to egocentric videos: limited field of view and occlusion make complementary modalities and exocentric views valuable sources of privileged supervision during training. Our proxies transfer this heterogeneous knowledge into egocentric RGB models, while sample-dependent proxy selection accommodates variation in each source’s relevance across observations. The final student requires only egocentric RGB at inference, without additional sensors or exocentric views. Therefore, egocentric video understanding provides a particularly suitable testbed for our method, which addresses observation constraints through privileged knowledge transfer. Its potential applicability beyond this setting does not diminish the intellectual merit in our contribution and offers opportunities for broader use.

W2/Q2. Clarification on the term “poly-modal”

We thank the reviewer for this question. We agree that egocentric and exocentric RGB videos represent different viewpoints, not different modalities. In our paper, “poly-modal” describes a pool of privileged teachers that varies along three distinct axes:

Input modality: RGB and skeleton.

Viewpoint: egocentric and exocentric.

Representation: geometry-oriented features from Depth Anything, visual features from DINOv2, and vision-language features from SigLIP.

As described in Lines 46-48, we treat each teacher, i.e. its input together with its pretrained encoder, as a virtual sensor providing a potentially complementary signal about the same action. We therefore use “poly-modal” as an umbrella term for teacher heterogeneity, without equating viewpoints or learned representations with distinct physical modalities.

This terminology is motivated by the broad definition of modality as “a way in which a natural phenomenon is perceived or expressed” [1] and related work incorporating derived signals: CMC [2] treats depth and optical flow as distinct views, while π-VAD [3] uses “poly-modal” for signals including depth, pose, and optical flow extracted from RGB. These precedents contextualize our usage, while the distinction among modalities, viewpoints, and representations remains important.

[1] P. P. Liang, A. Zadeh, and L.-P. Morency, “Foundations & trends in multimodal machine learning: Principles, challenges, and open questions,” ACM Comput. Surv., vol. 56, no. 10, Art. no. 264, 2024, doi: 10.1145/3656580.

[2] Y. Tian, D. Krishnan, and P. Isola, “Contrastive multiview coding,” in Proc. Eur. Conf. Comput. Vis. (ECCV), Glasgow, U.K., Aug. 2020, pp. 776–794, doi: 10.1007/978-3-030-58621-8_45.

[3] S. Majhi, G. D’Amicantonio, A. Dantcheva, Q. Kong, L. Garattoni, G. Francesca, E. Bondarev, and F. Brémond, “Just dance with π! A poly-modal inductor for weakly-supervised video anomaly detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2025, pp. 24265–24274.

Q3.Addressing Missing Modalities during training

We thank the reviewer for this question. When a training sample lacks a modality or viewpoint, such as an exocentric video (for example, 94 out of 1776 videos in EgoExoFitness), we mask the corresponding teacher-distillation loss in Level-I. The associated proxy still learns from action classification supervision on the egocentric RGB input. Level-II requires only the trained proxies and egocentric RGB inputs, so it does not require the original modalities or viewpoints to be available.

Furthermore, Table 10 (Appendix E) evaluates the case where exocentric videos are entirely unavailable. Using only ego-view teachers, π\large \pi-EGO still improves over the baseline by 2.8% on EEF, 1.5% on A101, and 0.8% on EE4D. These results demonstrate that the framework remains effective without the complete teacher set. We have revised Section 4.1 to make this process explicit [L313 - L314].

AuthorsOpenReview ↗

Response to Reviewer WMis [Part 2/2]

Q4. Experiment on EgoMe dataset

We thank the reviewer for this suggestion. Following the reviewer’s suggestion, we evaluated π\large \pi-EGO on EgoMe [4], which contains 7,902 paired egocentric–exocentric videos spanning 184 action categories. Following GCEAN [5] and Ego-PMOVE [6], we retain only correctly imitated pairs (Video type = True), yielding 3,709 training, 800 validation, and 1,643 test pairs under the official split. We compare π\large \pi-EGOEgo against an egocentric RGB TimeSformer baseline and direct multi-teacher distillation. We also report the mean ± standard deviation of accuracy across the nine proxies.

MethodAccuracy
RGB Baseline80.9
Nine proxies (mean ± std)82.1 ± 0.4
Multi-teacher Distillation82.2
π\large \pi-EGO84.4

π\large \pi-EGO outperforms the RGB baseline by 3.5% and direct multi-teacher distillation by 2.2%. These results provide additional evidence that the benefits of our framework extend across datasets. We have revised the manuscript to include this dataset. [L344-L347; L357-L367]

[4] H. Qiu, Z. Shi, L. Wang, H. Xiong, X. Li, and H. Li, "EgoMe: A new dataset and challenge for following me via egocentric view in real world," 2025, arXiv:2501.19061.

[5] Z. Shi, H. Qiu, L. Wang, Q. Wu, F. Meng, and H. Li, "Unsupervised ego- and exo-centric dense procedural activity captioning via gaze consensus adaptation," in Proc. 33rd ACM Int. Conf. Multimedia (MM), Dublin, Ireland, 2025, pp. 3731–3740, doi: 10.1145/3746027.3755167.

[6] H. Qiu, L. Wang, T. Zhao, Z. Shi, X. Li, L. Xu, and H. Li, "Ego-PMOVE: Prompt-aware mixture of view experts network for egocentric gaze prediction," in Proc. AAAI Conf. Artif. Intell., vol. 40, no. 10, 2026, pp. 8565–8573, doi: 10.1609/aaai.v40i10.37808.

Thanks for your rebuttal. I'd like to keep my positive rating.

AuthorsOpenReview ↗

Thank you for feedback and positive score.

We thank the reviewer again for the valuable feedback and for the positive score.

Reviewer_VuCiOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes π-EGO, a framework for learning egocentric video representations from heterogeneous teachers spanning different viewpoints, modalities, and foundation models. The proposed ProxyDist framework consists of three components: (1) Proxy Learning, which independently distills nine heterogeneous teachers into homogeneous egocentric RGB proxies; (2) Proxy Merging, which learns a convex combination of proxy parameters to initialize the final student; and (3) Selective Proxy Distillation (SPD), which performs sample-wise proxy selection based on classification correctness and confidence.

The framework is evaluated on EgoExo-Fitness, Assembly101, and EgoExo4D across action recognition, video retrieval, and temporal action segmentation. The results show consistent improvements over direct multi-teacher distillation while requiring only egocentric RGB input at inference.

Critical strengths

S1. Practical framework for heterogeneous knowledge distillation.

The paper addresses a meaningful challenge in egocentric video understanding: transferring complementary knowledge from heterogeneous modalities and viewpoints into a single RGB encoder. The proxy-mediated framework provides a coherent approach to separating heterogeneous knowledge transfer from multi-source consolidation, without introducing additional inference-time requirements.

S2. Consistent empirical improvements and useful diagnostic analyses.

The proposed method improves Top-1 accuracy over direct multi-teacher distillation from 81.5% to 84.7% on EgoExo-Fitness, 48.2% to 50.7% on Assembly101, and 40.6% to 41.1% on EgoExo4D. The authors also evaluate video retrieval, temporal segmentation, and multiple backbone architectures. The CKA and gradient-conflict analyses provide additional insights into the potential mechanisms underlying the observed improvements.

Critical weaknesses

W1. Insufficient controls to isolate the contribution of heterogeneous teacher knowledge.

The central concern is whether the observed improvements genuinely result from heterogeneous knowledge transfer or primarily from additional supervised training and model consolidation.

According to Eq. (1), each proxy is trained using both feature distillation and ground-truth cross-entropy supervision. Furthermore, the framework trains nine independent proxies before optimizing the final student, introducing substantially more training computation than the direct multi-teacher baseline.

A critical missing baseline is to train the nine proxies using classification loss alone, without teacher feature distillation, and subsequently apply the identical Proxy Merging and SPD procedures. Without this comparison, it is difficult to determine how much of the improvement can be attributed specifically to the heterogeneous teachers.

A compute-matched comparison would further strengthen the experimental conclusions.

W2. The theoretical motivation and empirical validation of Proxy Merging are not fully convincing.

Proposition 1 establishes that an optimally merged model achieves classification loss no greater than the best individual proxy. However, this property follows directly from the optimization domain containing every individual proxy as a feasible solution and does not fundamentally require the convexity assumption.

Moreover, the implementation optimizes merging coefficients using Adam for only two epochs, without guaranteeing the global optimum assumed in the proposition.

More importantly, the theoretical result concerns training loss and does not establish that the merged initialization lies in a flatter or better-generalizing region of the loss landscape.

Indeed, Appendix Table 11 shows that the learned model-level merging achieves lower pre-SPD test accuracy than the best individual proxy on both EgoExo-Fitness (81.4% vs. 81.9%) and Assembly101 (48.3% vs. 49.5%). Although this does not contradict the training-loss proposition, it suggests that the claimed optimization and generalization advantages require stronger justification.

Questions for authors

Q1. Could the authors provide a control experiment in which all nine proxies are trained using only the classification loss, without teacher feature distillation, while keeping Proxy Merging and SPD unchanged? This would help isolate the contribution of heterogeneous teacher knowledge from additional supervised training and model consolidation.

Q2. Are all proxy models initialized from identical pretrained weights? If not, how do the authors ensure parameter compatibility during model merging? Additionally, how is feature-level alignment maintained between independently trained proxies and the final student?

Q3. Could the authors clarify the relationship between Proposition 1 and the practical optimization procedure? In particular, why should minimizing the training classification loss over convex merging coefficients lead to a flatter or better-generalizing solution?

Q4. Could the authors report results across multiple random seeds, especially on EgoExo4D, where the improvement over the strongest competing method is only 0.4 percentage points? Mean performance and standard deviations would help establish the robustness of the reported improvements.

Q5. Could the authors provide a compute-matched comparison against direct multi-teacher distillation, including the total GPU-hours required by ProxyDist? This would clarify the performance gains relative to the additional training cost.

3 comments
AuthorsOpenReview ↗

Response to Reviewer VuCi [Part 1/2]

Q1. Control experiment to isolate heterogeneous teacher knowledge

We thank the reviewer for suggesting this control experiment. For isolating the contribution of heterogeneous teacher knowledge from additional supervised training and model consolidation, we trained nine RGB proxies using only the action classification loss, disabling teacher feature distillation. We denote these as classification-only proxies. We then applied the same Proxy Merging and SPD procedures used in π\large \pi-EGO. The results are reported below; proxy-level mean ± standard deviation is computed across the nine proxies.

Method / DatasetEEFA101
Baseline80.347.6
Classification-only proxies (mean ± std)80.6 ± 0.447.6 ± 0.2
Teacher-distilled proxies (mean ± std)81.0 ± 0.748.4 ± 0.4
Classification-only proxies + Proxy Merging + SPD82.149.0
ProxyDist(Ours): teacher-distilled proxies + Proxy Merging + SPD84.750.7

We observe that distillation from classification-only proxies improves over the baseline, confirming that additional supervised training and model consolidation contribute to performance. However, it remains below π\large \pi-EGO by 2.6 points on EEF and 1.7 points on A101, despite using the same Proxy Merging and SPD procedures. This gap is consistent with the lower average standalone accuracy of classification-only proxies (Rows 2-3). These results demonstrate that heterogeneous teacher knowledge provides additional benefits beyond RGB-only ensemble distillation. We have added this comparison in the revised manuscript. [L384; L427-L446]

Q2. Parameter compatibility and feature-level alignment

We thank the reviewer for this question. Yes, all nine proxies are initialized from identical ImageNet-pretrained weights using TimeSformer’s default configuration. They share the same architecture and parameter indexing, while their common initialization provides a shared starting point for subsequent teacher-specific training. These choices enable corresponding parameters to be merged directly.

For feature-level distillation, the proxies and final student use the same backbone and produce features of identical dimensionality, so no additional projection is required for dimensional matching. In contrast, Level-I distillation uses projectors to accommodate dimensional differences between heterogeneous teachers and their corresponding proxies (see L305-308).

We know that matching architectures and dimensions does not, by itself, guarantee semantic alignment. However, in π\large \pi-EGO, all proxies process the same egocentric RGB input and learn the same recognition task, providing a common basis for their representations. The student is initialized through Proxy Merging and subsequently learns to match the selected proxy features through the SPD objective; no separate alignment stage is introduced. Figure 3 provides empirical support for reduced representational discrepancy: the proxies exhibit higher pairwise CKA similarity than the original heterogeneous teachers. This supports the role of proxy learning in reducing the representational gap, without implying that independently trained proxy features are perfectly aligned.

AuthorsOpenReview ↗

Response to Reviewer VuCi [Part 2/2]

Q3. Proposition1 and model merging

We thank the reviewer for this question. The relationship between Proposition 1 and the generalization properties of the merged initialization operates through two complementary mechanisms, which we acknowledge Section 3.2 did not articulate clearly enough.

Proposition 1 establishes a conditional guarantee under local convexity of the classification loss: minimizing over convex merging coefficients yields a model that achieves no higher classification loss than the best individual proxy. This ensures the merging procedure is well-posed as an initialization strategy. However, as the reviewer correctly identifies, a lower training classification loss does not by itself imply better generalization.

The generalization intuition draws separately from the model-merging literature: convex combinations of models sharing a common initialization tend to reside in broader, flatter loss landscape regions than individual models (Izmailov et al., 2018; Frankle et al., 2020). Our proxies are consistent with this setting, i.e. sharing identical pretrained weights and architecture, differing only in their distillation targets, making this intuition applicable. Importantly, the merged model encodes a consensus across all proxy representations rather than the bias of any single one, which we argue provides a more neutral and stable starting point for subsequent distillation.

Most directly, Table 11 provides empirical evidence for this: despite marginally lower pre-SPD accuracy than the Best Proxy (81.4% vs. 81.9% on EEF; 48.3% vs. 49.5% on A101), model-level merging yields strictly better post-SPD accuracy (84.7% vs. 83.7% on EEF; 50.7% vs. 50.6% on A101). This consistent dissociation across both datasets confirms that the advantage of Proxy Merging lies not in its standalone classification performance but in its ability to place π\large \pi-EGO in a region more amenable to subsequent distillation. We have revised Section 3.2 to make this reasoning explicit [L244 - L252].

Q4. robustness of the reported improvements

We agree that evaluating multiple random seeds is important when interpreting the small performance difference on EgoExo4D. Thus, we have repeated our experiments with three random seeds {0,1,2} on all three datasets. The mean accuracy and standard deviation are reported below:

DatasetAccuracy (mean ± SD)
EgoExo-Fitness84.3 ± 0.4
Assembly-10150.6 ± 0.1
EgoExo4D41.0 ± 0.1

We have added this comparison in the revised manuscript. [L1026-L1036]

Q5. compute-matched comparison against direct multi-teacher distillation

We thank reviewer VuCi for this question. We evaluated direct multi-teacher distillation with an extended training schedule to match the GPU-hour budget of ProxyDist on EgoExo-Fitness and Assembly101. Let (NN) denote the cost of training one proxy. Then, training nine proxies and subsequently distilling into one student requires approximately (10N10N) GPU-hours, assuming comparable costs for each proxy and the final student. With (N=1.5N=1.5) GPU-hours on EgoExo-Fitness and (N=28N=28) GPU-hours on Assembly101, these stages require approximately 15 and 280 GPU-hours, respectively.

DatasetTraining budget (GPU-hours)Direct multi-teacher distillationProxyDist
EgoExo-Fitness1582.584.7
Assembly10128049.250.7

Under these matched training budgets, ProxyDist outperforms direct multi-teacher distillation by 2.2% on EgoExo-Fitness and 1.5% on Assembly101. These results indicate that the gains cannot be attributed solely to additional training compute and support the effectiveness of the proposed proxy-based knowledge transfer. We have revised manuscript to included this comparison. [L1040-L1058]

We thank the reviewer VuCi again for the valuable feedback.

We would appreciate hearing whether our response has addressed the reviewer’s concerns, and we would be happy to provide further clarification if any concerns remain.

Reviewer_EbE4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ProxyDist for learning an egocentric RGB representation from multiple teachers. Instead of distilling the different teachers directly into one student, it first trains a separate RGB proxy for each teacher. The proxies share the student's architecture, are merged to initialize it, and then provide supervision according to their reliability on each training example. The experiments cover action recognition, retrieval, and temporal action segmentation on three ego-exo datasets.

Critical strengths
  • I think the idea of translating different teachers into a common input and architecture before combining them is reasonable. Teachers observing different modalities and viewpoints may provide conflicting supervision, so giving each one its own intermediate model could make the final learning problem easier.
Critical weaknesses
  • My main issue is that the pipeline is super complicated and whether this heavy training process is necessary. The method trains nine separate RGB proxies, merges them, and then trains another student with sample-dependent proxy selection. Every proxy also learns from action labels. This means the final student benefits from several independently trained, task-specific RGB models, not just from a better way of translating heterogeneous teacher knowledge. A natural question is whether training several RGB models with action supervision and distilling their ensemble would already give much of the improvement. Without that comparison, I find it difficult to tell whether the proxy idea is solving the stated modality/viewpoint problem or mostly providing a stronger collection of supervised teachers. The extra training may still be worthwhile, but I would like to understand that trade-off against a simpler distillation recipe.
Questions for authors

See Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer EbE4

W1. Comparison with action-supervised RGB ensemble distillation

We thank Reviewer EbE4 for raising this concern. We agree that this comparison is essential for distinguishing the benefits of training multiple task-specific RGB models from those of transferring heterogeneous teacher knowledge. Therefore, we trained nine RGB models using action classification loss alone, without heterogeneous teacher distillation, and evaluated their ensemble and subsequent distillation into a single student, both with and without SPD. The results are reported below, with mean ± standard deviation computed across the nine RGB models.

Method / DatasetEEFA101
RGB Baseline80.347.6
Nine classification-only RGB models (mean ± std)80.6 ± 0.447.6 ± 0.2
Nine-model RGB ensemble82.149.1
RGB-only distillation with SPD82.149.0
RGB-only distillation without SPD82.349.2
π\large \pi-EGO (Ours)84.750.7

We find that RGB-only ensemble distillation improves over the baseline, confirming that training and consolidating multiple action-supervised models contributes to performance. Nevertheless, π\large \pi-EGO outperforms the strongest RGB-only distillation variant by 2.4 % on EEF and 1.5 % on A101. Thus, the gains cannot be explained solely by access to multiple task-specific RGB models; heterogeneous teacher supervision provides an additional benefit.

Interestingly, we also observe that SPD does not improve performance in this classification-only setting: distillation without SPD achieves slightly higher accuracy on both datasets. This suggests that sample-dependent selection is beneficial in π\large \pi-EGO as it exploits differences in proxy reliability arising from heterogeneous supervision. Although all proxies receive RGB inputs, those in π\large \pi-EGO learn from teachers spanning different modalities, viewpoints, and representations, whereas classification-only models share the same source of supervision. The observed results further confirms that SPD is more useful when such complementary supervision is available, rather than merely from increasing the number of RGB models.

These comparisons clarify the trade-off: the simpler RGB-only distillation recipe provides meaningful gains, while our full training pipeline yields further improvements on both datasets. The additional complexity is confined to training; inference uses a single egocentric RGB model. We have include this comparison in the revised manuscript. [L384; L427-L446]

Reviewer_EbE4OpenReview ↗

Final Review

Thank you authors for the rebuttal and the additional experiments. It's great to see π\pi-Ego benefits from training and consolidating multiple action-supervised models compared to RGB-only distillation. The ablation for SPD is also interesting to know in what scenarios it helps. As my question has been answered, I raised my score to Weak Accept.

AuthorsOpenReview ↗

Thank you for feedback and updated score.

We thank the reviewer again for the valuable feedback and for updating the score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to all reviewers for their thoughtful feedback (Summary of reviews)

We thank the reviewers for their thoughtful feedback. We appreciate their recognition of two central aspects of our work:

Motivation and problem setting. Reviewers described the paper as “well-motivated” [WMis], addressing “a meaningful challenge in egocentric video understanding” [VuCi], and noted that the “idea of translating different teachers into a common input and architecture before combining them is reasonable” [EbE4].

Framework design. Reviewers recognized that the proxy-based framework can “make the final learning problem easier” [EbE4], “provides a coherent approach” [VuCi], and is “efficient at inference” [WMis].

The reviews raised important questions about the source of the improvements, training cost, theoretical justification, and applicability. In response, we provide additional control experiments with classification-only RGB models, a compute-matched comparison against direct multi-teacher distillation, and evaluation on EgoMe. These results help distinguish the contribution of heterogeneous teacher knowledge from additional supervised training and model consolidation. We address the remaining concerns in our individual responses.

We have revised the manuscript accordingly and highlighted the changes in blue.

Dear Reviewers,

Authors posted their responses to your queries. Please read these messages and post your replies so that we get a chance to clarify points which are possibly still unclear and may require further discussion.

Thank you for supporting the ICLR revision process and contributing to discussion.

Best, The AC

AuthorsOpenReview ↗

Summary of Responses and Additional Experiments

We sincerely thank the Reviewers and AC for their time and thoughtful feedback. As the author-involved discussion period draws to a close, we would like to briefly recap the initial reviews and what we have added in response.

We were encouraged by the positive feedback on the motivation, problem setting, and framework design from all reviewers. The main concerns focused on strengthening the empirical validation and clarifying methodological details. In response, we completed all requested experiments and provided additional explanations. These experiments consistently converged on that π\large \pi-EGO’s performance improvements do not originate from additional parameters, compute, or training data, but from the proposed framework itself. We believe these experiments directly address the gain-attribution concerns raised by Reviewers EbE4 and VuCi. Reviewers EbE4 and WMis have confirmed their concerns were addressed by our rebuttal and both hold positive ratings. We believe that our experiments and clarifications address Reviewer VuCi’s concerns as well, and we still welcome their response before the deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.