acceptodds
Under review as a conference paper at ICLR 2027

Fusion-to-Branch Distillation for Label-Efficient Multimodal Activity Recognition

Abstract

Label-efficient multimodal learning uses complementary views to reduce the need for labeled data. In multimodal human activity recognition (HAR), this requires a strong complete-view predictor and modality-specific predictors that remain accurate when only one sensor stream is available. Existing semi-supervised methods either create targets from one view or exchange predictions between views, so no branch learns from targets based on both modalities. We introduce fusion-to-branch distillation (F2B), which transfers the complete-view posterior to each modality branch on paired unlabeled examples. A single training run produces one fusion predictor and two single-modality predictors. F2B requires neither a pretrained teacher nor an additional network at inference. We evaluate F2B on five HAR datasets using four label budgets. Compared with supervised training, F2B improves fusion performance in 17 of 20 dataset-budget settings. It improves the direct branches in 38 of 40 evaluations. In 17 of 20 settings, the average branch improvement is also larger than the fusion improvement. Ablations on three datasets show that each branch benefits more from predictions based on the other modality than from its own, supporting cross-modal transfer, although the small number of independent evaluation units limits statistical power. At inference, using one branch requires 44-56% fewer floating-point operations than full fusion. Overall, F2B improves single-modality prediction while retaining the benefits of multimodal fusion in one model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal learning, semi-supervised learning, knowledge distillation, human activity recognition, sensor fusion, label-efficient learning, missing-modality robustness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_HF9fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies label-efficient activity recognition from paired geometric (pose/skeleton) and inertial inputs. F2B jointly trains a fusion predictor and two independently deployable modality branches. On selected unlabeled pairs, the detached weak-view fusion distribution supervises the strongly augmented branches, alongside supervised training and fusion pseudo-label learning. The paper evaluates five datasets and four label budgets, with teacher-source ablations on three datasets and comparisons with unimodal specialists. The complete training recipe improves most branches over supervised training; matched ablations show that the incremental benefit of the F2B loss depends on the dataset and target branch.

Critical strengths
  1. A useful joint training and deployment objective. One training run retains both complete-input fusion and independently usable single-sensor predictors, without a separately pretrained teacher. This addresses a concrete deployment need when sensor availability changes, and its value does not require every branch to outperform a dedicated unimodal model.

  2. Meaningful controls and transparent reporting. The paper includes matched no-distillation controls, teacher-source comparisons, specialist comparisons, and evaluation-unit-based uncertainty estimates. Table 5 provides positive evidence for the F2B loss on CZU-MHAD: geometric and inertial branches improve by 3.71 and 6.48 percentage points, respectively, with unadjusted 95% confidence intervals excluding zero. These results establish that the core loss can be beneficial, while the reported negative and uncertain results help delimit its applicability.

Critical weaknesses
  1. The broad gains of the complete recipe are stronger than the evidence for the core F2B loss. The 38/40 branch improvements over supervised training also include the contribution of unlabeled fusion training, as the authors acknowledge. In the matched 10%-label ablation, the geometric branch changes by −0.51 pp on MM-Fit and −0.67 pp on mRI; inertial gains are +2.83 and +2.50 pp, but all four intervals include zero. Together with the positive CZU-MHAD results, this supports conditional effectiveness rather than a consistently established incremental benefit. Tables 2 and 3 evaluate the complete training recipe at four label budgets. However, the comparison that removes only the F2B loss while retaining the remaining training components is reported at the 10% budget in Tables 4 and 5. The independent contribution of the distillation loss at other budgets therefore remains unresolved.

  2. The added value of the full fusion teacher remains insufficiently established relative to close alternatives. Online ensemble-to-branch distillation has been studied in ONE, while prior incomplete-modality learning and multimodal distillation methods have explored knowledge transfer under missing-input settings. The paper acknowledges related work in both areas. F2B’s sensor-specific branches and deployment design are meaningful differences. However, the clearest teacher-source finding—other-branch teaching outperforming same-branch teaching—supports cross-modal transfer without isolating the advantage of the full fusion teacher. The paper also reports that no reduced teacher dominates across datasets. Given the overlap with prior work, stronger evidence that the full fusion teacher improves on simpler cross-branch teaching is needed to establish the method’s contribution.

Questions for authors
  1. Clarification using existing controls: What are the paired performance differences between the full fusion teacher and the other-branch teacher for the direct branches, and how variable are these differences across evaluation units? Do the existing results show an additional benefit from including the student’s own modality in the teacher?

  2. Isolating the distillation effect across label budgets: Tables 2 and 3 report results at four label budgets. Beyond the 10% setting, are there comparisons that remove only the F2B loss while retaining the same class-adaptive threshold and other training components? If available, do these comparisons show the same dataset- and branch-dependent pattern as Table 5?

2 comments

Thank you for the careful reading and for two questions that go straight to the core of the paper. Your distinction between the full training recipe and the F2B loss itself was especially useful and shaped the revision. In short:

  • Q2 (the F2B loss beyond 10% labels): the loss raises the branch mean in most comparisons, but its effect depends on the dataset, branch and label budget.
  • Q1 (fusion teacher versus other-branch teacher): our controls do not show a consistent advantage for the fusion teacher, so the revision no longer claims one.

Q2: the F2B loss at every label budget

As you suggested, we removed only the F2B loss and kept the class-adaptive threshold and every other component, at all four budgets on four datasets, in two pipelines. The main pipeline is the setup of Tables 2–3 (batch size 32 on CZU-MHAD; the 25% representation selector on mRI and UTD Kinect V2) and reproduces their F2B means at every budget. The control pipeline is the setup of Table 5.

Main pipelineControl pipeline
Comparisons where the loss raises the mean26 of 3222 of 32
Gains that survive Holm correction25
Decreases that survive Holm correction11

F2B minus no distillation, in pp (geometric / inertial):

Dataset10%15%25%50%
Main pipeline
CZU-MHAD+3.7 / +2.8+1.9 / +5.5+2.0 / +3.1+0.9 / +0.8
mRI+0.3 / +1.8+2.0 / +1.5+0.7 / −0.3+0.2 / +1.7
UTD Kinect V2+4.7 / −2.0+0.1 / +4.3+1.4 / +2.7+3.0 / +3.2
Control pipeline
CZU-MHAD+3.7 / +6.5+2.7 / +4.3+2.5 / +3.5+0.4 / +1.6
mRI−0.7 / +2.5+0.3 / +5.0−0.3 / +2.2−0.2 / +0.7
UTD Kinect V2+3.0 / −0.8+6.0 / −0.2+6.3 / −0.9+2.6 / +2.6
Both pipelines
MM-Fit (macro-F1)−0.5 / +2.8+2.8 / +4.8−0.5 / +1.8−1.8 / −1.9

The gains that survive Holm correction are MM-Fit inertial at 15% in both pipelines, CZU-MHAD inertial at 25% in the main pipeline, and CZU-MHAD inertial at 10% and 15%, CZU-MHAD geometric at 25% and UTD Kinect V2 geometric at 15% in the control pipeline. Each is positive in every held-out unit. The one significant decrease is MM-Fit geometric at 50%.

Your expectation of a dataset- and branch-dependent effect holds. CZU-MHAD gains at every budget in both pipelines, while the mRI geometric branch stays near zero in the control pipeline. The loss can also hurt: at 50% labels it lowers both MM-Fit branches. On the segmented datasets, the main pipeline has more positive means (22 vs 18 of 24) but fewer significant gains (1 vs 4), and no cell is Holm-significant in both. The CZU-MHAD 10% gains you highlighted (+3.7 / +6.5) become +3.7 / +2.8. Section 5.7, Table 7 and supplementary Sections I.2–I.3 report all values.

Q1: fusion teacher versus other-branch teacher

This is the right test, and the answer narrows our claim. Other-branch minus fusion teacher at 10% labels (control pipeline at Section 5.4), as mean [unadjusted 95% CI] and the units where the other-branch teacher is higher. A negative value means the fusion teacher is better.

DatasetGeometricInertial
MM-Fit−0.53 [−3.57, 2.51], 3/5−2.14 [−3.12, −1.16], 0/5
mRI+0.67 [−1.03, 2.37], 4/5+1.50 [−2.19, 5.19], 3/5
CZU-MHAD+0.72 [−3.48, 4.93], 1/5+1.07 [−3.87, 6.02], 3/5

The fusion teacher has the higher mean in two of these six cells, and only MM-Fit inertial has an interval that excludes zero. Across all budgets in the main pipeline, it has the higher mean in 16 of 24 comparisons, and none survives Holm correction. There, cross-sensor teaching has the higher mean than self-teaching in 18 of 24 comparisons (one survives Holm correction), but in only 3 of 8 on mRI. The student's own modality in the teacher has a branch-dependent effect: removing its direct logit changes MM-Fit geometric by +0.98 pp (5/5 units) and inertial by −1.34 pp (0/5) (supplementary Table 3), the gate and residual still see both sensors.

We agree that the added value of the fusion teacher is not established, and the revision removes that claim (abstract, introduction, Section 5.4, conclusion). We keep the fusion head as the teacher because it is also the two-sensor predictor (Section 3.2) and unlike in ONE, each branch observes a different sensor and is evaluated alone. This does not affect Q2, which compares F2B with no distillation. The contribution we claim is the one your review identified: one training run that keeps the fusion predictor and both single-sensor predictors without a pretrained teacher, together with the matched evidence above for the loss.

Your review made the paper more precise. We would appreciate a reassessment in light of the matched ablations now covering every label budget under the main-table protocol.

Thank you for your response.

Reviewer_6PnQOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Fusion-to-Branch Distillation (F2B), which transfers knowledge from a multimodal fusion predictor to other single-modality branches using paired unlabeled data. Experiments on five datasets show consistent improvements, particularly for single-modality prediction.

Critical strengths
  1. The paper shows well improvements on five datasets.
  2. The paper is easy to follow and understand.
  3. Idea and motivation are all clear.
Critical weaknesses
  1. The novelty is very limited. The method is close to online knowledge distillation, self-distillation, or multimodal-to-unimodal distillation. It is intuitive that this approach would work.

  2. The distillation method itself is very simple. The authors seem to claim it as a general distillation method. Although it is validated on multiple HAR datasets, it may need to be tested in other domains, such as medical applications or more general multimodal classification benchmarks. If it is only validated on HAR datasets, it may be more suitable for venues such as WACV or IMWUT. The contributions demonstrated by the paper fall short of ICLR standards and lack novelty.

  3. F2B branches are not shown to significantly outperform matched semi-supervised FixMatch specialists. The main benefit is flexibility rather than clearly superior unimodal accuracy.

  4. Adapted CoSMo, DMCL, and MESEN are relatively old works from 2021–2023, and recent baselines are missing. Distillation has also been studied for a long time in missing-modality learning, while comparisons with well-known methods such as ShaSpec are absent.

  5. Regarding the writing, the abstract is too long and should be more concise. Section 3.3, “What Each Branch Learns,” also seems unnecessary here and could be integrated into the introduction, since readers should be able to understand this point after reading the paper.

Questions for authors

See above

2 comments

Thank you for the review, and for finding the paper clear and the improvements consistent across five datasets.

Novelty relative to online, self- and multimodal-to-unimodal distillation

The paper positions F2B against co-training, ONE and DML, and the revision adds MvPL (Xiong et al., ICCV 2021). The setups differ in who sees what:

Teacher seesEach student seesSeparately trained teacher
Online KD (ONE, DML), self-distillationthe same input as the studentsthe same inputno
Two-stage multimodal-to-unimodal KDboth modalitiesone modalityyes
F2Bboth sensors (fusion head)one sensorno

We claim neither the online-distillation nor the cross-view principle. The contribution is one jointly trained model that keeps and evaluates fusion and both single-sensor predictors, with controlled evidence of where its gains appear.

"It is intuitive that this approach would work"

The overall improvement does not show whether the distillation loss contributes, or whether fusion is the best teacher. Our controls test both, and two plausible expectations fail:

  • The loss does not always help. Adding the F2B loss to an otherwise matched no-distillation control improves the branches in 26 of 32 comparisons in the main pipeline of Tables 2–3 (two survive Holm correction) and 22 of 32 in the control pipeline (five survive). At 50% labels on MM-Fit, where the two pipelines coincide, it lowers both branches; the geometric decrease survives correction.
  • A teacher that sees both sensors is not consistently better. Cross-sensor teachers usually beat self-teaching, but the fusion teacher shows no consistent advantage over an other-branch teacher.

Validation only on HAR

Agreed. Claims are now limited to paired pose/skeleton and inertial HAR, Section 6.3 states that other domains are untested, and we make no claim of a general distillation method. The question studied, when a jointly trained fusion teacher helps single-sensor students, is a learning question that the controls above test.

No significant gain over FixMatch specialists

No difference is detected (Table 6, control pipeline, 10% labels on mRI and CZU-MHAD: −1.67-1.67 to +1.29+1.29 pp, all intervals include zero), and the paper now says so. We claim output coverage, one model with fusion and both single-sensor predictors, not superior unimodal accuracy.

Recent baselines and ShaSpec

We ported the official ShaSpec code (shared and specific encoders, α=0.1\alpha = 0.1, β=0.02\beta = 0.02, two-stage schedule with random missing modalities) to our encoders, splits and seeds; supplementary Section I.1 lists four implementation differences. ShaSpec is supervised, so we add the semi-supervised losses of our no-distillation control. ShaSpec minus F2B in the control pipeline, in pp (two-sensor / geometric / inertial):

Dataset10%15%25%50%
MM-Fit (macro-F1)−0.1 / −3.4 / −21.1−0.5 / −5.8 / −16.7−0.5 / −3.9 / −9.6+3.9 / +1.4 / −2.4
CZU-MHAD−6.5 / −19.7 / −15.8−5.7 / −19.5 / −13.4−5.0 / −16.6 / −12.9−1.7 / −8.8 / −6.7
mRI−9.2 / −14.8 / −12.5−3.8 / −12.8 / −15.2−2.0 / −8.7 / −12.0−0.7 / −9.3 / −17.3
UTD Kinect V2−7.9 / −6.6 / −12.1−3.4 / −14.4 / −9.3−1.2 / −9.3 / −9.2−7.3 / −19.8 / −10.9

F2B is higher in 31 of 32 single-sensor cells (unadjusted 95% interval excludes zero in 29) and 15 of 16 two-sensor cells (4 excluding zero, all CZU-MHAD). The model without the F2B loss is also higher in all 32 single-sensor cells, so this advantage does not depend on the loss. On MM-Fit, seven of ShaSpec's eight single-sensor outputs are below our supervised model, and the port is not tuned per dataset, so it may understate ShaSpec. More recent incomplete-modality methods are not compared (Section 6.3).

Writing

The abstract is shorter, and the former Section 3.3 is now a paragraph in Section 3.2.

I sincerely appreciate the authors' efforts in conducting additional experiments and providing further clarifications. While these additions have strengthened the empirical evaluation, I still have some reservations about whether the technical novelty meets the high standards typically expected at ICLR. Therefore, I will maintain my original rating.

Reviewer_mS2cOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This work proposes a method which builds on FreeMatch, but with a fusion based teacher. It proposes a gated fusion head and two single-sensor branches which are trained jointly, and the fusion head’s confident weak-view prediction becomes a soft target for each branch’s strongly augmented view. Across five small scale skeleton+IMU action-recognition datasets, branch accuracy improves in 38 of 40 settings, giving one model that can fall back to whichever sensor is available.

Critical strengths
  • The results show an improvement over a range of IMU+Pose datsets and label availabilities
  • The limitations are clear and provide a realistic framing for this work
Critical weaknesses
  • The datasets used are small in scale and limited to the pose + IMU setting. They don't provide a strong enough foundation to point to for this method to be considered general.
  • Both the results and methods of this work directly follow from prior work namely Multiview Pseudo-Labeling for Semi-supervised Learning from Video (Xiong et al 2021). There are some methodological differences in the modality but the overall method of using a FreeMatch approach with the idea of using a multimodal label to supervise unimodal models to improve the unimodal performance is consistent.
  • Results for the fused are consistent but not strong enough to show significant effect as you mention in Section 5.2. The direct branches also show an improvement over selected prior works, but not the broader literature (including unimodal specialist as shown in S5.5)
  • The claim that this training method "transfers the complete-view posterior" seems over stated. Since the other modalities data is never available to a direct branch except through the fusion supervision the best it can do is the average of the fused prediction over the other modalities signals. A more accurate description may be that a branch learns the complete-view posterior with the other sensor average out.
Questions for authors
  1. What is the novelty claim in light of (Xiong et al 2021)?
1 comment

Thank you for the careful review, and for noting the range of datasets and label budgets and the clear limitations.

Q1. Novelty in light of MvPL (Xiong et al., ICCV 2021)

We should have cited MvPL, and the revision does. Both methods supervise each view with a pseudo-label aggregated across views of a weakly augmented example, and MvPL's Table 1c shows that cross-view and aggregated pseudo-labels outperform a view's own. We do not claim that principle. F2B applies it in a different setting:

MvPLF2B
InputsRGB, flow and temporal-gradient views computed from one clip, one shared networkTwo separate sensors with separate encoders; either may be missing at test time
TeacherAverage of the view predictions, used as a hard labelLearned fusion head trained in the same run; its soft posterior is the target
Predictors keptRGB modelFusion predictor and both single-sensor predictors, all evaluated

We make no claim that the fusion teacher outperforms an averaged cross-view teacher and the head is kept because it is also the two-sensor predictor.

What we add is the use of this principle for paired sensors with a fusion teacher, together with evidence addressing three distinct empirical questions:

  1. Where the gains of the full method appear. Relative to supervised training, the branches gain more than fusion in 17 of 20 settings in absolute points, and in 12 of 20 after normalizing each gain by 100100 minus the supervised score.
  2. The distillation loss itself. A matched ablation at every label budget on four datasets, in two pipelines. The loss raises the branches in 26 of 32 comparisons in the main pipeline of Tables 2–3 (two survive Holm correction) and in 22 of 32 in the control pipeline (five survive). The effect depends on dataset, branch and budget, and can reverse: in both pipelines the loss lowers both MM-Fit branches at 50% labels, and the geometric decrease survives correction.
  3. The teacher source. Cross-sensor teachers usually beat self-teaching, while the fusion teacher shows no consistent advantage over an other-branch teacher.

"Transfers the complete-view posterior"

Thanks for raising, the phrase was imprecise. For a fixed teacher pfp_f and an unrestricted branch predictor, the expected distillation loss is minimized by

πm∗(⋅∣xm)=arg⁡min⁡π  E[KL(pf(⋅∣xg,xa) ∥ π(⋅∣xm)) ∣ xm]=E[pf(⋅∣xg,xa) ∣ xm].\pi_m^*(\cdot \mid x_m) = \arg\min_{\pi}\; \mathbb{E}\big[\mathrm{KL}\big(p_f(\cdot \mid x_g, x_a)\,\|\,\pi(\cdot \mid x_m)\big) \,\big|\, x_m\big] = \mathbb{E}\big[p_f(\cdot \mid x_g, x_a) \,\big|\, x_m\big].

On confident pairs, each branch receives the full fusion posterior as a per-example target, but from xmx_m alone it can at best learn that target averaged over the unobserved sensor, which is the description you propose. With the confidence mask, the average is over the selected pairs. The identity concerns the student distribution used in the KL loss; with temperature softening (τt=1\tau_t = 1, τs=2\tau_s = 2), it applies to the softened student distribution, not to the inference posterior. The revised abstract, introduction and Section 3.2 (Eq. 6) now say this.

Fusion results

Agreed. Fusion improves in 17 of 20 settings, by small margins. We report these mean increases but claim neither a uniform nor a statistically established fusion improvement at paper's Section 5.2.

Branches versus unimodal specialists

In the two-dataset control, the direct branches show no statistically detected difference from semi-supervised FixMatch specialists; the revision says this instead of claiming parity. We do not claim that the branches beat the broader unimodal literature. The benefit is that one training run yields a fusion predictor and two single-sensor predictors, instead of separately trained models.

Dataset scale and modality pair

Agreed. The five datasets are small, and all pair pose or skeleton with inertial data. The revision claims nothing beyond paired pose/skeleton and inertial HAR.

In summary, beyond applying cross-view pseudo-labeling to paired sensors, the revised study separates the gains of the full method from the incremental effect of the distillation loss and tests whether fusion provides a better teacher. It reports both benefits and failures while retaining and evaluating all three predictors.

Comments on the paper

1 comment

We thank all three reviewers for their time and their thoughtful reviews. Each review pointed to a real gap, and the paper is clearer and more precise for it. We have uploaded a revised version of the paper, with all changes highlighted in colour. We summarize the main changes below.

  1. MvPL and related distillation. Reviewer mS2c rightly pointed us to MvPL by Xiong et al. at ICCV 2021. We should have cited it, and the revision now does. It is the closest prior work, since both methods supervise each view with a pseudo-label aggregated across views. We do not claim this principle. What we add is its use for paired sensors with a fusion teacher, and controlled evidence of where the gains appear and where they fail. MvPL computes its views from one video clip, processes them with one shared network and keeps only the RGB model at test time. F2B works with two separate sensors and separate encoders, uses a learned fusion head as the teacher, and keeps and evaluates the fusion predictor and both single-sensor predictors. Reviewer 6PnQ rightly placed F2B near online and multimodal-to-unimodal distillation. We claim neither principle, and unlike two-stage multimodal-to-unimodal distillation, our fusion teacher is trained in the same run.
  2. The F2B loss at every label budget. Following Reviewer HF9f, we ran the control that removes only the F2B loss at all four budgets on four datasets and in two pipelines. In the main pipeline of Tables 2–3, the loss raises the branch mean in 26 of 32 comparisons, and two gains survive Holm correction. In the control pipeline, it does so in 22 of 32, with five surviving. The loss can also hurt: on MM-Fit at 50% labels it lowers both branches, and the geometric decrease survives correction.
  3. ShaSpec. As Reviewer 6PnQ requested, we added ShaSpec. In the control pipeline, F2B is higher than our ShaSpec port, which we did not tune per dataset, in 31 of 32 single-sensor comparisons. The same model without the F2B loss is higher in all 32, so this advantage does not depend on the loss.
  4. What a branch learns. Reviewer mS2c correctly noted that our wording overstated this point. Each branch receives the fusion posterior as a per-example target, but from its own input it can at best learn that target averaged over the unobserved sensor. The text now says so.
  5. Teacher source. Cross-sensor teaching has the higher mean than self-teaching in all six 10%-label comparisons in the control pipeline on MM-Fit, CZU-MHAD and mRI. Across budgets in the main pipeline, it has the higher mean in 18 of 24 comparisons, though only 3 of 8 on mRI, and one difference survives Holm correction. The fusion teacher has the higher mean than the other-branch teacher in 16 of 24 comparisons, but none survives correction. These controls do not establish a consistent advantage for the fusion teacher. We keep it as the teacher because it is also the two-sensor predictor.
  6. Scope and contribution. Following Reviewers mS2c and 6PnQ, all claims are now limited to paired pose or skeleton and inertial HAR. The branches show no statistically detected difference from FixMatch specialists, and the paper now says so. What we claim is one training run, with no pretrained teacher, that keeps a fusion predictor and two single-sensor predictors, and its branches improve over supervised training in 38 of 40 settings.

Two pipelines. We want to be transparent about one point. Tables 5–6 and the ShaSpec comparison use a control pipeline, and the original submission did not say how it differs from the main evaluation. It uses the same folds, seeds and labeled subsets for every method and matches the main evaluation on MM-Fit. On the segmented datasets it differs in two settings. CZU-MHAD uses batch size 16 instead of 32, and mRI and UTD Kinect V2 omit a learned representation selector that keeps 25% of each encoder's coordinates. Restoring these settings reproduces the F2B means of Tables 2–3 at every budget to displayed precision, and we reran the loss and teacher ablations in this main pipeline, as reported in supplementary Section I.3. Section 4.2 now states the selector, supplementary Section A corrects the batch sizes, and Section 5.7 explains the difference.

Summary of revisions. We have shortened the abstract, discussed MvPL in the introduction and related work, and discussed ShaSpec in the related work and the new Section 5.7. Claims throughout the paper have been revised to match the evidence. Section 5.7 and Table 7 report the loss ablation at every budget and the ShaSpec comparison, and supplementary Section I gives all per-budget values.

Thank you again for your time and guidance. We would appreciate a reassessment in light of the matched ablations at every budget, the clarified teacher-source findings and the explicitly scoped empirical contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.