acceptodds
Under review as a conference paper at ICLR 2027

LEAR: Evidence-Aware Local Learning with Missing Modalities

Abstract

Learning with missing modalities requires a single model to predict from different observed subsets. Under block-local training, each module must learn from the available evidence without receiving gradients from later blocks. We introduce Layer-wise Evidence-Aware Representation Learning (LEAR), extending Forward-Forward-style local learning to prediction over every nonempty modality subset. LEAR turns the predictions of observed modality heads into a detached evidence reference for each sample. Within each block, this reference drives confidence-weighted distillation and coverage-bounded routing; across blocks, normalized detached representations preserve gradient locality. Evidence-Guided Inference (EGI) further combines fusion and available-evidence scores at prediction time. Compared with matched LocalCE, LEAR improves mean-subset Macro-F1 by 0.91 percentage points on WISDM. On CREMA-D, the actor-averaged mean-subset Macro-F1 gain is 9.79 points, including a 3.08-point contribution from evidence-aware training beyond EGI alone. Improvements hold across subject-disjoint folds, modality cardinalities, and 18 of 19 held-out CREMA-D actors. These results establish available evidence as an effective learning signal for coordinating gradient-isolated local modules under changing multimodal inputs.

Primary area:
learning theory
Keywords:
Missing Modalities, Evidence-Aware Learning, Multimodal Representation Learning, Dynamic Modality Subsets, Evidence Distillation, Gradient Locality

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_eumhOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Layer-wise Evidence-Aware Representation Learning (LEAR) for missing-modality classification under block-local optimization. Each block constructs a detached evidence reference from observed modality-head predictions to guide confidence-weighted distillation and coverage-bounded routing while preserving inter-block gradient isolation. Evidence-Guided Inference (EGI) combines fusion and evidence scores before aggregating them across blocks. Experiments on WISDM, CREMA-D, and PAMAP2 report improvements over matched Local CE baselines, with local/global comparisons and component ablations examining the contributions of evidence-aware training and inference-time guidance.

Critical strengths

The paper addresses a relevant setting by combining missing-modality classification with block-local optimization. Its detached evidence reference uses observed modality-head predictions to guide local learning without requiring an external teacher or unavailable full-input information. The experiments cover both sensor-based activity recognition and audio-visual emotion recognition, with matched local/global controls and component ablations that provide useful comparisons of the training and inference mechanisms. On WISDM, LEAR improves over Local CE across all subject-disjoint folds and achieves performance close to its globally optimized counterpart, supporting the feasibility of the approach under gradient isolation.

Critical weaknesses

W1. The paper does not establish what evidence mismatch remains under subset-aware Local CE or how inter-block gradient isolation causes or worsens it.

W2. LEAR outperforms Local CE+EGI on CREMA-D in only one of three seeds, while actor-level bootstrapping after seed averaging does not capture training variability.

W3. CE+EGI uses separately selected checkpoints, so its reported gain over CE does not isolate inference-time guidance from model selection.

W4. Comparisons with established local-learning methods are missing, LEAR trails CoMiss on WISDM, and coverage-bounded routing adds only 0.09 percentage points beyond available-evidence KD+EGI.

W5. The combination of self-distillation, adaptive weighting, and score aggregation lacks clear novelty specific to block-local learning, with little analysis of same-block supervision or confidently incorrect evidence.

W6. Exponential subset enumeration remains untested at larger scales, WISDM update time increases by 49%, and detailed encoder/fusion architectures and absolute PAMAP2 results are missing.

Questions for authors

Q1. How is evidence mismatch defined or measured? What failures remain under subset-aware Local CE that are attributable to inter-block gradient isolation, and is this mismatch demonstrably more severe than under end-to-end training?

Q2. Why does LEAR outperform Local CE+EGI on CREMA-D in only one of three seeds (Table A7)? Please provide additional runs, paired per-seed gains, convergence analyses, and uncertainty estimates accounting for variation across training seeds and test samples.

Q3. Can EGI be evaluated on identical Local CE checkpoints without retraining or checkpoint reselection to isolate its inference-only contribution? Please also compare it with simpler weighted combinations of modality-head and fusion-head predictions.

Q4. How does Local CE differ from established locally supervised methods, and why are Forward-Forward or other block-local baselines absent? Is the routing gain consistent across datasets and seeds? Is the routing distribution differentiated through its dependence on the fusion loss, or detached when computing the weighted objective?

Q5. What does the detached evidence reference add beyond ground-truth supervision? How often are high-confidence evidence predictions incorrect, and how does this affect distillation? Please compare entropy-based weighting with unweighted distillation and alternative confidence estimates.

Q6. How does training cost scale with modality count under exhaustive subset enumeration? Can the authors evaluate larger modality sets or a practical alternative to enumeration? Please specify the encoder and fusion architectures, including layer types, dimensions, and input representations, and report complete absolute results on PAMAP2.

Q7. Could the authors revise the overview figure to ensure consistent font sizes and a more compact layout, reducing excessive whitespace and improving readability?

Reviewer_z7dkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LEAR is a supervised block-local method for classification with missing input modalities. At each block, supervised modality heads form a detached average-logit evidence target that guides confidence-weighted distillation and subset routing; Evidence-Guided Inference (EGI) combines fusion and evidence scores at test time. LEAR improves matched Local CE by 0.91 mean-subset Macro-F1 points on WISDM and by 9.66 pooled points (9.79 actor-averaged) on CREMA-D, with additional positive PAMAP2 results. The ablations show that EGI explains 0.60 points on WISDM and 6.71 points on CREMA-D, while evidence-aware training adds a smaller residual gain.

Critical strengths

None

Critical weaknesses
  1. The current experiments do not fully distinguish whether the improvement mainly comes from LEAR’s training strategy or from EGI at inference time. For example, the gain of LEAR over CE+EGI on WISDM is relatively small. Additional controlled experiments and statistical analysis would help clarify the contribution of the training strategy itself.
  2. The experiments mainly cover settings with a small number of modalities and complete training data. Because the method considers different modality combinations, its computational cost may increase considerably as the number of modalities grows. It would be helpful to discuss its applicability to settings with more modalities or missing modalities during training. In addition, the training objective differs from that of the original Forward-Forward method, so the relationship between the two could be explained more precisely.
Questions for authors
  1. Are the routing weights q(l,S) detached before the fusion loss? If not, gradients flow through q because the regret contains the fusion loss. Please state the exact gradient and compare detached versus differentiable routing.
  2. Please report paired uncertainty for LEAR minus CE+EGI on WISDM and compare against a tuned fusion/unimodal ensemble, including EGI weight or temperature tuning, under identical checkpoint selection. Would this change the claimed training benefit?
  3. Can LEAR train without enumerating all subsets and when training samples themselves have missing modalities? Results or complexity beyond four modalities would materially affect the scope assessment.
Reviewer_jqtiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LEAR, an evidence-aware block-local learning framework for missing-modality classification, combining detached modality predictions, confidence-weighted distillation, adaptive routing, and evidence-guided inference (EGI). Although the problem is relevant, the novelty and empirical justification of the proposed framework remain insufficient.

Critical strengths
  1. The paper addresses a clearly defined research problem: how to coordinate local learning objectives under varying modality availability states while maintaining inter-block gradient isolation.

  2. The paper is relatively well-structured and organized.

Critical weaknesses
  1. The manuscript has noticeable formatting issues, particularly on the first page, which cannot be displayed properly in full. The line-numbering and citation styles also appear incorrect.

  2. The proposed method lacks sufficient novelty, and the experimental evaluation is inadequate. Comparisons with recent strong baselines are limited, and the reported performance is not sufficiently competitive.

Questions for authors
  1. Table 1 shows that Local LEAR underperforms Global LEAR. Moreover, Table 1 only includes CoMiss as a recent strong baseline, whereas Table 2 only includes MARS. Why are the baseline methods inconsistent across the two datasets? The comparison with recent competitive methods is insufficient, and including only CoMiss on WISDM is far from convincing.

  2. LEAR largely combines existing techniques, including supervised local learning, knowledge distillation, adaptive loss weighting, and ensemble inference. The novelty of this combination is unclear, and the motivations for the proposed mechanisms are insufficiently explained.

  3. The paper claims its largest improvements on CREMA-D, yet comprehensive component-wise ablation studies are missing for this dataset.

Reviewer_5sP7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents an approach of handling missing modalities during prediction by using layer-wise training. By ensuring that gradients do not propagate across layers, it builds evidences with available modalities and uses that as a reference to distill against the fused modality prediction. The experiments show good performance over baselines on CREMA-D and WISDM benchmarks.

Critical strengths
  1. The paper uses an appropriate matched baseline: Local CE uses the same architecture and training setup. This is
  2. Well structured idea of using predictions from the observed modalities as a "reference" to guide inference.
  3. This problem setting is very relevant - how do we deal with missing modalities and how can we build models that can compensate for that fact.
  4. Paper is honest about the fact that LEAR doesn't necessarily beat CoMiss on WISDM dataset, whereas shows great performance gains on CREMA-D on local-learning baselines while isolating gradients to the blocks.
Critical weaknesses
  1. What is mask-conditioned fusion head? It is not clearly defined. What is the mask, here? The fusion module is not explained in any way preventing a clear understanding of the method.
  2. The confidence score might be improperly designed: The evidence reference is the averaged modality logits and then softmaxed. But there is no guarantee that the modality logits are properly scaled or calibrated w.r.t each other. Perhaps averaging the probabilities is better than using logits. There is no comparison for using probabilities vs logits.
  3. The ablations for the different components for the loss - modality CE, fusion CE, and distillation loss are not thoroughly investigated/presented.
  4. The paper does not explain how missing subsets are represented.
  5. The paper claims to be tightly coupled to Forward-Forward, but the approach is seemingly closer to greedy layer wise supervised learning than the goodness based positive / negative learning of FF.

Minor:

  1. Line 216: "Here B is the minibatch size" - there is no mention of B in the surrounding text. This is confusing. I see it then later used at line 236. Colocate terms being introduced to the immediate equations / text.
  2. Is the available-evidence reference the "teacher" formed by averaging the available modality logits? That is what I understood after multiple passes. If so, this is not clearly explained.
  3. The motivation and sensitivity of η\eta are not concretely explained.

Overall, while reading the paper, I was having trouble understanding the motivation and how it inspired their approach. As mentioned, the paper introduces the problem as an extension to Forward-Forward, but midway through the paper the approach doesn't seemingly have any grounding on Forward-Forward. There were parts of the implementation that randomly was presented with no explanation to the details / intuition for why that design choice was made (like the Fusion module, η\eta). While the paper designed proper baselines and substantial empirical results, the general flow of the paper's narrative seems cluttered and I don't believe it has reached the quality of an ICLR paper submission. The paper needs re-structuring starting from the Introduction to motivate the problem properly and spell out implementation specifics for a reader to easily digest to make it a publishable contribution.

Questions for authors
  1. Why was logits chosen for the averaging?
  2. What is η\eta (line 281, eq 13)? Why is it set to 2?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.