acceptodds
Under review as a conference paper at ICLR 2027

TraceDuo: Provenance-Aware Continual Learning Beyond Label Aggregation

Abstract

Continual learning from weak supervision must account for shared source errors, reliability drift, and partial corruption. TraceDuo combines temporal sourcecluster label estimation, an exactly conserving source-deletion gradient decomposition, component-wise gradient clipping, and certified update routing. Full and adapter-only candidates are frozen before protected-risk and current-domain utility bounds govern commitment. Our analysis characterizes the independentcluster support and score-separation regimes in which the pseudo-label bound becomes informative, and gives local alignment conditions for beneficial deletionbased component control. Under fresh conditional IID sampling, the V2 certificate bounds the probability of any invalid commitment by 0.05 per stream and limits net risk increase relative to a fixed initial anchor to 0.02 on the simultaneous coverage event. Across the V1 experiments on CIFAR-100 and the binary Amazon, Elliptic, and TON tasks, domain final balanced accuracies are 98.10%, 99.20%, 98.80%, and 99.30%, respectively. Their macro average is 98.85%, compared with 97.55% for RobustAgg+DERpp. A common task–time definition gives 99.17% current-task accuracy and 0.40 percentage points of forgetting. Factorial routing and optimizer comparisons quantify component effects, while a separate numerical check tests no-clipping equivalence. Known-risk V2 certification experiments record 95 erroneous five-window trajectories out of 100,000 with joint correction, versus 11,236 without it, and quantify when 512 observations provide useful endpoint power

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Dependent Weak Supervision, Source Provenance, Dependency-Aware Label Aggregation, Post-Aggregation Updates, Leave-One-Component-Out Residuals, Component-Wise Gradient Clipping, Selective Model Updates, Risk-Controlled Update Acceptance

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_fZjwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies continual learning with multiple weak labeling sources that may share errors or become unreliable over time. The authors propose TraceDuo, which tracks source reliability and controls source-specific influence during model updates. At each learning window, the method chooses between a full, partial, or rejected update using fresh labeled data for certification. The paper also provides theoretical results for the aggregation and certification procedures.

Critical strengths
  1. This paper considers a realistic weak-supervision setting, where different labeling sources may share errors and become unreliable over time.
Critical weaknesses
  1. The theoretical guarantees have limited relevance to the main empirical setting. In particular, the pseudo-label error bound becomes trivial under the CIFAR-100 setting, so it is unclear how much the theory actually explains the reported robustness.
  2. The method relies on several strong practical assumptions, including known source provenance and fresh labeled certification data at each learning window.
Questions for authors
  1. How are the provenance clusters constructed in practice? What information is used to determine whether different weak sources share the same provenance and how reliable is this information in realistic settings?
  2. How much does TraceDuo rely on fresh trusted labels at each learning window? Clean labels can be expensive to obtain, so this requirement may limit the practical use of the method.
  3. Since the method combines many different components, which component of TraceDuo should be considered the main methodological contribution?
Reviewer_sJy7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates continual learning from weak supervision, where weak sources may share error, change their reliability over time, and be partially corrupted. The authors propose TraceDuo, a method consisting of four components: 1) temporal label estimation over a declared partition of sourced into dependency clusters; 2) a source-deletion gradient decomposition; 3) component-wise gradient clipping; and 4) a certified routing step. The authors also provide a conditional label-error bound, and a guarantee on invalid commitments and net risk increase relative to the initial model. The experiments on CIFAR-100 and three binary tasks show an advantage compared to the strongest baseline.

Critical strengths
  1. Relevant problem: dependent and drifting weak sources are a realistic issue in practical pipelines, and combining weak supervision with continual learning is an interesting direction.

  2. Explicit decision rule: separating candidate training from certification, freezing candidates before the certification labels are used, and allowing a 'reject' option make each update inspectable.

  3. Clear scope of the guarantee: the paper is clear in stating what Theorem 1 does and does not cover.

Critical weaknesses
  1. Missing essential experimental details:the backbone architecture is never specified, datasets and baselines are not described nor cited, it's unclear how weak sources/drifts/corruptions are generated. Furthermore, the link to the anonymous repo doesn't work.

  2. Results are difficult to interpret: the final balanced accuracy on CIFAR100 seems to be very high for a continual learning setting with weak supervision.

  3. Continual learning setting: from the description in Section 6, it seems that every task contains all classed of the domain. Furthermore, the reported forgetting is very small for all methods. It is therefore unclear whether the setting poses a real continual learning challenge. Could you please explain where does the shift come from?

  4. Many design choice without justification: the method relies on many fixed hyperparameters without a clear justification. Could you please describe how these values were selected and whether the method is sensitive to changes in such values?

  5. Presentation: the paper is very difficult to follow. Many terms are introduced but never really explained. The main problem comes from the description of the method and its V1/V2 variants. I couldn't find a detailed description of V1 (which is the method used for the main results), but only one-sentence descriptions (e.g., "V1 compares candidates with the incoming model"; "V1 uses local-only protection;"). The abstract contains too many numerical values. I would suggest substantially restructuring the method and its main motivation in the main text, describing thoroughly the experimental setup, and moving some details to the appendix if space is limited.

Overall, I vote for rejecting the paper. While the problem is relevant, the quality of the paper is below the acceptance bar. The poor presentation and the omission of many experimental and methodological details make the contribution difficult to assess. Addressing these concerns would require substantial changes to the paper that go beyond the scope of a rebuttal.

Questions for authors

See above.

Reviewer_7SD6OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies continual learning with weak supervision, where label sources may share errors, change over time, or become corrupted. The authors propose TraceDuo, a framework that estimates labels from multiple sources and controls their influence on model updates. The method trains full and partial update candidates and uses risk and performance checks to decide whether to accept an update. The paper also provides theoretical analyses of label estimation, gradient updates, and risk certification. Experiments on four datasets report improved accuracy and reduced forgetting compared with the evaluated baselines.

Critical strengths
  1. The paper addresses an important problem in continual learning with unreliable weak supervision.

  2. The paper provides many experiments and analyses to evaluate the proposed approach.

Critical weaknesses
  1. The presentation is poor, with inconsistent PDF and OpenReview titles and unexplained labels and diagnostic symbols in Figure 1. The abstract mainly lists components and results without clearly explaining the research gap or why the proposed method is needed.

  2. The contribution beyond existing source-aware and drift-adaptive methods is not established. For example, source-aware influence functions [1] and drift-adaptive weak supervision [2].

[1] Zhang et al. Understanding Programmatic Weak Supervision via Source-aware Influence Function. NeurIPS 2022.

[2] Mazzetto et al. An Adaptive Method for Weak Supervision with Drifting Data. AISTATS 2025.

  1. The experiments do not specify the actual weak supervisors, distribution shifts, corruption process, backbone models, or protected populations.
Questions for authors

What do "CFIP-S" and the diagnostic symbols in Figure 1 represent, and how do they relate to the proposed method?

Others are the same to weaknesses.

Reviewer_sjUFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

TraceDuo is a continual learning approach for weakly supervised settings where labeling sources could share errors. These sources are grouped into clusters based on provenance metadata (e.g., same checkpoints), and labels are estimated by a reliability-based aggregation across clusters that adapts to drift. Specifically, during training, the approach removes each source group to measure how the pseudo-label changes and these changes are used to split the gradient into components to be clipped separately. The method is tested on CIFAR-100 and Amazon, Elliptic, and TON where it achieves an average final balanced accuracy of 98.85%, compared to 97.55% in prior work, and reduces average forgetting from 0.95 to 0.40 percentage points.

Critical strengths
  • Correlated weak sources in drifting streams are common in practice so the setting does have real world applicability.
  • The method obtains consistent improvements across all four domains and the transfer setups.
Critical weaknesses
  • Paper writing feels very abstract and is really hard to follow. There's no narrative, no motivating example, no intuition, and a big part of paper reads like facts.

  • Experimental setup is unclear and critical details are missing. For instance, how were the weak sources & corruptions generated? It’s also unclear how the approach yields results so high: 98.1% CIFAR-100 and 99.2% Amazon.

  • In the experiments, TraceDuo uses 1664 clean labels per window & 6K weak labels. Results of supervised baselines on these clean labels are missing.

  • Related work section is too brief and does not explain how TraceDuo differs from similar methods.

Questions for authors
  • Some baseline methods and ablation variants share candidate masks and certification routing. Do baselines get the same clean labels, partial candidates, and certification routing?
  • The main experiments use a local-only certification rule, while the guarantee highlighted in the abstract is only tested on synthetic classifiers. Why not apply it to the trained models?
  • What would happen when dependent sources are incorrectly placed in separate groups?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.