- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies continual learning with multiple weak labeling sources that may share errors or become unreliable over time. The authors propose TraceDuo, which tracks source reliability and controls source-specific influence during model updates. At each learning window, the method chooses between a full, partial, or rejected update using fresh labeled data for certification. The paper also provides theoretical results for the aggregation and certification procedures.
Critical strengths
- This paper considers a realistic weak-supervision setting, where different labeling sources may share errors and become unreliable over time.
Critical weaknesses
- The theoretical guarantees have limited relevance to the main empirical setting. In particular, the pseudo-label error bound becomes trivial under the CIFAR-100 setting, so it is unclear how much the theory actually explains the reported robustness.
- The method relies on several strong practical assumptions, including known source provenance and fresh labeled certification data at each learning window.
Questions for authors
- How are the provenance clusters constructed in practice? What information is used to determine whether different weak sources share the same provenance and how reliable is this information in realistic settings?
- How much does TraceDuo rely on fresh trusted labels at each learning window? Clean labels can be expensive to obtain, so this requirement may limit the practical use of the method.
- Since the method combines many different components, which component of TraceDuo should be considered the main methodological contribution?