- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies video classification under multi-instance partial-label supervision, where each video is associated with a candidate label set containing one true class, without temporal annotations. The proposed PIVOTMIPL constructs a class–prototype–time assignment using contextualized video features and occupancy-regularized spherical matching. During training, candidate-restricted inference produces a proposal, whose class marginal is refined through momentum updates. A dual-marginal KL projection constructs a teacher that combines these refined class beliefs with the proposal’s temporal occupancy. The full-space predictor is trained through a plan-level KL objective.
The paper analyzes the relationship between candidate re-solving and masking, the concentration properties of occupancy regularization, and the decomposition of the joint objective. Validation covers three constructed video benchmarks with four feature representations, seven MIPL baselines, component ablations, an encoder-matched comparison, and efficiency analyses. The reported results achieve the highest mean ACC and BACC in 10 of 12 configurations.
Critical strengths
-
Coherent structured supervision mechanism. The candidate-restricted proposal, momentum-refined class beliefs, and dual-marginal teacher form a well-connected learning procedure. In particular, the teacher construction provides a concrete way to combine historical class information with the current temporal allocation. This is a technically meaningful contribution beyond simply attaching an additional temporal encoder to a bag-level classifier.
-
Careful analysis supported by relevant empirical controls. The paper precisely characterizes when re-solving differs from masking and establishes the class–time decomposition and current-point gradient equivalence of the objective. The appendix also appropriately distinguishes concentration control from correct localization. Empirically, the joint-loss and teacher-projection ablations show consistent directional improvements across the twelve tested single-fold configurations, while the encoder-matched comparison provides useful evidence that the encoder alone does not explain all reported gains.
Critical weaknesses
-
The candidate-generation protocol appears to permit indirect use of downstream test labels. Sections 4.1 and B.3 describe generating out-of-fold candidate labels using the same five-fold partition subsequently used for downstream evaluation. Under this procedure, when fold F1 is the downstream test set, candidates for training samples in F2 are generated by a recognizer trained on F1, F3, F4, and F5, including the exact labels of F1. Thus, being out-of-fold for each individual sample does not ensure isolation from the downstream test labels. The supplementary code loads precomputed candidates and reuses them across downstream folds, but does not include the candidate-generation pipeline needed to resolve this concern. If the protocol operates as described, the reported results do not establish performance under a strictly held-out evaluation. Sharing these candidates across methods does not eliminate the issue or establish that relative comparisons are unaffected.
-
The experiments do not sufficiently isolate the value of the central class–time mechanism over simpler alternatives. The classification results and ablations are encouraging, but the theoretical distinction between candidate re-solving and masking is not accompanied by a direct empirical comparison. A simple class-specific temporal aggregation model with a PLL objective and the same encoder is also missing. Moreover, the evidence for the proposed temporal-allocation explanation is indirect: improved ACC/BACC does not establish that the learned allocation better identifies supporting evidence or reduces reliance on background. This matters because improved class–time association is central to the motivation. The key ablations are limited to one fold, with some differences being small, leaving the reliability of those component-level conclusions uncertain.
Questions for authors
-
Protocol clarification: For each downstream test fold, were all candidate generators used to construct the downstream training candidates trained without any examples or exact labels from that test fold? Please provide the relevant generation code or explicit split mapping. If candidates were generated once using the shared five-fold procedure described in B.3, can you provide results with candidate generation nested entirely within each outer training split?
-
Evidence needed to assess the central mechanism: Do you have a controlled comparison between candidate-restricted re-solving and global masking followed by renormalization, with the remaining teacher construction and training procedure unchanged? How does the method compare with a simpler class-specific temporal aggregation model using the same encoder and a PLL objective?
-
Reliability and interpretation of the ablations: Are multi-fold or repeated-seed results available for A3 and A4? Is there direct evidence, such as temporal-annotation-based analysis or controlled snippet/background perturbations, that the joint objective improves the quality of supporting temporal evidence rather than only the final classification accuracy?
-
Configuration clarification: Table 5 specifies four prototypes per class for Breakfast, whereas all four supplied Breakfast scripts use
--num_prototypes 6. Which setting produced the main results and the full-model reference in the ablations?