- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies unsupervised temporal partonomy induction: discovering part–whole relationships between fine-grained actions and higher-level events in videos without temporal annotations. The authors propose RePart, which uses current-level representations and prediction residuals to compute Bayesian composition evidence for merging adjacent temporal units. The same predictor is then reapplied to the resulting coarser sequence, recursively building a variable-depth hierarchy. On Breakfast-Part, 50Salads-Part, and Assembly101-Part, RePart achieves the highest hierarchical F1 (hF1) among the unsupervised methods compared. Action anticipation experiments on Assembly101 further show that the induced partonomies provide useful context for downstream prediction.
Critical strengths
1. The paper goes beyond detecting action boundaries to study how fine-grained actions compose into higher-level events. This part–whole structure is useful for understanding how activities unfold in long videos, giving the work a clear and practically relevant goal.
2. The method is well motivated, fits the problem, and is easy to follow. The experiments cover comparisons with multiple baselines across different datasets, and the appendix includes additional hyperparameter sensitivity analyses.
3. The paper is clearly organized and readable. Figure 1 provides an intuitive overview of RePart, making it easier to understand how predictive contextualization and recursive composition work together.
Critical weaknesses
1. Some training components provide only modest gains in hierarchical performance. In Table 2, using only the prediction loss already achieves an hF1 of 31.66, compared with 32.02 for full RePart. In Table 3, the variant without predictive adaptation still achieves an hF1 of 31.76.
2. RePart requires 25 epochs of sequence-specific unsupervised adaptation for each video, which could become costly for long videos or large video collections. Reporting processing times across different video lengths would help readers assess the method’s computational efficiency.
Questions for authors
1. The limitations discussion notes that hard recursive composition is irreversible: once adjacent temporal units are merged, they cannot be split at later levels, so early grouping errors can propagate through the hierarchy. Do the authors have plans to mitigate this issue, for example through recoverable composition?
2. On Breakfast-Part and 50Salads-Part, RePart performs worse than PARSE on fine-grained segmentation. This does not undermine its value for temporal partonomy induction, but it suggests a trade-off between hierarchical agreement and fine-scale boundary accuracy. Further analysis of this relationship would be helpful.
3. Appendix C examines sensitivity to the objective weights and coherence temperature, but not to the composition threshold , which directly determines whether adjacent temporal units are merged. Please include a sensitivity analysis showing how performance changes with this threshold.
Thanks for the extra results. They help answer some of my questions. I'm still a bit concerned about how sensitive the method is to the threshold, and the adaptation seems quite expensive for a fairly small gain. I'm also not fully convinced by the hierarchy depth results. Still, I think the main idea is interesting, so I'll keep my rating.