- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies a template acquisition bottleneck in visual tracking. The initial template is accurately localized but becomes outdated, while later templates capture appearance changes but may contain localization errors. The authors use a current-frame oracle unavailable at test time to measure this gap. They then propose PATT, where a teacher uses accurate target crops from past, current, and future frames, while a student uses only historical templates. The student predicts the teacher’s multi-layer search features, with the training loss weighted by the teacher’s relative advantage and absolute IoU. Experiments show consistent gains on short- and long-term benchmarks.
Critical strengths
-
The paper clearly defines the inference-time template acquisition bottleneck and uses a non-deployable oracle to turn the intuition about template staleness and localization uncertainty into a measurable research question.
-
The experiments are broad in scope, covering seven benchmarks, two model scales, and multiple evaluation regimes.
Critical weaknesses
Table 2 shows that adding an EMA teacher and a representation prediction branch brings little improvement without accurate target crops from the current and future frames. This suggests that the gains mainly come from privileged appearance information. However, the tracking loss in Eq. (11) directly optimizes only the student, while the teacher is updated through an EMA of the student’s parameters. The paper does not report the teacher’s AUC, how often it outperforms the student, or the fraction of samples with nonzero reliability weights. It therefore remains unclear whether the teacher consistently outperforms the student when given privileged inputs, and how many training samples provide useful auxiliary supervision.
Questions for authors
-
Please report how the mean IoU difference and the fraction of samples with nonzero reliability weights change across training epochs.
-
Historical templates are cropped using ground-truth boxes during training. Are localization perturbations applied to the historical templates during training, beyond the jitter applied to the search crop? Do the teacher and student use the same template-cropping and augmentation procedures?
2 comments
Follow-up on our response
Dear Reviewer xxPU,
Thank you again for your careful review. As the discussion period ends soon, we would like to check whether our response addresses your questions, in particular the teacher statistics across training (Appendix G) and the template-cropping procedure (Appendix A). We would be glad to clarify anything further.
Thank you very much.
Response to Reviewer xxPU: teacher advantage and effective supervision during training, and template cropping
Thank you for the careful review. We agree with your reading of Table 2 that the gain mainly comes from privileged appearance, which is the paper's central claim. Your question is the right test of this claim, and we report the requested statistics below.
W and Q1. Teacher AUC, teacher advantage, and fraction of non-zero weights. With privileged current- and future-frame crops, the final EMA teacher reaches 88.3 LaSOT AUC, a non-deployable setting. With causal (past) templates, the same weights reach 75.4, identical to the student, so the teacher's advantage comes from its privileged inputs. Over training, we record the following statistics on all training samples of PATT-B256. Since utea>ustu≥0 implies utea>0, Eq. 10 gives w>0⟺utea>ustu.
We have added these statistics to Appendix G of the revised PDF.
Q2. Template perturbation and cropping procedures.
We hope these statistics resolve your concern, and we would appreciate your considering them in the final assessment.