- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a pipeline for identity-aware video captioning and person-centric question answering, where known character identities and locations are supplied to a Video-MLLM through explicit spatial cues. The authors construct an LSMDC-derived benchmark of 750 manually verified shots and 3,000 questions, compare five identity-grounding representations within the Qwen3-VL family at three model scales, and introduce BAC through caption-only LoRA fine-tuning. Cross-family and frontier-model comparisons use face visual boxes combined with textual coordinates. Evaluation covers captioning and identity-targeted QA. The reported results show benefits from visual grounding over coordinates alone and further improvements from task-specific adaptation on the constructed benchmark.
Critical strengths
- A practically relevant task formulation and evaluation resource. Associating descriptions and actions with the correct characters is important for useful movie understanding. The benchmark evaluates character attribution directly and includes people who cannot be assigned a supplied identity. Manual verification strengthens its usefulness as an evaluation resource. This is a substantive contribution because correct character attribution can be obscured by conventional caption-quality metrics.
- Meaningful benefits from lightweight adaptation, including transfer between tasks. BAC is trained on captions but improves person-centric QA across three model sizes. This provides evidence that the benefit extends beyond reproducing the directly supervised output task. The matched-backbone comparisons support the practical value of adaptation for compact models, even though the mechanism and broader transfer of the improvement remain insufficiently characterized.
Critical weaknesses
- The evaluation does not sufficiently distinguish temporal identity association from per-frame recognition and adaptation to the annotation protocol. Identity-targeted QA measures attribution accuracy, but higher aggregate accuracy does not establish that the model must associate identities across time. The benchmark consists of short, single-shot videos, and questions about appearance, posture, position, and some actions may be answerable from one informative frame. Short duration alone does not establish simplicity; however, without a single-frame comparison, the contribution of temporal information remains unclear. The category-wise analysis in Appendix D does not resolve this issue, since an “action” or “interaction” question is not necessarily temporally dependent. More diagnostic evidence would evaluate questions requiring identity information from a different frame, particularly when the queried person lacks an annotation in the frame containing the relevant action. Caption-to-QA transfer is encouraging, but both tasks share the grounding convention, so this transfer does not rule out adaptation to that convention as an explanation for some gains. A matched identity-agnostic caption adaptation control, alternative input representations, and consistently remapped identity labels would help establish what capability BAC acquires beyond more reliable use of supplied spatial cues.
- The benchmark informs method selection and is subsequently reused for final evaluation, leaving the selected recipe without independent confirmation. The same benchmark is used to compare grounding representations and identify the strongest backbone. The selected representation and backbone then underpin BAC, which is fine-tuned on separate training data and evaluated on that benchmark. Consequently, the final evaluation is not independent of the decisions used to construct the method. Selecting configurations according to their performance on a finite evaluation sample can favor configurations that benefit from sample-specific characteristics; evaluating the resulting method on the same sample cannot establish whether that advantage generalizes. This concern is distinct from training directly on test examples. The separate validation split used for LoRA hyperparameters and checkpoint selection does not address the earlier grounding and backbone decisions. The reported within-benchmark improvements remain meaningful, but they provide weaker evidence for the general superiority of the selected recipe. Selecting these components on development data, freezing the choices, and evaluating on an untouched test set would provide stronger confirmation. The cross-group comparison compounds this limitation. The five representations are systematically compared only within Qwen3-VL, while other groups receive the selected FVB+FTC configuration. Applying the same configuration to every model does not establish that it is equally suitable for each group. Thus, the rankings are conditional on a representation selected based on Qwen, and the abstract suggests broader validation of the grounding strategies than the experiments provide. Independent data or annotation-protocol validation is therefore particularly important. Suitable adaptations should preserve comparable identity and spatial information, and dataset overlap should be checked explicitly. This also matters for novelty: visual region labels and learning to interpret them have precedents [1], [2]. Combining established components can be valuable, but the paper needs stronger evidence of what new capability or broadly applicable empirical insight its known-identity formulation contributes.
- The presentation does not clearly delineate the framework and the responsibilities of its components. The constituent methods are described, but the overall pipeline and its boundaries are difficult to follow. Figure 1 illustrates a task example rather than the complete framework. An integrated diagram should distinguish offline character identification and tracking, synthetic annotation and benchmark construction, BAC adaptation, and inference-time inputs and outputs. It should make explicit which identity correspondences are supplied by preprocessing and which associations BAC must infer. Clear input specifications and promptexamples would also help readers understand the role of FVB+FTC during training and evaluation. This is a presentation concern but it affects the assessment of the model’s claimed contribution.
Questions for authors
- Clarification: What proportion of questions genuinely requires information from multiple frames? How are identity answers verified when the queried person is unannotated in the relevant frame but has an identity-linked annotation elsewhere in the shot?
- Additional evidence for the current capability claim: How do BAC and its base model compare under full-video and single-frame inputs, particularly on questions requiring cross-frame identity association?
- Additional evidence concerning the source of improvement: Does BAC retain its advantage under alternative grounding representations or consistently remapped identity labels? Can a matched identity-agnostic caption adaptation control help separate identity-linked learning from general task and format adaptation?
- Additional evidence concerning generality: Does the grounding-strategy finding hold in another model family, or does BAC retain its gains on an independent evaluation protocol with suitable input adaptation and verified absence of training-data overlap? Either result would materially strengthen the scope of the contribution.