- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Image LVMs have recently been used in gait recognition to independently encode each input frame before gait head aggregation. The paper analyzes the effect of replacing image LVMs with video LVMs, where the temporal structure is already present in the foundation representation. Experiments on frame-wise and clip-wise V-JEPA 2.1 show that encoding frames in context helps most where appearance is least reliable (cross-domain, persistent body part occlusions), but performs worse when visibility changes incoherently.
Critical strengths
- The paper raises a relevant, under-explored question in gait recognition research - whether (and when) video-native representations could be utilized to improve identification.
- The paper analyzes different scenarios such as in-domain vs. cross-domain, various types of occlusions, and how temporal context reshapes the information given to the gait recognition head.
- The findings are based on quantitative measurements and are also well-explained through controlled analyses.
- The paper's idea is simple yet creative and effective, and its conclusions can be useful for the gait recognition community in exploring new research directions and boosting performance.
Critical weaknesses
-
The comparison in Table 2 is insufficient to draw such broad conclusions about the effect of encoding mode on identification. The table shows a single checkpoint (V-JEPA 2.1-B), a single gait head (of BiggerGait), and only 4 benchmark results. To substantiate the paper claims, the matched comparison should be significantly extended to additional checkpoints/models that support both frame-wise and clip-wise execution (such as V-JEPA 2.1-L and Omnivore), additional gait heads, and gait benchmarks. In-the-wild gait benchmarks, such as Gait3D/GREW, would strengthen the paper claims about cross-domain transfer.
-
Figure 3(a) shows that clip-wise encoding drifts less than frame-wise encoding at visible positions and more at occluded positions. The paper suggests that context protects what remains rather than filling in the missing parts. However, it is demonstrated only for V-JEPA 2.1, which was trained by making predictions in a learned latent space, rather than directly in the input space. It is unclear whether masked reconstruction video LVMs (as mentioned in line 467) would exhibit the same behavior.
-
Minor weakness - related work section should be located after the introduction for better readability and flow of the paper.
Questions for authors
-
As the authors mention (line 141) - frame-wise encoding uses the native image pathway (2D patch embedding and image token) and clip-wise encoding uses the video pathway (3D two-frame tubelets), making the two pathways tokenize differently. That means that "Frame-wise" and "Clip-wise" in Table 2 differ in both the temporal context and the image/video pathway. Could the authors further isolate the contribution of temporal context by keeping the video tokenization fixed (within the video pathway) but disabling cross-tubelet interaction? It would help ablate the effect of each factor (temporal context/different pathways) on the gaps shown in Table 2.
-
Could the authors provide visual examples of gait-head input representations? It would help understand the difference between clip-wise and frame-wise encoding, and support the paper claims, especially in Findings 2 and 3. For example, gait feature visualizations such as activation/attention maps over time could be useful.
5 comments
I appreciate the authors’ rebuttal, including the additional experiments, visualizations, and explanations. After also considering the other reviews, I have two small follow-up points that I believe would help make the paper clearer:
-
Could the authors provide a few more details on the latency protocol in Table 2? In particular, which GPU is used, which parts of the pipeline are timed, and over how many runs is the median computed? For frame-wise encoding, are the 32 frames processed sequentially or batched together in a single GPU forward? Finally, is the reported latency measured per 32-frame chunk or for full sequences of different lengths?
-
Could the authors reproduce the Fig. 3(a) token-drift analysis for InternVideo-Next-B? Since InternVideo-Next-B uses a different pretraining recipe that also incorporates pixel-level reconstruction, although it is not purely reconstruction-based like VideoMAE, it would be interesting to see whether the visible/occluded token-drift trend generalizes beyond V-JEPA 2.1.
These are relatively minor points, and my main concerns from the initial review have been mostly addressed.
Thank you.
Follow-up with Reviewer GkCH
We thank Reviewer GkCH for the thoughtful follow-up and for considering the broader discussion to help sharpen our paper. Both updates are in the revised PDF (in blue).
1. Latency Protocol (§3.2; §4.1, Table 2; Appendix A.2)
Thank you for pointing out these helpful details for a fair runtime comparison:
- Hardware & precision: Single NVIDIA RTX 6000 Ada Generation GPU with FP16 autocast.
- Timed scope: Median end-to-end model-forward latency from GPU-resident RGB tensors to final gait embeddings (
frozen LVMmask branchHuman-Space CNNHPP & gait head), bracketed bytorch.cuda.synchronize(). Disk I/O, DataLoader/CPU preprocessing, host-to-device copies, gallery–probe retrieval, andDDPgather are excluded. - Runs & median: 3 global warm-ups, followed by 1 sequence warm-up and 3 timed runs per sequence (averaged per sequence), taking the median across all ~8,178 CCPG evaluation sequences.
- Batched
Frame-wiseexecution (not sequential): Frames within each chunk are folded into the batch dimension for parallel GPU execution: 1 batched forward of(32, 3, 1, H, W)per 32 frames onV-JEPA 2.1-B/L(frames_chunk_size=32), 2 batched forwards of(16, 3, 1, H, W)onInternVideo-Next-B(frames_chunk_size=16, matching itsClip-wisechunk size), and 1 batched forward onDINOv2-S/L(below its 96-frame cap). - Full sequences vs. 32-frame GFLOPs: While
Tokens/32fandGFLOPs/32fstandardize theoretical cost per 32 frames,Latency (ms)measures full variable-length CCPG sequences (batch_size = 1,all_ordered, capped at 250 frames), internally split into contiguous chunks (32frames onV-JEPA,16frames onInternVideo-Next) before gait-head pooling.
2. InternVideo-Next-B Token Drift (§6.2, Figure 3a; Appendix B.1, Table 8)
Thank you for this helpful suggestion. Running the same pre-gait-head cosine-drift analysis on InternVideo-Next-B yields results well aligned with expectations:
| Model | Region | Clip-wise | Frame-wise | Clip - Frame | Gap (%) |
|---|---|---|---|---|---|
V-JEPA 2.1-B | Still-visible | 0.0141 | 0.0169 | -0.0028 | -16.4% |
V-JEPA 2.1-B | Occluded | 0.3456 | 0.2743 | +0.0713 | +26.0% |
InternVideo-Next-B | Still-visible | 0.0563 | 0.0952 | -0.0389 | -40.9% |
InternVideo-Next-B | Occluded | 0.5156 | 0.6158 | -0.1002 | -16.3% |
- Consistent protection at still-visible positions:
Clip-wiseencoding reduces visible-token drift by-40.9%onInternVideo-Next-B(0.0563vs.0.0952; Figure 3a; Appendix B.1). As inV-JEPA 2.1-B, temporal context consistently stabilizes features on the body regions that remain visible. - Occluded-token drift reflects the pretraining objective: Unlike purely latent-predictive
V-JEPA 2.1-B(+26.0%),InternVideo-Next-Bincorporates pixel-level diffusion reconstruction in Stage-1 pretraining, which regularizes uniform mask patches soClip-wisedrifts-16.3%less thanFrame-wise(0.5156vs.0.6158). However, because persistent masks hide the same region across all frames, occluded tokens still change substantially (0.5156,9.2xthe visible-token drift of0.0563), and theClip-wisedrift reduction remains2.5xstronger on visible tokens (-40.9%vs.-16.3%)—indicating that context still acts mainly by stabilizing visible regions rather than recovering hidden content.
We sincerely thank you again for your thoughtful engagement across both rounds—your suggestions have greatly helped us clarify key experimental details, sharpen the scope of our claims, and make the overall study much more rigorous and complete. Please let us know if any further clarification would be helpful.
Response to Reviewer GkCH
We thank Reviewer GkCH for the supportive review and constructive experimental suggestions, all of which we have implemented in the revised manuscript.
W1: Extending the matched comparison to additional models, benchmarks, and gait heads
We extended the matched comparison to two additional checkpoints and full sub-condition breakdowns (full table in Global Response §1; §4.2, Table 3 and Appendix Table 6), while clarifying benchmark and gait-head scope:
V-JEPA 2.1-LandInternVideo-Next-Balongside the complete 8-condition cross-domain breakdown on SUSTech1K. All three checkpoints exhibit the exact same pattern: minimal change in-domain, alongside substantial cross-domain gains concentrated where appearance fails (+11.41% to +13.20% at night; +4.45% to +9.62% under clothing change).W2: Token drift in Figure 3(a): Latent prediction (V-JEPA 2.1) vs. masked pixel reconstruction LVMs
We address this distinction from three complementary angles:
[MASK]tokens, so even a pixel-reconstruction encoder has no visible views of that body part in neighboring frames to reconstruct its appearance. Moreover, when hidden parts do reappear across frames (SmoothandRandommasks, Figure 2c), theClip-wiseadvantage shrinks from +7.3% to +1.6% or reverses to -7.7%, ruling out cross-frame pixel completion.InternVideo-Next-B: Unlike V-JEPA 2.1,InternVideo-Next-Bincorporates pixel-level reconstruction via a conditional diffusion decoder during Stage-1 pretraining, yet shows the exact same robustness trend across all 9 persistent-occlusion conditions (Figure 2b, Appendix Figure 4) and unseen domains (Table 3, Appendix Table 6).V-JEPA 2.1-B, whose latent-prediction objective may respond more strongly at overwritten regions than pure pixel-reconstruction encoders such as VideoMAE. We have explicitly added this qualification in §6.2 and §7.W3: Placement of Related Work
Following your suggestion, we moved Related Work to Section 2, immediately after the Introduction, improving the narrative flow of the paper.
Q1: Isolating temporal context by keeping video tokenization fixed (
Tubelet-Localablation)Implemented in §6.1 (Table 5) and detailed in Global Response §2: keeping
V-JEPA 2.1-B's 2-frame 3D video tokenizer fixed while blocking cross-tubelet self-attention (Tubelet-Local) confirms that isolated 3D tubelets do not explain our gains (44.10% on CASIA-B* CL), whereas enabling full-clip cross-tubelet context brings +2.89% on CCPG CL, +3.59% on CCGR-MINI R1, +2.10% on SUSTech1K Overall, and +8.01% on CASIA-B* CL.Q2: Visual examples of gait-head input representations over time
Added in Appendix C (Figures 6–8) and summarized in Global Response §4: pre-gait-head spatial activation maps across four benchmarks (Figure 6), across four walking time steps (Figure 7), and under 40% persistent occlusion (Figure 8) visually illustrate Findings 1–3 by showing holistic, spatially uniform whole-body and lower-limb activations under
Clip-wiseencoding versus fragmented, localized activations underFrame-wiseencoding.We hope these new experiments, ablations, and visualizations thoroughly address your questions, and we welcome any further discussion.