- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes WindowFold, which replaces sliding window inference in CLIP-based semantic segmentation with a single whole image pass. WindowFold learns small attention updates and spatial weights from a frozen sliding-window teacher using unlabeled images. Empirical results show lower inference cost with comparable accuracy in many settings
Critical strengths
- The paper examines how image resolution, object size, and spatial mixing affect the gap between whole-image and sliding-window inference. These experiments help explain why windows are useful and motivate the two proposed components.
- The evaluation covers multiple dense-CLIP methods, datasets, resolutions, and backbones, with ablations covering
Critical weaknesses
- My main concern is novelty. The central idea has already been presented in SPAR [1]. WindowFold introduces LoRA and learned Gaussian spatial weights, but the paper neither discusses nor compares with SPAR, making it difficult to assess what these additions contribute beyond the existing approach.
- I also find the presentation unclear and inconsistent in places. For example, “surgery” is used in the abstract without a clear explanation of what it refers to. The name “WindowFold” also refers to the LoRA-only variant in Table 4 but the full method with Gaussian spatial weights in Table 9. Although the captions explain this distinction, defining terms at first use and naming the variants consistently would make the method and results easier to follow. The author should consider a major polishing of this paper to make it more accessible.
- I’m not convinced that the paper has shown that LoRA and the Gaussian weights repair two separate failure modes. Table 1 shows that restricting spatial mixing helps the frozen model, but it does not test the claim that LoRA cannot learn such a correction. The paper then uses the roughly additive gains in Table 8 as evidence for the two mechanisms. Two partial corrections to the same problem could also work better when combined. It would help to measure how LoRA alone changes the spatial range of attention before assigning each component a separate role.
- The claim that WindowFold beats sliding-window inference in all 12 settings under the published protocol needs to be qualified. Table 4 reports LoRA alone and uses a short side of 336 pixels for every dataset, including Cityscapes. However, the published Cityscapes setting is 560 pixels, as the paper itself notes in Appendix A. At that resolution, Table 9 shows that even the full method remains below sliding-window inference for SCLIP and ClearCLIP. The abstract and conclusion should make clear which model and evaluation settings the 12/12 result refers to.
- Section 5.6 overstates the average improvement from adding the spatial weights. Table 9 shows an average gain of 0.86 mIoU for ProxyCLIP and 0.73 for SC-CLIP, whereas the text claims gains of 4.3 and 4.4. Those larger numbers appear to refer to Cityscapes alone. This should be corrected because it gives a misleading impression of how much the spatial weights contribute across datasets. 0.86 and 0.73 improvements might just be noise, since no standard deviation reported.
[1] SPAR: Single-Pass Any-Resolution ViT for Open-vocabulary Segmentation. CVPR 2026.
Questions for authors
Refer to weaknesses.
3 comments
Follow-up (1/2): new evidence on W5 (variance) and W1 (SPAR + footprint)
W3 and W5 led to the two most informative new experiments in the revision. Since our first reply we have completed the two experiments your review asked for most directly. Both are on full validation sets under the paper's protocol, and both are now in the revised PDF (App. I, Tables 21 and 22, in blue).
W5. Is the footprint gain noise? It is not, and we now report where it comes from.
We retrained the LoRA-only and LoRA + footprint models with 3 seeds each for ProxyCLIP and SC-CLIP. For every seed we stored per-image TP/FP/FN, which lets us run a paired image-level bootstrap (B = 2000, same resampled images for both arms).
| mean ± std, 3 seeds | VOC20 | PC59 | Stuff | ADE | City@560 | 5-bench avg |
|---|---|---|---|---|---|---|
| ProxyCLIP, LoRA | 84.65±0.06 | 38.14±0.02 | 26.62±0.03 | 18.85±0.07 | 30.99±0.22 | 39.85±0.03 |
| ProxyCLIP, + footprint | 84.56±0.25 | 38.05±0.06 | 26.40±0.06 | 18.79±0.04 | 35.30±0.13 | 40.62±0.08 |
| Δ | −0.09 | −0.10 | −0.21 | −0.06 | +4.31±0.33 | +0.77±0.11 |
| SC-CLIP, LoRA | 86.20±0.17 | 41.14±0.07 | 27.84±0.06 | 19.99±0.02 | 34.43±0.19 | 41.92±0.04 |
| SC-CLIP, + footprint | 85.66±0.10 | 41.01±0.07 | 27.80±0.07 | 20.12±0.02 | 38.52±0.12 | 42.62±0.06 |
| Δ | −0.53 | −0.13 | −0.04 | +0.13 | +4.09±0.28 | +0.70±0.07 |
- The five-benchmark gain is +0.77 [+0.59, +0.95] for ProxyCLIP and +0.70 [+0.56, +0.84] for SC-CLIP (paired bootstrap 95% CI, pooled over seeds, p(Δ ≤ 0) < 0.001 for both). It is 7–10× the seed-to-seed standard deviation. The single-seed values in the paper (+0.86 / +0.73) sit slightly above the 3-seed means.
- The gain comes from Cityscapes@560. For every one of the 6 model pairs, its 95% CI lies inside [+3.2, +5.3].
- On the four 336-pixel benchmarks the footprint is neutral to slightly negative: −0.12 [−0.24, +0.01] for ProxyCLIP and −0.14 [−0.26, −0.04] for SC-CLIP. Two cells show a consistent cost: SC-CLIP VOC20 (−0.5) and ProxyCLIP COCO-Stuff (−0.2).
This answers your W5 point more sharply than our first reply did. An average over five benchmarks is the wrong summary of the footprint. It is a range correction. It does almost nothing when the test grid is close to the teacher's window, and it gives about +4 mIoU on the wide, high-resolution benchmark where the range problem appears. This matches the attention-range measurement in Table 20. Section 5.6 now gives the 3-seed means and intervals next to the single-seed numbers, and states that the gain is concentrated on Cityscapes with a small cost at 336.
W1. Is the footprint just SPAR plus a trick? We added it on top of a SPAR-style student.
Our first reply compared the SPAR-style student (last 2 blocks fine-tuned, 14.78M parameters) with our method. A fair objection is that the footprint might help only weak students. We therefore trained SPAR-style + footprint on ClearCLIP with the same teacher, corpus and 12k steps (Table 21):
| ClearCLIP B/16 | VOC20 | PC59 | Stuff | ADE@336 | ADE@448 | ADE@672 | City@560 | City@672 |
|---|---|---|---|---|---|---|---|---|
| SPAR-style (last 2 blocks) | 85.32 | 38.28 | 26.25 | 17.69 | 17.87 | 16.58 | 27.32 | 27.45 |
| SPAR-style + footprint | 85.48 | 38.14 | 26.07 | 17.61 | 17.89 | 16.69 | 28.30 | 29.10 |
| Δ | +0.16 | −0.14 | −0.18 | −0.08 | +0.02 | +0.11 | +0.98 | +1.65 |
The footprint still adds accuracy on top of SPAR-style fine-tuning. The gain grows with the token count (+0.98 at 560, +1.65 at 672) and is near zero at 336. So the footprint is complementary to SPAR. It can be added to SPAR's student at the cost of 2–6 scalars.
The gain is smaller than on the LoRA student (+4), which suggests the fine-tuned blocks already correct part of the range; that it still grows with the grid suggests they do not bound it. We did not measure the attention range of the SPAR-style student directly, so this reading is indirect. This run uses a single seed, and the SPAR-style baseline was evaluated before we added per-image dumps, so this table has no confidence interval. We say so in the appendix.
Follow-up (2/2): contribution beyond SPAR, W4, and summary of changes
What the paper now contributes beyond SPAR. In response to your Informativeness score, we state the takeaway plainly. Sliding windows do two separate things for dense CLIP: they correct the features, and they cap how far attention reaches. A single-pass student has to reproduce both. Our evidence that these are two different things:
- A learned attention correction only narrows the range. Table 20 shows the LoRA's range still grows from 9.7 to 12.4 patches with grid size.
- An explicit footprint holds the range fixed at 7.5–7.8 patches, close to the teacher's 6.9.
- The footprint adds accuracy on top of either student (LoRA: +4.1 to +4.3 on Cityscapes; SPAR-style: +1.0 to +1.7), and adds nothing where the range is already right (336).
We think this explains why single-pass students lose accuracy on large, wide inputs. It also suggests a cheap addition to SPAR-style students.
W4, for completeness. The qualified claim is now "12/12 for the LoRA alone at short side 336; 10/12 at 448". At the published Cityscapes@560 setting, the full method still trails the window ensemble for SCLIP (32.65 vs 32.92), ClearCLIP (32.14 vs 32.68), ProxyCLIP (35.29 vs 36.27) and SC-CLIP (38.55 vs 40.96). It is above the ensemble for NACLIP (36.04 vs 35.57). It does this at 1.5–2.4× fewer FLOPs. The abstract, introduction, Sec. 5.1, conclusion and the Limitations section say this explicitly.
Summary of changes for your points.
| Your point | Where | Status |
|---|---|---|
| W1 SPAR | Sec. 2; App. I, Table 19 (SPAR-style); new Table 21 (SPAR-style + footprint) | discussed, compared, shown complementary |
| W2 terminology / polish | Sec. 1 definitions; Sec. 3–4 rewritten (blue) | done |
| W3 LoRA vs. footprint roles | App. I, Table 20; claims softened in Abstract, Sec. 1, 3.5, 4.3 | measured as you suggested |
| W4 12/12 | Abstract, Sec. 1, 5.1, Conclusion, Table 4 caption | qualified |
| W5 averages / noise | Sec. 5.6; new Table 22 (3 seeds + bootstrap CIs) | corrected, significance shown |
Each of your five points led to a change in the paper. Some of these changes cut against our original claims: the 12/12 result is now qualified, and the footprint is shown to be neutral or slightly negative at 336. If the revision and the new evidence address your concerns, we would be grateful if you would consider updating your assessment. We are happy to run any further check before the discussion closes.
Response to Reviewer K5P7
W1. Novelty and relation to SPAR (CVPR 2026). Thank you for the pointer. SPAR is now cited and discussed in Section 2. Both methods distill a sliding-window teacher into one pass. They differ in what is adapted:
We trained a SPAR-style control on ClearCLIP with our teacher, corpus and 12k steps (Table 19):
The SPAR-style student is better on the three object-centric benchmarks at 336, and we say so in the Limitations. As the token count grows it falls behind: −4.8 on Cityscapes@560 and −6.4 at 672, and below the LoRA alone on ADE20K@672. Fine-tuning the last blocks under global attention does not bound how far attention spreads on a larger grid. That is the failure the footprint targets.
W2. Terminology and presentation. "Surgery" and "WindowFold" are now defined once at the end of the first paragraph of the Introduction. Ablations always say "LoRA only" or "footprint only". Following your request for a major polish, Sections 3 and 4 are rewritten in full (C10, in blue).
W3. Evidence that the components repair separable failures. We ran the measurement you suggested: how far ClearCLIP's last-block attention reaches (RMS distance in patches, 200 ADE20K images) for each variant, all trained with the same teacher, 12k steps and multi-resolution grid (new Table 20 in App. I).
You were right to ask: the LoRA alone does contract the range, by 42% at 42×42. It does not hold it fixed, though. Its range still grows from 9.7 to 12.4 patches as the grid grows, while the teacher's range is fixed by the crop. With the footprint the range stays at 7.5–7.8 patches, close to the teacher. We have therefore softened the claim in the Introduction to "a LoRA alone narrows the range but does not hold it fixed". The two components overlap partially, and the footprint supplies the part that does not depend on resolution. This reading is consistent with three other results:
(a) Near-additivity (Table 8): on SC-CLIP, Cityscapes@672, footprint alone +7.9, LoRA alone +12.8, both +18.1. (b) Selectivity: on ClearCLIP, whose single site already sits near σ ≈ 7, the footprint is close to inert; on SC-CLIP, which mixes globally, it closes most of the Cityscapes gap. (c) Budget-matched control (new Table 18): footprint only cannot reach the full method even at 3× the steps, so the LoRA supplies something the footprint cannot.
W4. "12/12 settings". You are right that it was over-stated. It holds for the LoRA alone at short side 336 (Table 4); at 448 it is 10/12, with Cityscapes as the exceptions. The abstract, introduction, Section 5.1, conclusion and Table 4 caption now say this.
W5. Section 5.6 gains. This was a drafting error, thank you. The corrected text reads:
On variance: across three σ initializations and two seeds, the SC-CLIP five-benchmark average stays within 0.07 (Table 13). This is a small number of runs, so we read it as indicative. As expected for a range correction, the gain is concentrated on the high-resolution, wide-scene benchmark, and Section 5.6 now says so.
We would be glad to answer further questions before the discussion closes.