- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FoRIS, a training-free framework for ICS that reformulates reference-query matching as a progressive foreground refinement process. The method consists of three components that foreground purification suppresses background interference and positional bias, foreground localization identifies reliable foreground regions through feature correspondence and multi-cue clustering, and foreground consolidation refines the segmentation results through semantic disagreement penalization and cluster-level reweighting. Experiments across multiple benchmarks demonstrate consistent improvements over existing training-free approaches.
Critical strengths
- The paper is well organized, and the motivation is clear.
- FoRIS achieves consistent improvements over existing methods across diverse datasets.
- The ablation studies are relatively comprehensive.
Critical weaknesses
- Although the method reformulates feature matching as a three-stage progressive refinement process, the unique contributions of the individual components could be further clarified, as several design choices appear closely related to those in INSID3.
- The response maps in Eqs. (13) and (17) are simply added together. Could the authors explain why this works well and whether different weights would affect the results?
- FoRIS achieves promising improvements but introduces considerable computational overhead compared with INSID3. Mybe providing a more detailed discussion of the trade-off between mIoU and inference efficiency would be better.
Questions for authors
Please see the questions raised in the Weaknesses section.
5 comments
Part 2/3
Q 2.
We thank the reviewer for pointing out the potential scale mismatch among the intermediate response maps and the importance of validating the additive fusion coefficients. To address this, we clarify that our framework employs weighted additive fusion rather than simple unweighted addition. For simplicity, the original manuscript omitted the explicit weighting coefficients and their default values. Following the reviewer's suggestion, we have now included them in Eqs. (13) and (17) of the revised manuscript:
The additive formulation is motivated by the fact that all response maps share the same target patch grid and provide spatially aligned complementary evidence. Specifically, provides the primary semantic response, while the auxiliary maps progressively refine foreground predictions through geometric localization and regional semantic consistency. Additive fusion integrates these complementary signals while preserving the original semantic response, without introducing additional learnable fusion modules. Moreover, the explicit weighting coefficients control the contributions of different auxiliary signals and prevent them from overwhelming the primary response. The default settings are and To further investigate the impact of different weights, we conduct systematic weight sensitivity experiments on COCO, ISIC, and Lung. Specifically, we independently vary each coefficient in Eqs. (13) and (17), while keeping the remaining coefficients fixed at their default values. The corresponding experimental results and analyses have been added to the revised manuscript.
Table 1. Sensitivity to the localization fusion weights
| COCO | ISIC | Lung | ||
|---|---|---|---|---|
| 0.00 | 0.25 | 60.20 | 63.80 | 85.40 |
| 0.20 | 0.25 | 60.90 | 62.90 | 87.60 |
| 0.50 | 0.25 | 56.85 | 58.60 | 88.50 |
| 1.00 | 0.25 | 49.40 | 53.10 | 88.00 |
| 0.20 | 0.00 | 59.10 | 61.30 | 88.20 |
| 0.20 | 0.50 | 60.85 | 61.90 | 85.60 |
| 0.20 | 1.00 | 59.60 | 54.10 | 80.90 |
Table 2. Sensitivity to the consolidation fusion weights
| COCO | ISIC | Lung | ||
|---|---|---|---|---|
| 0.0 | 0.0 | 58.45 | 60.20 | 86.20 |
| 0.0 | 1.0 | 60.17 | 62.60 | 87.10 |
| 0.5 | 1.0 | 60.65 | 62.90 | 87.40 |
| 1.0 | 1.0 | 60.90 | 62.90 | 87.60 |
| 2.0 | 1.0 | 61.12 | 62.60 | 87.80 |
| 1.0 | 0.0 | 59.90 | 61.10 | 87.50 |
| 1.0 | 0.5 | 60.60 | 62.30 | 87.80 |
| 1.0 | 2.0 | 60.83 | 63.60 | 86.50 |
Part 3/3
Q3: To provide a more comprehensive assessment, we conduct additional experiments on COCO-20$^i$ using a single RTX 3090, including a component-wise runtime analysis and a resolution-wise evaluation of the accuracy-efficiency trade-off. These experiments aim to identify the main sources of computational overhead and explore more efficient inference configurations.
(1) Accuracy-efficiency comparison with INSID3.
At the default input resolution of 1024 px, following the original setting of INSID3, FoRIS achieves 60.9% mIoU, outperforming INSID3 (57.6%) by 3.3 percentage points. Meanwhile, FoRIS requires 1,620.77 ms per in-context example, approximately 2.0$\times$ the 812 ms required by INSID3. To identify the sources of this additional overhead, we provide a component-wise runtime breakdown in Table below. Among the three refinement stages, Foreground Localization (346.44 ms) and Foreground Consolidation (342.50 ms) account for the largest computational costs, whereas Foreground Purification introduces only 38.74 ms of additional computation. The optional CRF refinement further contributes 190.82 ms. This analysis quantifies the computational cost of each component and identifies the main sources of inference overhead.
(2) Resolution-dependent accuracy-efficiency trade-off.
We further investigate the effect of input resolution on segmentation performance and inference efficiency to explore more efficient inference configurations. As shown in Table below, FoRIS achieves 59.5% mIoU at 512 px with an inference time of only 258 ms. Increasing the resolution to 720 px, 896 px, and 1024 px improves mIoU to 60.3%, 60.7%, and 60.9%, with corresponding runtimes of 593 ms, 1,108 ms, and 1,620.77 ms, respectively.
Notably, further increasing the resolution does not yield commensurate performance gains. For example, at 1440 px and 1600 px, the runtime increases to 5.47 s and 9.02 s, respectively, without improving mIoU. Although the 1760 px setting achieves a marginally higher mIoU of 61.0%, it requires 11.24 s per example, approximately 6.9$\times$ the runtime at 1024 px, for only a 0.1-point improvement. These results indicate that simply increasing the input resolution is computationally inefficient and provides limited benefits to segmentation performance.
Overall, FoRIS provides a flexible accuracy-efficiency trade-off through input resolution selection. We retain 1024 px as the default input resolution in our main experiments, following the original setting of INSID3 to ensure a consistent evaluation protocol and fair comparison. Meanwhile, our resolution-wise experiments demonstrate that lower resolutions can substantially reduce inference time while maintaining competitive segmentation performance, offering practical alternatives for efficiency-constrained applications.
Table: Detailed computational analysis of FoRIS on COCO-20$^i$.
We report the inference time of each component in milliseconds (ms, ) for a single in-context example. Runtime is measured at a resolution of 1024 px on a single RTX 3090.
| Component | Runtime |
|---|---|
| Encoder forward | 702.27 ms |
| Foreground purification | 38.74 ms |
| Foreground localization | 346.44 ms |
| Foreground consolidation | 342.50 ms |
| CRF refinement | 190.82 ms |
| Total inference time | 1620.77 ms |
| INSID3 | 812 ms |
Table: Resolution-wise computational analysis of FoRIS on COCO-20$^i$.
We report the inference time and mIoU of FoRIS under different input resolutions. Runtime is measured for a single in-context example on a single RTX 3090. The 1024 px setting follows the default resolution of INSID3 and is used in the component-wise analysis.
| Metric | 512 px | 720 px | 896 px | 1024 px | 1440 px | 1600 px | 1760 px |
|---|---|---|---|---|---|---|---|
| Runtime | 258 ms | 593 ms | 1108 ms | 1621 ms | 5468 ms | 9018 ms | 11235 ms |
| mIoU (%) | 59.5 | 60.3 | 60.7 | 60.9 | 60.8 | 60.1 | 61.0 |
Part 1/3
We sincerely thank you for your professional and insightful review. Our responses are provided below:
Q1:
As shown in the table above, the key distinction lies not in the shared encoder or positional basis, but in how visual evidence is processed and integrated. Specifically, FoRIS introduces three complementary improvements:
Foreground Purification (FP): APD adaptively removes positional bias, while explicit foreground-background contrast and feature gating improve the reliability of foreground representations.
Foreground Localization (FL): Instead of relying on the hard intersection of forward and backward correspondences, FoRIS integrates soft voting and clustering-based spatial evidence to construct a more robust localization response.
Foreground Consolidation (FC): FoRIS explicitly models disagreements among different sources of evidence and performs cluster-wise semantic refinement, progressively correcting the intermediate response before the final prediction.
Therefore, FoRIS differs from INSID3 not merely by adding individual operations, but by introducing a unified progressive refinement mechanism that changes both the role of intermediate evidence and the way the final segmentation mask is generated.