- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the integration of parameter-efficient fine-tuning and token reduction for pixel-level tasks. The authors point out that existing TR-PEFT methods are state-agnostic to the two distinct token states induced by TR (preserved and skipped tokens). To tackle this issue, they propose TRACE, a state-aware design that leverages TR-induced token states to regulate feature compensation and adaptation. Experimental results demonstrate that TRACE consistently outperforms existing approaches across multiple dense task benchmarks.
Critical strengths
-
The problem analysis of existing TR-PEFT approaches is well-conducted; characterizing the behavioral anomalies of state-agnostic designs from three measurable perspectives provides valuable insights for advancing the TR-PEFT direction.
-
The proposed method is concisely designed. The core mechanism lies in explicitly propagating the pre-existing selection signals from the Token Selector to the compensation mask and the adaptation-compensation mixture (Eqs. 10–12). It introduces virtually no complicated paradigms while keeping both trainable parameters and FLOPs strictly low.
Critical weaknesses
-
While the paper claims to “achieve better performance with reduced computational overhead”, the actual improvements over the baseline (DAR) are marginal and inconsistent, mostly hovering in the 0.005–0.01 range. Crucially, all reported results appear to stem from a single run without providing mean ± standard deviation or any statistical significance testing, making it impossible to rule out random seed variance. Furthermore, there are multiple notable performance degeneration: e.g., AdaptFormer w/o-TR outperforms both Ours and Ours^SD on CAMO (Table 1), and AdaptFormer maintains a clear lead on SOD and HKU-IS (Table 6).
-
Theoretical FLOPs do not equate to actual wall-clock inference acceleration, especially given that the reduction in FLOPs compared to DAR is merely 3.2%. Consequently, the paper's claim of "substantially reduced computation" lacks actual latency or throughput comparisons, making it an apparent overclaim.
-
The 94.9% reduction in NMSE (from 22.32 to 1.13) to corroborate that their compensation mechanism is more effective. However, DAR's compensator was not designed or trained to approximate FFN residuals; benchmarking a baseline on an unoptimized surrogate metric is methodologically unfair. Furthermore, the paper lacks evidence showing a meaningful correlation between lower NMSE and downstream segmentation performance.
-
The title targets general “pixel-level tasks”, but the validation is largely confined to binary segmentation tasks. The cross-task generalizability of TRACE across broader and more representative dense prediction benchmarks—such as multi-class semantic segmentation and depth estimation—remains unknown.
Questions for authors
-
All results in Table 1 appear to come from a single run. I suggest the authors report the mean ± std over at least three random seeds for Ours vs. DAR, and indicate whether the reported 0.005–0.01 improvements are statistically significant?
-
On CAMO, AdaptFormer w/o TR outperforms both Ours and Ours^SD, and it also leads on SOD and HKU-IS (Table 6). How do you explain this degeneration of TRACE?
-
TRACE's FLOPs are only 3.2% lower than DAR's. Do you have wall-clock latency and peak memory measurements on the same GPU for AdaptFormer/DyT/DAR and TRACE?
-
The title targets general "pixel-level tasks," yet all experiments are binary segmentation tasks. Do you have results on multi-class semantic segmentation or depth estimation? If not, would you consider narrowing the title/abstract claims to binary dense segmentation?
2 comments
Thanks for the authors' feedback. The ADE20K result and the throughput measurement address parts of my concerns. However, my two main concerns remain.
First, the three-seed experiment is limited to DUTS-TE, and the authors appropriately state that they do not extrapolate this repeatability result to every dataset. The response on CAMO, SOD, and HKU-IS acknowledges the remaining regressions but does not explain them.
Second, the NMSE clarification addresses only part of my concern, No evidence is provided that lower NMSE correlates with downstream segmentation performance.
Finally, the 3.66% throughput gain over DAR is useful but modest, so “substantially reduced computation” remains an overstatement in this comparison.
Overall, I maintain my score.
Response to Reviewer HvhU
W1. Reliability and performance variation. We repeated DAR and Ours on DUTS-TE with three random seeds each (mean ± sample standard deviation):
The mean gains are 0.016204, 0.004390, and 0.005031, respectively, supporting the repeatability of the DUTS-TE result. To examine the broader trend, we computed the unweighted mean difference from AdaptFormer without TR across the six datasets in Table 1 (positive means higher than AdaptFormer):
Ours narrows the average performance gap associated with TR, although improvements are not uniform. On CAMO, Ours remains below AdaptFormer in Weighted-F, but the gap narrows from 0.024 (DAR) to 0.013 (Ours). The regressions on SOD and HKU-IS also remain. We therefore limit our claim to improvement across multiple datasets, rather than gains on every benchmark.
W2. Actual inference throughput. We measured all methods on an NVIDIA RTX 3090 using the same benchmarking procedure:
Ours reduces FLOPs by approximately 3.2% versus DAR and improves measured throughput by 3.66%. Versus AdaptFormer, measured throughput improves by 5.98%. We report actual throughput separately from theoretical FLOPs reduction.
W3. Interpretation of NMSE. We compare the compensation output with the missing FFN residual only for patch tokens that actually skip FFN computation. NMSE is a diagnostic of compensation behavior, not a reconstruction training objective or a segmentation metric. The complete per-layer data underlying Fig. 2(b) are:
DAR's NMSE is about 1.1–1.6 in most layers but rises to 3.7539 and 22.3181 at L11/L12. Ours yields 1.5402 and 1.1276, respectively. Both methods skip 100% of patch-token FFN computation in these layers. Thus, Ours mitigates DAR's anomalous final-layer NMSE. DAR was not trained to fit FFN residuals, and Ours uses no dedicated FFN-residual reconstruction loss. Although Ours's L12 NMSE (1.1276) remains above the zero-compensation reference (~1.0), our analysis identifies and mitigates the anomalous NMSE spike observed in DAR. The relationship among NMSE, output scale, and segmentation performance warrants further investigation.
**W4. Beyond binary segmentation. ** We additionally evaluated multi-class semantic segmentation on ADE20K:
The 0.39-percentage-point gain provides initial evidence beyond binary segmentation.We will further evaluate the method on depth estimation in future work.
Q1. Are the gains reliable? See W1: both methods were run with three seeds on DUTS-TE, and Ours has higher mean scores on all three metrics. We do not extrapolate this repeatability result to every dataset.
Q2. What explains the regressions? See W1. CAMO's Weighted-F gap to AdaptFormer narrows from 0.024 to 0.013, but regressions on SOD and HKU-IS remain. Our conclusion concerns the overall trend across datasets, not a gain on every benchmark.
Q3. Is there actual speedup? See W2: Ours reaches 38.48 FPS versus DAR's 37.12 FPS on the RTX 3090, a 3.66% throughput gain.
Q4. Does the method work beyond binary tasks? See W4: Ours achieves 46.31% best mIoU on ADE20K versus DAR's 45.92%. This is preliminary multi-class evidence; depth estimation remains untested.