- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
STAGE refines continuous codes on a graph and uses them as targets for a binary hash encoder. The energy combines graph structure, quantization and semantic/code regularization. Projected updates preserve zero-mean codes, while learned positive per-bit spectral operators control refinement. The student fits these targets and encodes each sample independently at inference. Alongside batch refinement, the paper introduces a fixed-reference formulation with bounded centered readout, target caching by sample identity, and teacher feedback through a virtual student update. The theoretical results cover conservation, energy descent and binary imbalance under stated conditions. Experiments use frozen ResNet-18 features on CIFAR-10, CIFAR-100 and VOC 2007 with 8, 16 and 32 bits.
Critical strengths
-
The paper addresses a concrete deployment problem: codes optimized jointly on a graph must be learnable by an encoder that processes samples independently. The fixed-reference construction handles target range, centering and sample identity while retaining ordinary encoding and Hamming retrieval at inference. The theory also states the conditions under which its results hold; it does not claim an unconditional retrieval guarantee.
-
The internal controls are a strength. Direct-energy, scalar, bounded and optimizer-consistent feedback variants help separate refinement from generic regularization. Meta-scalar is particularly relevant because it shares the readout, refresh and outer supervision, isolating the additional spectral shape. STAGE achieves the highest reported means across the tested settings, so the evidence extends beyond a comparison with the base model.
Critical weaknesses
-
The small margins need a more reliable estimate of uncertainty. In the nine fixed-reference settings, gains over the strongest control range from 0.12 to 0.71 percentage points. These are the practical gains over available alternatives; the more closely matched Meta-scalar comparison answers the separate question of what spectral shape adds. Updated Table 5 means for CIFAR-100 and VOC lack updated standard deviations. Even marginal means and SDs would not establish uncertainty in the paired differences.
-
It is unclear whether the extra training machinery earns its cost over simpler alternatives. At 8 bits on CIFAR-10, Bounded T=0 reaches 69.78 mAP compared with 70.27 for the full model. At 8 bits on VOC, the gain over Reference base is only 0.54 points, despite a larger margin over Meta-scalar. Fixed-reference training takes about 2.31-2.39 times as long as the base model and about 1.10 times as long as Meta-scalar. The equal-time experiment covers only a subset of the batch protocol, leaving the fixed-reference results without that comparison.
Questions for authors
Which conclusions depend on the VOC data accessed during the pilot? Can you verify them with frozen hyperparameters on a split or setting that was not used in development? Please distinguish these results from the independent CIFAR validation/final-query protocol.
4 comments
Response to W[2/3]
Thank you for highlighting the practical trade-off between retrieval performance and training cost. Your examples show why this should be assessed against simpler alternatives as well as the closely matched Meta-scalar control.
We summarize the existing performance and timing results below to make these comparisons explicit.
Table R2. Eight-bit fixed-reference retrieval performance and training time from Tables 5 and 10(b). Results are three-seed means, with sample SD where available. Training time covers 60 epochs and excludes feature extraction, graph/pair/triplet setup, validation, and retrieval.
| Dataset | Method | mAP (%) | Training time (s) |
|---|---|---|---|
| CIFAR-10 | Reference base | 66.94 ± 4.45 | 7.50 ± 0.17 |
| CIFAR-10 | Bounded T=0 | 69.78 ± 0.43 | 9.82 ± 0.41 |
| CIFAR-10 | Meta-scalar | 69.89 ± 0.49 | 15.91 ± 0.52 |
| CIFAR-10 | STAGE-Ref | 70.27 ± 0.22 | 17.44 ± 0.08 |
| VOC 2007 | Reference base | 67.81 ± 0.13 | 5.37 ± 0.21 |
| VOC 2007 | Bounded T=0 | 64.97 ± 0.29 | 6.39 ± 0.44 |
| VOC 2007 | Meta-scalar | 64.44 ± 0.75 | 11.27 ± 0.23 |
| VOC 2007 | STAGE-Ref | 68.35† | 12.42 ± 0.63 |
† The sample SD corresponding to the updated VOC mAP mean is unavailable.
On CIFAR-10, STAGE-Ref improves over Bounded T=0 by 0.49 percentage points while requiring approximately 1.78 times its training time. On VOC, the improvement over Reference base is 0.54 points at approximately 2.31 times the training time. We agree that these are modest accuracy gains with a measurable computational cost.
The comparison with Meta-scalar addresses the incremental cost within the shared fixed-reference training design. STAGE-Ref requires approximately 10% more training time than Meta-scalar on these two datasets. However, the larger VOC gain over Meta-scalar should be considered alongside the smaller gain over Reference base when assessing practical value.
The additional graph refinement and teacher computations are confined to training. At inference, the method retains independent image encoding and standard Hamming-distance retrieval. This preserves the deployment procedure, while the additional training cost remains a relevant consideration.
We therefore describe the reported improvement as an accuracy–training-cost trade-off under the fixed-epoch protocol. The existing equal-time results in Table 9 concern the batch protocol. An equal-time advantage for STAGE-Ref remains unverified, and we do not extend the batch-protocol evidence to that claim.
Response to W[3/3]
Thank you for asking which conclusions depend on the VOC data accessed during the pilot. This is an important distinction between separation within the final evaluation protocol and independence from earlier development decisions.
Appendix A.3.1 discloses that the official VOC test set was accessed during the initial pilot and subsequently reused for final evaluation. The CIFAR final-query blocks were unused in that pilot. We summarize the relevant protocol details below.
Table R4. Validation and final-query usage in the reported evaluation.
| Dataset | Validation queries | Final queries | Final-query block accessed during the initial pilot |
|---|---|---|---|
| CIFAR-10 | 500 | 1,000 | No |
| CIFAR-100 | 500 | 1,000 | No |
| VOC 2007 | 500 | 4,952 | Yes |
For VOC, the validation queries are drawn from trainval, and all 4,952 official test images serve as final queries. The final queries are excluded from the retrieval database. These protocol details establish the separation used in the final runs, but the extent to which the earlier VOC observations influenced method design or experimental choices remains undocumented.
Consequently, all reported VOC retrieval comparisons inherit this limitation. This includes the 8-bit gains of 3.91 percentage points over Meta-scalar and 0.54 points over Reference base, as well as broader empirical claims that rely on the VOC results. We therefore treat these measurements as results on a benchmark accessed during development, with the corresponding qualification.
The CIFAR results use their separately reserved final-query blocks and support the reported conclusions on those datasets. They do not independently verify the specific VOC findings. The mathematical results on conservation, stationary points, and conditional descent are derived under their stated assumptions and do not depend on VOC measurements.
We agree that a stronger confirmation would freeze the method, hyperparameters, and checkpoint-selection rule before evaluation on a split or setting genuinely unused during development. Fixing hyperparameters and rerunning the previously accessed VOC test set would not restore that independence. Such an independent confirmation remains outstanding in the current evidence.
Our response therefore makes the provenance and scope of the VOC findings explicit and keeps them separate from the CIFAR final-query protocol. We appreciate your careful attention to this distinction, together with your assessment of the independent-encoding formulation and matched controls. Your comments help us state both the contribution and the remaining evidential limitations more precisely.
Official Comment by Authors
We sincerely thank you for your careful review and your recognition of the constrained formulation and independent encoding at inference. Your questions about the small spectral gains, their uncertainty, and the relationship between energy reduction and retrieval performance highlight where the evidence needs a more precise interpretation. Your attention to computational cost, evaluation scope, and the VOC pilot also helps clarify the conditions under which the results should be understood. We appreciate the rigor of your assessment and hope that our point-by-point responses make the contribution, supporting evidence, and remaining limitations clearer. We would welcome any further questions.
Response to W[1/3]
Thank you for distinguishing the contribution of spectral shape from the practical gains over competitive alternatives, and for recognizing the value of the matched controls. We agree that the small margins make uncertainty in the paired differences important for interpreting these results.
Meta-scalar shares the bounded readout, target refresh, and outer supervision with STAGE-Ref, providing a matched comparison for assessing the contribution of spectral parameterization. Bounded T=0 provides a simpler target construction for assessing the benefit of the full refinement procedure.
We compute each gain from the displayed mean mAP values.
ΔmAP=mAPSTAGE-Ref−mAPcomparator.
Table R1. Eight-bit fixed-reference retrieval results from Tables 5–6. mAP is summarized over three seeds, with sample SDs shown where available. ΔmAP is in percentage points (pp).
† The sample SD corresponding to the updated mean is unavailable.
At 8 bits, the mean gains over Meta-scalar are 0.38, 0.80, and 3.91 pp on CIFAR-10, CIFAR-100, and VOC 2007, respectively. Across the nine settings in Table 2, the gains over the strongest control evaluated in each setting range from 0.12 to 0.71 pp.
The reported sample SDs describe seed-to-seed variation within each method. Quantifying uncertainty in the paired differences requires matched seed-level scores, including those underlying the updated CIFAR-100 and VOC 2007 means. We therefore interpret the results as descriptive evidence of higher mean mAP under the reported settings. Statistical significance and consistency across seeds remain to be established.