- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SCOPE, a forward-only test-time adaptation framework that uses the batch dimension to explore multiple perturbation directions without additional batch forward passes. Instead of applying a shared perturbation to every sample, SCOPE assigns each sample a Rademacher-coded combination of shared Gaussian basis directions. Two perturbed forward passes produce sample-wise responses, which are jointly decoded through least squares to estimate directional coefficients for a shared prompt update. The paper also derives an exact sample-wise decomposition of feature-statistic alignment loss differences, making this batch-coupled objective compatible with the coded measurements.
Validation includes theoretical analysis of subspace coverage, first-order decoding unbiasedness, and convergence under stated assumptions. Experiments cover mild, continual, and wild adaptation, natural distribution shifts, and W8A8 quantization using ViT-B/16. SCOPE reports average accuracies of 63.47%, 63.35%, and 61.41% in the mild, continual, and quantized settings, respectively. Component ablations and gradient diagnostics provide additional evidence for the proposed design
Critical strengths
- A creative approach to obtaining richer adaptation feedback within a tight forward budget.
The central idea is appealing: different samples receive complementary combinations of perturbation directions, allowing the method to collect more diverse feedback within the existing batch computation. The generated responses are used to update a shared prompt, and the same two forward passes also provide predictions through logit fusion. This provides a concrete mechanism for improving forward-only adaptation without backpropagation or batch expansion. The reported gains across full-precision and quantized settings make this a potentially useful contribution to resource-constrained adaptation.
- The handling of batch-coupled objectives is technically substantive and supported by targeted ablations.
The statistic-alignment decomposition addresses an important obstacle to sample-wise perturbation measurements. Appendix A.3 establishes both exact conservation of the loss difference and first-order compatibility with the coded estimator under additional regularity conditions. Table 5 also provides a useful same-objective comparison: the complete method improves mild-TTA accuracy from 60.93% to 63.47% relative to shared-direction ZO with statistic alignment. The poor performance of uniform allocation further supports the importance of preserving the correspondence between samples and perturbation codes. These results give the proposed mechanism more substance than an aggregate accuracy comparison alone
Critical weaknesses
- Baseline fidelity and computational accounting are insufficiently documented for the main comparative claims.
My main concern is whether the reported ZOA and CAZO results faithfully represent these methods under the stated constraints. Section 5.1 specifies SCOPE’s configuration and assigns forward budgets to the baselines, but it does not adequately document the method-specific changes used to obtain those budgets. In particular, it is unclear whether CAZO retains its original adapter parameterization and whether ZOA retains its domain-knowledge management mechanism. Differences from earlier results can legitimately arise from reduced budgets or different evaluation protocols. However, the submission needs to explain those differences through concrete configurations rather than leaving readers to infer them
The absence of a review-accessible implementation makes these ambiguities harder to resolve. The abstract promises a future code release, and the reproducibility statement points to descriptions and pseudocode. Given the missing baseline configurations, executable code or a substantially more complete reproduction specification is important for assessing the reported improvements.
- The incremental value of least-squares decoding is not isolated from simpler sample-wise perturbation aggregation.
Table 5 supports the complete framework relative to shared-direction updates, but it does not compare joint decoding with a simpler estimator using the same sample-wise responses and perturbations: An algebraic consequence of Eqs. (9) and (13) is that SCOPE’s estimator becomes exactly this expression when . For the random, nonorthogonal codes actually used, the estimators differ. A controlled comparison would therefore clarify whether correcting empirical code correlations through least squares materially improves gradient estimation and adaptation, beyond the benefit of sample-wise perturbation diversity and statistic decomposition.
This distinction also matters for interpreting the theory. Proposition 1 establishes geometric subspace coverage, and Appendix A.2 establishes first-order unbiasedness in expectation over the code matrix. Neither result by itself establishes lower finite-batch estimation error than a simpler sample-wise estimator. Such a comparison would materially strengthen the evidence for the specific encoding–decoding design.
Questions for authors
Q1. Baseline implementation and reconciliation of results — clarification. How exactly were ZOA and CAZO implemented for the reported comparisons? Was CAZO evaluated with its original adapter parameterization, or was its optimizer transferred to the three-token prompt setting? Open source related code may lead to better ways.Can the time and memory overhead be compared with similar methods such as CAZO FOZO?
Q2. Reproducibility of the reported comparisons — clarification and supporting evidence. Could you provide an anonymized implementation, or the complete executable configurations and evaluation scripts, for SCOPE and the main forward-only baselines? The most important details are the exact pretrained checkpoint, feature layers and loss normalization, baseline-specific hyperparameters, and stream construction and reset rules. The paper reports averages over five independent runs. Please provide the corresponding standard deviations, particularly for the main baseline comparisons. For the efficiency results, please specify the hardware and timing procedure and include the relevant BP-free competitors. These details would directly affect my confidence in the reported accuracy and efficiency gains.
Q3. Contribution of joint decoding — targeted validation of the current claim. Under the same objective, perturbation scale, batch size, and prediction-fusion rule, how does least-squares decoding compare with direct aggregation ? A second useful control would use independent sample-wise perturbations with the same statistic decomposition and two-forward budget. Do these controls differ materially in gradient-estimation error, gradient alignment, and final adaptation accuracy? This comparison would clarify whether the specific shared-subspace decoding mechanism is responsible for the gains. I genuinely appreciate the creativity of using the batch dimension for complementary exploration, as well as the effort to make batch-coupled objectives compatible with this design. If the authors can resolve these questions, correct the forward-pass accounting where necessary, and provide evidence supporting the baseline comparisons, I would consider raising my score.