acceptodds
Withdrawn from ICLR 2027

Subspace-Coded Parallel Exploration for Forward-Only Test-Time Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Zeroth-order (ZO) optimization enables forward-only test-time adaptation (TTA), avoiding the computation and memory overhead of backpropagation. However, existing direction-centric approaches face an exploration–exploitation dilemma in capturing gradient energy: exploiting promising directions may overlook complementary gradient components, whereas broader exploration typically requires additional forward evaluations. To address this limitation, we propose **S**ubspace-**C**oded **O**ptimization via **P**arallel **E**xploration (**SCOPE**), a novel TTA framework that shifts from individual direction selection to higher-dimensional subspace exploration. Specifically, we introduce a perturbation encoding scheme that assigns different samples within a batch complementary combinations of shared basis directions, rather than applying the same perturbation to all samples. A lightweight least-squares estimator jointly decodes their loss differences to estimate batch-average directional responses for a shared prompt update. To support batch-coupled objectives such as feature-statistic alignment, we further decompose their batch-level loss differences into exact sample-wise contributions, preserving the original adaptation objective while making its feedback compatible with the perturbation codes. Together, these designs embed multi-directional measurements into existing batch computation, broadening gradient-energy coverage with only two forward passes. Extensive experiments across mild, continual, and wild TTA settings, together with natural-shift and quantized models, demonstrate the effectiveness and efficiency of SCOPE. Code will be publicly available.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Test-time adaptation, Zeroth-order optimization, Forward-only adaptation, Gradient estimation, Subspace exploration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_55bdOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SCOPE, a forward-only test-time adaptation framework that uses the batch dimension to explore multiple perturbation directions without additional batch forward passes. Instead of applying a shared perturbation to every sample, SCOPE assigns each sample a Rademacher-coded combination of shared Gaussian basis directions. Two perturbed forward passes produce sample-wise responses, which are jointly decoded through least squares to estimate directional coefficients for a shared prompt update. The paper also derives an exact sample-wise decomposition of feature-statistic alignment loss differences, making this batch-coupled objective compatible with the coded measurements.

Validation includes theoretical analysis of subspace coverage, first-order decoding unbiasedness, and convergence under stated assumptions. Experiments cover mild, continual, and wild adaptation, natural distribution shifts, and W8A8 quantization using ViT-B/16. SCOPE reports average accuracies of 63.47%, 63.35%, and 61.41% in the mild, continual, and quantized settings, respectively. Component ablations and gradient diagnostics provide additional evidence for the proposed design

Critical strengths
  1. A creative approach to obtaining richer adaptation feedback within a tight forward budget.

The central idea is appealing: different samples receive complementary combinations of perturbation directions, allowing the method to collect more diverse feedback within the existing batch computation. The generated responses are used to update a shared prompt, and the same two forward passes also provide predictions through logit fusion. This provides a concrete mechanism for improving forward-only adaptation without backpropagation or batch expansion. The reported gains across full-precision and quantized settings make this a potentially useful contribution to resource-constrained adaptation.

  1. The handling of batch-coupled objectives is technically substantive and supported by targeted ablations.

The statistic-alignment decomposition addresses an important obstacle to sample-wise perturbation measurements. Appendix A.3 establishes both exact conservation of the loss difference and first-order compatibility with the coded estimator under additional regularity conditions. Table 5 also provides a useful same-objective comparison: the complete method improves mild-TTA accuracy from 60.93% to 63.47% relative to shared-direction ZO with statistic alignment. The poor performance of uniform allocation further supports the importance of preserving the correspondence between samples and perturbation codes. These results give the proposed mechanism more substance than an aggregate accuracy comparison alone

Critical weaknesses
  1. Baseline fidelity and computational accounting are insufficiently documented for the main comparative claims.

My main concern is whether the reported ZOA and CAZO results faithfully represent these methods under the stated constraints. Section 5.1 specifies SCOPE’s configuration and assigns forward budgets to the baselines, but it does not adequately document the method-specific changes used to obtain those budgets. In particular, it is unclear whether CAZO retains its original adapter parameterization and whether ZOA retains its domain-knowledge management mechanism. Differences from earlier results can legitimately arise from reduced budgets or different evaluation protocols. However, the submission needs to explain those differences through concrete configurations rather than leaving readers to infer them

The absence of a review-accessible implementation makes these ambiguities harder to resolve. The abstract promises a future code release, and the reproducibility statement points to descriptions and pseudocode. Given the missing baseline configurations, executable code or a substantially more complete reproduction specification is important for assessing the reported improvements.

  1. The incremental value of least-squares decoding is not isolated from simpler sample-wise perturbation aggregation.

Table 5 supports the complete framework relative to shared-direction updates, but it does not compare joint decoding with a simpler estimator using the same sample-wise responses and perturbations: g^avg=1B∑i=1BqiCvi.\widehat g_{\mathrm{avg}}=\frac{1}{B}\sum_{i=1}^{B}q_i^C v_i. An algebraic consequence of Eqs. (9) and (13) is that SCOPE’s estimator becomes exactly this expression when C⊤C=(B/K)IC^\top C=(B/K)I. For the random, nonorthogonal codes actually used, the estimators differ. A controlled comparison would therefore clarify whether correcting empirical code correlations through least squares materially improves gradient estimation and adaptation, beyond the benefit of sample-wise perturbation diversity and statistic decomposition.

This distinction also matters for interpreting the theory. Proposition 1 establishes geometric subspace coverage, and Appendix A.2 establishes first-order unbiasedness in expectation over the code matrix. Neither result by itself establishes lower finite-batch estimation error than a simpler sample-wise estimator. Such a comparison would materially strengthen the evidence for the specific encoding–decoding design.

Questions for authors

Q1. Baseline implementation and reconciliation of results — clarification. How exactly were ZOA and CAZO implemented for the reported comparisons? Was CAZO evaluated with its original adapter parameterization, or was its optimizer transferred to the three-token prompt setting? Open source related code may lead to better ways.Can the time and memory overhead be compared with similar methods such as CAZO FOZO?

Q2. Reproducibility of the reported comparisons — clarification and supporting evidence. Could you provide an anonymized implementation, or the complete executable configurations and evaluation scripts, for SCOPE and the main forward-only baselines? The most important details are the exact pretrained checkpoint, feature layers and loss normalization, baseline-specific hyperparameters, and stream construction and reset rules. The paper reports averages over five independent runs. Please provide the corresponding standard deviations, particularly for the main baseline comparisons. For the efficiency results, please specify the hardware and timing procedure and include the relevant BP-free competitors. These details would directly affect my confidence in the reported accuracy and efficiency gains.

Q3. Contribution of joint decoding — targeted validation of the current claim. Under the same objective, perturbation scale, batch size, and prediction-fusion rule, how does least-squares decoding compare with direct aggregation g^avg=B−1∑iqiCvi\widehat g_{\mathrm{avg}}=B^{-1}\sum_i q_i^C v_i? A second useful control would use independent sample-wise perturbations with the same statistic decomposition and two-forward budget. Do these controls differ materially in gradient-estimation error, gradient alignment, and final adaptation accuracy? This comparison would clarify whether the specific shared-subspace decoding mechanism is responsible for the gains. I genuinely appreciate the creativity of using the batch dimension for complementary exploration, as well as the effort to make batch-coupled objectives compatible with this design. If the authors can resolve these questions, correct the forward-pass accounting where necessary, and provide evidence supporting the baseline comparisons, I would consider raising my score.

Reviewer_canVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SCOPE, a forward-only framework for test-time prompt adaptation. It assigns different samples within a batch coded combinations of shared random basis directions. Two batch forward passes produce sample-indexed responses, which are jointly decoded through least squares to estimate batch-average directional responses for updating a shared prompt. To accommodate batch-coupled feature-statistic alignment, the authors derive an exact sample-wise decomposition of the statistical loss difference. Experiments cover several distribution-shift settings and quantized models, indicating improvements.

Critical strengths
  • Using the batch dimension to collect multi-directional feedback in parallel is well-motivated.
  • The decomposition of batch-coupled statistical loss differences is useful.
Critical weaknesses
  1. The method does not separate the contributions of the adaptation objective and the gradient estimator well. The proposed method is an approximation of back propagation, but in Table 4 it slightly outperforms the backpropagation-based ReCAP in accuracy. However, the compared methods differ in their adaptation objectives, trainable parameters, and update rules, making the comparison conclusions unclear. Since SCOPE updates the prompt using a gradient estimate,

  2. The proposed method relies on batched data, which might not be realizable in real applications and could lead to overstated performance gains that are unfair for compared TTA methods. Specifically, SCOPE relies on feedback from different samples to estimate multiple directional responses, with a default configuration of batch size 64. Generally, in online applications, only a few samples (sometimes only one sample) are available at a time. The reported batch-size analysis starts at 16 and does not analyze how the method respond to small-batch scenarios.

  3. The method encourages the feature mean and std of a test batch to approach their source-domain counterparts, which is not sufficiently justified. Departure from source statistics does not necessarily indicate a harmful shift, as target distributions would naturally disagree with source's. Therefore, how would the alignment objective benefits target-domain classification is not clearly established.

Questions for authors
  1. It would be informative to establish how exact backpropagation performs with the same prompt parameterization and objective. Please add a controlled comparison between SCOPE and backpropagation that matches the entropy and feature-statistic alignment objective, trainable prompt parameters, and prediction procedure, while reasonably tuning the optimization hyperparameters for each approach.

  2. Please evaluate batch sizes below 16, specify how the subspace dimension is selected, and report accuracy and latency relative to a shared single-direction zeroth-order baseline. This would clarify the method’s dependence on batch size and its deployment limits. Please indicate how the fairness is ensured for batch-based SCOPE and the other TTA methods.

  3. Please clarify the assumptions under which encouraging target distribution to match source is appropriate and discuss possible failure cases. A more targeted comparison with and without alignment across different degrees of class imbalance would help assess the possibility of negative transfer.

Reviewer_vTMtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SCOPE, a forward-only test-time adaptation framework that explores multiple perturbation directions within two batch forward passes. Instead of applying a shared perturbation to all samples, SCOPE assigns different coded combinations of random basis directions to individual samples and uses least-squares decoding to estimate batch-average directional responses. The authors further introduce an exact sample-wise decomposition of feature-statistic alignment loss differences to support batch-coupled objectives. Experiments across mild, continual, and wild TTA settings, natural distribution shifts, and quantized models report improvements over several forward-only baselines.

Critical strengths
  1. Interesting use of batch-level parallelism. Distributing complementary perturbations across samples is an intuitive way to collect more diverse optimization feedback without increasing the number of batch forward passes.

  2. Useful treatment of batch-coupled objectives. The exact decomposition of feature-statistic alignment loss differences enables sample-wise perturbation measurements while preserving the original batch-level loss difference.

  3. Encouraging empirical results. SCOPE achieves improvements across multiple distribution shifts and quantized-model settings. The component ablations and gradient-alignment experiments also provide useful evidence for the proposed framework.

Critical weaknesses
  1. The novelty relative to existing sample-wise ZO methods is not sufficiently established. EVA-0 [1] already introduces sample-wise symmetric two-sided perturbations and averaged perturbed predictions under the same two-forward budget, but is neither discussed nor compared. More importantly, independent sample-wise perturbations can span up to B directions, whereas SCOPE restricts its perturbations to a shared K-dimensional subspace (K=16, B=64 by default). Although SCOPE introduces a different encoding–decoding mechanism, its additional advantage over existing sample-wise exploration remains unclear.

  2. Broader subspace coverage does not necessarily imply better gradient estimation. Proposition 1 establishes that K random directions capture an expected K/d fraction of gradient energy, but this geometric property does not guarantee that the corresponding directional responses can be accurately recovered. In Eq. (8), the observations contain sample-specific gradient variation in addition to finite-difference errors. Although the appendix establishes unbiasedness in expectation over random codes, it does not sufficiently characterize the finite-batch estimation error caused by these variations. Figure 3 reports improved gradient alignment against existing baselines, but a comparison with independent sample-wise estimation is still needed to support the claimed advantage of subspace coding.

  3. The contribution of least-squares decoding is not sufficiently isolated. Table 5 demonstrates the benefit of the complete framework over shared-direction ZO, but does not compare the proposed decoder with direct aggregation of the same coded sample-wise responses. In fact, when the code matrix satisfies �0��0�, SCOPE's estimator is algebraically equivalent to the simpler estimator �1��1�. For nonorthogonal random codes, their difference should be evaluated directly. Without this comparison, it remains unclear whether least-squares decoding provides meaningful benefits beyond sample-wise perturbation diversity and loss decomposition.

  4. The practical advantage under limited deployment resources remains unclear. SCOPE relies on multiple samples to decode directional responses, yet the batch-size analysis only considers sizes of 16 or larger. With batch size 1, the method cannot explore multiple independent basis directions, limiting its applicability to strict online adaptation. Furthermore, runtime and memory comparisons mainly focus on BP-based methods rather than the closest two-forward baselines, making the actual accuracy–latency trade-off difficult to assess.

[1] Chen et al., EVA-0: Test-Time Model Evolution with Only Two Forward Passes per Sample.

Questions for authors
  1. Novelty and information content. Given that independent sample-wise perturbations can explore up to B directions under the same two-forward budget, what additional information or estimation advantage does SCOPE's K-dimensional coded exploration provide over EVA-0-style estimation?

  2. Necessity of least-squares decoding. Under identical coded perturbations and sample-wise responses, how does least-squares decoding compare with direct aggregation, �0��0�, in terms of gradient MSE, alignment, and final adaptation accuracy?

  3. Finite-batch estimation quality. How does sample-gradient heterogeneity affect decoding error, particularly as K approaches B? Can the authors provide a variance or finite-batch error analysis beyond unbiasedness in expectation?

  4. Controlled comparison and deployment efficiency. How does SCOPE compare with standard ZO and exact backpropagation under identical adaptation objectives and prompt parameters? Can the authors also clarify the ZOA/CAZO configurations and report matched runtime and memory measurements, including small-batch settings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.