acceptodds
Withdrawn from ICLR 2027

EMO: A Closed-Loop Test-Time Computation Framework via Alternating Context Optimization and Logits Steering

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Test-time context optimization adapts frozen language models through textual feedback, but ordinary conditional generation provides no separate control over how strongly to reinforce the distributional changes induced by a context update. We introduce EMO, a training-free framework that couples question-specific context optimization with logits steering. The E-step revises a system context using textual gradients from an external judge without access to gold answers. The M-step contrasts next-token distributions under the revised and preceding contexts at a shared decoding prefix, transforms this contrast into a steering vector, and applies a confidence-constrained logit correction. Feedback on the steered response drives the next context update, coupling adaptation across rounds with intervention within each generation. A logit-space analysis connects the steering vector to the context-induced logit displacement and identifies sufficient conditions for one-round target-KL improvement. Under greedy decoding across eight models and six benchmarks, EMO improves the zero-shot baseline in 39 of 40 aggregated settings (47 of 48 before merging the two AIME editions), averaging percentage points over the 40 model–task-group pairs. With the same judge, it adds points on average over context-only optimization, winning in 25 settings and tying in four, with MMLU-Pro gains across all eight models. At matched target-model generation budgets, accuracy advantages over best-of- are strongest on Llama, while oracle coverage is higher in four of six comparisons despite fewer candidates. The coverage–selection gap motivates allocating test-time computation jointly to candidate generation and verification.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Test-Time Optimization, Logits Steering, Context Optimization, LLM, Reasoning, Expectation-Maximization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AdFVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes EMO, a training-free framework that alternates per-question system-context optimization with token-level logits steering. An external judge provides feedback for context revisions. The target model then contrasts distributions under successive contexts at a shared prefix, applies a Jacobian-weighted log-ratio correction with a confidence mask, and feeds the steered response into the next update. Experiments cover eight models and six reasoning benchmarks. The paper reports a 9.10-point average gain over zero-shot decoding and a 1.58-point gain over context-only optimization, alongside nominal generation-budget comparisons and a conditional target-KL analysis.

Critical strengths
  • Useful improvements across several backbones. The comparison with context-only optimization controls the judge and candidate count, and steering improves MMLU-Pro accuracy for all eight models. This provides evidence that decoding interventions can add value to feedback-driven context updates.

  • Candidate generation and selection are evaluated separately. Reporting oracle coverage alongside selected accuracy reveals meaningful bottlenecks. For example, on Llama-3.1-8B/GPQA-D, EMO raises coverage to 73.7% versus 60.3% for independent sampling, despite generating fewer candidates.

Critical weaknesses
  1. Dependence on a stronger external judge. EMO relies on a 122B external judge for both feedback and candidate selection, while the target models are 3B–32B. The paper does not compare against directly using the judge as a solver, and its compute-matched comparisons exclude judge/rewrite overhead. This makes it difficult to assess whether EMO provides a meaningful efficiency or capability advantage over simply allocating computation to the stronger model. Erroneous judge feedback can also be amplified by subsequent context updates and steering.
  2. Insufficient isolation of the M-step contribution. The paper does not compare the proposed SVDecode-style Jacobian/KL steering against a simpler direct context-contrast baseline using the same Ck,Ck+1C_k,C_{k+1}. Without this comparison, it is unclear whether the gains come from the proposed steering geometry, from simply amplifying successive-context differences, or from the confidence mask.
  3. The motivation for steering is not empirically diagnosed. Although EMO improves over context-only TextGrad on average, the paper does not show that these gains arise specifically when the optimized context contains the right guidance but ordinary decoding fails to realize it. Per-instance case studies or analyses of TextGrad-fail/EMO-success examples are needed to establish the claimed learning–execution gap.
  4. Judge feedback and selection are confounded. Replacing the external judge changes both the feedback used for context optimization and the final candidate selection rule, so the current ablation cannot determine whether gains come from better adaptation or simply stronger reranking.

[1] Jerry Zhi-Yang He, Sashrika Pandey, Mariah L. Schrum, and Anca Dragan. Context Steering: Controllable Personalization at Inference Time. ICLR 2025.

[2] Senkang Hu et al. Distribution-Aligned Decoding for Efficient LLM Task Adaptation. NeurIPS 2025.

Questions for authors
  • How does EMO compare with directly using the external judge as the solver under a comparable end-to-end compute budget? The main experiments use Qwen3.5-122B-A10B as the external judge for both feedback and candidate selection, while the target models are 3B–32B, and the current matched-budget comparisons exclude judge/rewrite costs. If direct inference from the judge achieves comparable or better performance at similar total cost, this would substantially change how I interpret EMO’s practical value.

  • Is the SVDecode-style M-step actually necessary compared with a simpler direct context-contrast intervention? Given the same previous and updated contexts, a closely matched baseline could directly amplify the context-induced logit displacement, i.e., use z_new + beta * (z_new - z_old), similar to Context Steering / contrastive decoding. Can the authors compare this with the proposed Jacobian/KL-based steering under the same confidence mask and tuning budget? This would clarify whether the gain comes from the proposed steering geometry or simply from amplifying successive-context differences.

  • What evidence shows that EMO specifically addresses a failure of execution rather than a failure of context optimization? Among cases where TextGrad fails but EMO succeeds, how often does the optimized context already contain the correct or useful guidance, while ordinary decoding fails to realize it and steering changes the outcome? A case-level or aggregate analysis of these TextGrad-fail/EMO-success examples would materially affect whether the motivation for steering is supported by the experiments.

  • Can the effects of judge feedback and candidate selection be disentangled? In the current “w/o Judge” ablation, replacing the external judge changes both the feedback used to update the context and the mechanism used to select the final response, so the reported difference conflates adaptation quality with reranking quality. Can the authors report results that vary feedback source and selection source separately, or otherwise clarify how much of the gain is attributable to each role?

7 comments
AuthorsOpenReview ↗

Overview of Our Responses to Weaknesses and Questions

Thank you for recognizing the improvements across backbones and the distinction between candidate coverage and final selection. We organize our responses around your four weaknesses and four corresponding questions, followed by the bibliography concern.

  • W1/Q1: Strong-model dependence and complete cost. We explain the Backward Engine/optimizer and selector roles, report the same 122B checkpoint as a direct solver, and distinguish target adaptation from deployment efficiency. A separate post gives the complete self-judge ablation, where no external model is used.
  • W2/Q2: Isolating the steering direction and mask. We compare direct context contrast with Jacobian/KL steering at shared C0/C1, using the same mask and tuning budget, then report mask removal and multi-round coverage.
  • W3/Q3: Evidence for an execution gap. We trace shared guidance, ordinary-decoding errors, and steered responses in concrete cases, and delimit what the diagnostic sample establishes about frequency.
  • W4/Q4: Separating feedback and selection. We freeze two candidate-pool sources and cross them with three selectors; Pass@5 evaluates coverage independently of final selection.

Historical tables retain the original grading and selections. New controlled experiments and cost measurements are identified separately. Each response below addresses the corresponding concern directly.

AuthorsOpenReview ↗

Response for W1&Q1 (Part I): Auxiliary-Model Roles and Direct-Solver Reference

1. Why use a stronger auxiliary model?

TextGrad uses a Backward Engine to produce textual gradients and a Textual Gradient Descent optimizer to revise text variables. Its prompt-optimization experiment uses GPT-4o. We follow this division of roles, while optimizing context per question rather than learning one task-level prompt. The stronger model's task understanding motivates using it to diagnose responses and guide context updates. TextGrad §3.3

Our Qwen3.5-122B-A10B-FP8 auxiliary checkpoint has approximately 10B active parameters per token. In the main experiments, its three operations use greedy decoding with thinking disabled:

RoleInputOutput
Textual backward / Backward EngineQuestion, context, target responseTextual gradient for the context
Context rewriting / TGD optimizerContext and textual gradientRevised context
Candidate evaluationQuestion and one target candidateScore and evaluation rationale

No auxiliary call receives gold answers. The scoring rationale is separate from the textual gradient and is not used for rewriting. Selection uses scores, with earliest-round tie-breaking. The target generates every candidate with frozen weights. Four refinements produce R0–R4; the historical loop also executes a terminal context update, included in complete-run accounting.

2. Auxiliary-model access as a deployment expense

We view auxiliary-model access as an optimization/evaluation service alongside target serving. For an API deployment, its economic expense depends on input/output tokens and service prices. Equal total FLOPs compares resource efficiency, but does not fully determine monetary cost. This motivates reporting auxiliary usage separately while still charging all model calls in the compute account.

This perspective also charges BoN/SC for their scoring calls. EMO additionally uses backward and rewriting, and its per-question updates remain a recurring expense. Our current evidence records tokens, latency and estimated FLOPs; it does not establish a price-based advantage without a specified tariff and target-serving cost.

3. Direct solving with the same checkpoint

As requested, we give the same 122B checkpoint only the question and ask it to solve directly: greedy decoding, thinking off, and an 8,192-token cap (12,288 for AIME). These are full-set greedy runs. Per-query FLOPs are estimated as

\overline F=N^{-1}\sum_q\widehat F(q),\quad P_{\mathrm{active}}\approx10\mathrm B.$$ Input/output tokens come from saved successful API calls. The table reports confirmed-correct answers and mean estimated TFLOPs per query. This first-order proxy omits attention-length, routing and communication costs; it is not a hardware FLOPs measurement. | Dataset | Questions | Verified correct | Verified accuracy (%) | Estimated FLOPs (TFLOPs/query) | | --- | --- | --- | --- | --- | | MMLU-Pro | 560 | 488 | 87.14 | 31.63 | | GPQA-D | 198 | 154 | 77.78 | 83.35 | | MATH-500 | 500 | 479 | 95.80 | 31.28 | | GSM8K | 1319 | 1281 | 97.12 | 9.58 | | AIME-2025 | 30 | 23 | 76.67 | 124.35 | | AIME-2026 | 30 | 24 | 80.00 | 108.24 | Verified accuracy divides confirmed-correct answers by all questions; answers left unresolved by verification remain in the denominator. GPQA uses the corrected-input solver reference. We acknowledge that direct 122B solving outperforms EMO with either tested 8B target in the separate same-question sampled audit. The direct-solver results demonstrate strong task competence and support our choice of a capable feedback/evaluation model. They support the **capability premise behind this choice**; the effectiveness of its textual guidance remains an empirical question addressed by the ablations and role-separation analysis. EMO focuses on improving a specified frozen target with auxiliary guidance. Part II tests whether such improvement requires access to an external model at all.
AuthorsOpenReview ↗

Response for W1&Q1 (Part II): Adaptation Without an External Model

The paper's “w/o Judge” ablation removes the external judge: the target checkpoint itself performs textual backward, context rewriting, scoring, and selection. “w/o Both” uses the same self-model roles with context-only TextGrad. All roles use the original greedy configuration; no 122B or other larger model is called.

We report the complete original-paper comparison below. Self-EMO and Self-TG each generate R0–R4 and retain their historical score-based selections. Acc measures the selected answer; Pass@5 measures whether the five-candidate pool contains a correct answer. Base is the initial unoptimized response. These are historical greedy runs, not three-seed means. Bold compares Base/Self-TG/Self-EMO accuracy and Self-TG/Self-EMO Pass separately within each dataset–target row.

DatasetTarget / self modelQuestionsBase accuracy (%)Self-TG accuracy (%)Self-TG Pass@5 (%)Self-EMO accuracy (%)Self-EMO Pass@5 (%)
MMLU-ProQwen3-8B56063.5763.7569.8265.1876.79
MMLU-ProQwen3-32B56071.4374.8282.3275.3684.46
GPQA-DQwen3-8B19848.9954.5564.1451.0163.64
GPQA-DQwen3-32B19859.6064.1474.2461.6273.74
MATH-500Qwen3-8B50075.0078.0086.6078.6086.40
MATH-500Qwen3-32B50078.6082.6087.4083.2087.80
GSM8KQwen3-8B131991.4392.4295.5391.8195.75
GSM8KQwen3-32B131991.3691.7495.9191.6695.83
AIME-2025Qwen3-8B3016.6720.0043.3326.6750.00
AIME-2025Qwen3-32B3010.0026.6750.0030.0053.33
AIME-2026Qwen3-8B3023.3330.0066.6736.6770.00
AIME-2026Qwen3-32B3026.6743.3363.3346.6770.00

Self-EMO improves selected accuracy over Base in all 12 configurations. Giving each of the six datasets equal weight, Qwen3-8B increases from 53.17% to 58.32% (+5.16 pp), and Qwen3-32B from 56.27% to 64.75% (+8.48 pp). AIME-2025/2026 gains are +10.00/+13.33 pp for 8B and +20.00/+20.00 pp for 32B.

Relative to Self-TG, mean selected accuracy improves by 1.87/0.87 pp and Pass@5 by 2.75/1.99 pp for 8B/32B. Each model wins on four of six datasets in selected accuracy; GSM8K and GPQA do not show an advantage. These are descriptive macro averages, with each AIME edition counted separately, rather than significance tests.

The table retains the paper's original question sets, correctness, and candidate selections, including the full historical GPQA set. The results establish that external-model access is not necessary for obtaining improvements over the unoptimized target. They do not isolate feedback from selection, since both roles change together in the original ablation; the crossed-selector experiment in our W4/Q4 response addresses that distinction.

Reviewer_4QSqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EMO, a training-free test-time computation framework that alternates between context optimization and logit steering for frozen language models. For each question, the E-step revises the system context using textual feedback from an external judge without access to gold answers. The M-step compares next-token distributions under the revised and preceding contexts at a shared decoding prefix, converts their contrast into a steering vector, and applies a confidence-constrained logit correction. Feedback on the resulting response drives the next context update, forming a closed loop between context refinement and decoding intervention. The authors provide a logit-space interpretation of these operations and sufficient conditions for a one-round reduction in KL divergence to a target distribution. Experiments cover eight models and six benchmarks, reporting average gains of 9.10 percentage points over zero-shot baselines and 1.58 points over context-only optimization using the same judge. Component ablations and comparisons at matched target-model generation budgets further examine the contributions of the two steps, candidate coverage, and final answer selection.

Critical strengths
  1. The paper integrates Context Optimization with Logits Steering, using context-induced distribution changes to guide decoding and feedback from steered responses to refine subsequent contexts. This closed-loop design enables dynamic interaction between context optimization and decoding without parameter updates.
  2. The paper evaluates EMO across 8 language models and 6 reasoning benchmarks, demonstrating consistent improvements over baseline models and additional gains over TextGrad in several settings. Component ablations and comparisons across decoding strategies and inference budgets further support its effectiveness.
  3. The paper connects context updates with decoding interventions in logit space and provides a conditional theoretical guarantee of target KL improvement. Its analysis of Oracle Coverage and answer selection also highlights distinct bottlenecks in candidate generation and verification, offering useful insights into test-time reasoning.
Critical weaknesses

1.The comparisons with Best of N and Self Consistency match only the target model generation budget, excluding the costs of the external Judge, Feedback Generation, and Context Rewriting. Given the use of Qwen3.5-122B-A10B, these costs may be substantial. Reporting end to end inference time, token consumption, and comparisons under matched total compute budgets would better establish EMO's practical efficiency.

2.The individual contributions of Distribution Contrast, Confidence Constraint, and Closed loop Feedback Coupling remain unclear. The direct benefits of Logits Steering are also not separated from its effects on subsequent Context Optimization. Given the modest average improvement of 1.58 percentage points over TextGrad and occasional performance degradation, further ablations would help clarify the contribution of each component.

Questions for authors

Q1.Could the authors provide additional computational cost metrics, such as end-to-end inference time, total token consumption, or FLOPs, and conduct comparisons with Best-of-N and Self-Consistency under a unified computational budget that accounts for all additional overhead? This would help determine whether EMO's performance advantages remain under comparable practical inference costs.

Q2. Could the authors introduce a TextGrad + Logits Steering baseline without feedback coupling, where steering is applied along the context update trajectory generated by TextGrad, but the steered responses are not used to update subsequent contexts, and compare it with the full EMO framework?

5 comments
AuthorsOpenReview ↗

Overview of Our Responses to Weaknesses and Questions

Thank you for recognizing the connection between context optimization and logits steering, the breadth of the evaluation, and the distinction between candidate generation and selection. We organize the response around your two weaknesses and two questions:

  • W1 and Q1: Full computational cost. You ask whether the BoN/SC comparison remains informative after including judge evaluation, feedback, rewriting, token consumption, and end-to-end time. Replies 1–2 distinguish new instrumented runs from historical estimates, account for all roles, and compare performance within the measured-run BoN budget. SC uses the paper's score-based tie-break rule.
  • W2: Distribution contrast and confidence constraint. Reply 3 derives the steering direction from the observed context update, then compares direction and mask while holding C0/C1 fixed. Multi-round coverage is reported separately because later contexts can diverge.
  • W2 and Q2: Closed-loop feedback coupling. Reply 4 implements your requested TextGrad + steering baseline: contexts follow ordinary TextGrad responses, while steered responses never update subsequent contexts. It compares this frozen-trajectory replay with full EMO.

Acc denotes selected-answer accuracy; Pass@N denotes whether a pool contains a correct candidate. Greedy and sampled experiments are labeled separately. All FLOPs are estimates; runtime and token usage are recorded where stated.

AuthorsOpenReview ↗

Response for W1&Q1 (Part I): Auxiliary-Model Roles, All-Role BoN/SC Costs and Measured Runtime

We first explain the auxiliary model's role, then account for its use in the BoN/SC comparison.

1. Why use a stronger auxiliary model?

TextGrad combines a Backward Engine for textual gradients with Textual Gradient Descent for text updates, of which prompt-optimization experiment uses GPT-4o feedback. We follow this role assignment while optimizing context per question rather than one task-level prompt. The auxiliary model guides updates and scores candidates; the target generates them. Stronger task understanding motivates the choice, without guaranteeing better guidance. TextGrad §3.3

In an API deployment, auxiliary usage is an explicit service expense. Monetary cost depends on prices and tokens; total FLOPs measures resource use. We report role-level usage and include all calls in the compute comparison.

2. Measuring all inference components

We record full EMO K4 and BoN-10 executions on 10 fixed MMLU-Pro questions (selection seed 20261113), with seeds 0/1/2: 30 trials per target–method pair. Target sampling uses T=0.6, top-p=0.95; backward/rewriting use T=0.7, top-p=0.95; scoring is greedy. RTX 4090 D GPUs run targets; four RTX A6000 GPUs host Qwen3.5-122B-A10B-FP8. Output caps are 12,288 for Llama and 24,576 for Qwen, matched across methods.

F^total=F^target+F^score+F^backward+F^rewrite.\widehat F_{\rm total}=\widehat F_{\rm target}+\widehat F_{\rm score}+\widehat F_{\rm backward}+\widehat F_{\rm rewrite}.

FLOPs use recorded forward shapes and API usage with architecture-based accounting. Both EMO streams are charged; shared output tokens count once. Score and returned rationale form one fully charged call. Entries are three-seed arithmetic means, averaging the same 10 questions within each seed. EMO K4 includes five candidates and terminal updates; BoN has ten. Auxiliary tokens sum all roles.

Target / methodTarget input / output tokensAuxiliary input / output tokensEstimated PFLOPs/query
Llama-3.1-8B / EMO K44551.2 / 3935.433713.2 / 9091.00.942216
Llama-3.1-8B / BoN-102580.0 / 10040.818944.5 / 4957.40.619932
Qwen3-8B / EMO K44645.3 / 17378.777158.3 / 8840.52.230011
Qwen3-8B / BoN-102443.0 / 37218.551311.1 / 5404.61.724378

3. Comparing performance within BoN-10's total budget

Using mean cost alone, we select the largest tested EMO depth fitting BoN-10's budget. Answer-ready accounting charges generation, scoring and required context updates, excluding unused terminal backward/rewrite calls. K3 yields four candidates. Costs and accuracies average seeds 0/1/2 over the same questions; candidates are not pooled across seeds.

TargetMethodEstimated PFLOPs/queryCost / BoNAccuracy (%)
Llama-3.1-8BBoN-100.6199321.00063.33
Llama-3.1-8BSC-10, score tie-break0.6199321.00063.33
Llama-3.1-8BEMO K30.5986790.96663.33
Qwen3-8BBoN-101.7243781.00066.67
Qwen3-8BSC-10, score tie-break1.7243781.00070.00
Qwen3-8BEMO K31.5544190.90170.00

SC votes over extracted answers, breaking ties by summed judge scores, then earliest occurrence. Unextractable answers abstain; all-abstain pools use the first response. BoN takes the highest-scoring candidate. Both reuse ten precomputed scores, so coverage and model-call costs coincide; accuracy need not. CPU voting/sorting is outside model FLOPs; SC was not independently timed.

EMO matches SC's point accuracy at 0.966×/0.901× BoN cost (Llama/Qwen). EMO−BoN differences are 0.00/+3.33 pp, with question-cluster 95% CIs [−30.00, 30.00]/[−10.00, 16.67]. There are 10 independent questions.

4. End-to-end time

Full EMO K4 / BoN-10 wall-time ratios are 2.765/1.890 (Llama/Qwen), including API waits and terminal updates. They do not measure budget-matched K3 latency.

For comparison within context optimization, the table gives EMO/TextGrad ratios for full R0–R4 runs from the same study: geometric means of 30 paired question–seed ratios.

Target modelTrial wall-time ratio [95% CI]Target-generation time ratioAPI round-trip time ratioOutput-token ratio
Llama-3.1-8B1.067 [1.026, 1.110]1.1891.0580.988
Qwen3-8B1.117 [0.951, 1.302]1.3310.9981.048

Trial time includes all roles and terminal updates; API time includes queue/network waits. Output tokens sum R0–R4, counting the shared sequence once. CIs resample 10 question clusters, retaining all seeds (20,000 draws; seed 20261115). Judge concurrency is 1 for seeds 0/1 and 4 for seed 2, matched within pairs. Shared-device/recovery effects remain in these deployment timings.

AuthorsOpenReview ↗

Response for W1&Q1 (Part II): Full-Set Estimates and Hybrid Allocation

The ten-question study measures execution directly. We now examine full-dataset coverage and cost, then test whether the order of adaptation and sampling improves the candidate pool.

1. Comparing full-dataset coverage and total cost

This comparison asks whether iterative adaptation finds correct candidates with fewer attempts, and what total computation those attempts require. Pass measures whether the pool contains a correct answer, independently of which candidate the selector chooses.

The table reports historical Llama-3.1-8B/Qwen3-8B results on MMLU-Pro (560) and GPQA-D (198), retaining original inputs/grading/selections. Target uses T=0.6, p=0.95, seeds 0/1/2; EMO backward/rewrite use T=0.7, p=0.95; scoring is greedy.

Costs use 2Pactive(I+O)2P_{\rm active}(I+O) per call: active parameters and input/output tokens, charging both EMO streams and all roles. Missing auxiliary outputs use sampling-regime calibration; unused terminal updates are excluded. These are historical estimates, separate from Part I's recorded-execution accounting.

The table compares each displayed EMO depth with BoN-10. Within/Above indicates whether its mean cost fits BoN's budget. Costs and Pass are mean ± population SD over seeds 0/1/2 (ddof=0), after averaging questions within each seed; candidate pools are never merged across seeds.

DatasetTarget modelEMO configBoN/SC estimated PFLOPs/queryEMO estimated PFLOPs/queryCost ratio (EMO / BoN)Budget statusBoN/SC Pass@10 (%)EMO Pass@K+1 (%)Pass difference (pp)
MMLU-Pro (560)Llama-3.1-8BK30.554703 ± 0.0029690.650781 ± 0.0447591.173Above77.44 ± 0.3080.18 ± 1.31+2.74
MMLU-Pro (560)Qwen3-8BK41.365642 ± 0.0063881.517410 ± 0.0065551.111Above82.56 ± 0.3084.64 ± 1.68+2.08
GPQA-D (198)Llama-3.1-8BK20.721599 ± 0.0022260.651973 ± 0.0119520.904Within61.78 ± 1.6762.12 ± 1.24+0.34
GPQA-D (198)Qwen3-8BK43.152439 ± 0.0159993.260337 ± 0.0144391.034Above80.13 ± 0.2479.12 ± 0.63-1.01

BoN/SC share pools and precomputed scores, giving the same coverage and model-call costs. Ratios and differences compare seed means. SD reflects seed variation, not calibration uncertainty. MMLU-Pro gains coverage at higher cost; Llama GPQA K2 gains within budget.

2. Testing when to adapt context and when to sample

We next test whether adapting context before independent sampling improves the pool, reversing their order at fixed target-generation composition and candidate count:

  • A: BoN → EMO. Sample five responses under the initial context, select the highest-scoring one, and apply textual backward/rewriting to initiate two steered refinement rounds.
  • B: EMO → BoN. Generate R0 and two steered refinements, freeze the context obtained by backward/rewriting from R2, and sample four additional responses.

Both retain seven candidates from five ordinary and two dual-stream generations: nine nominal equivalents. This matches composition, not total FLOPs; total-budget comparisons appear above and in Part I.

The table compares the hybrids' coverage and final selection with BoN-9 and full EMO. Original results cover Llama-3.1-8B, Llama-3.2-3B, and Qwen3-8B on MMLU-Pro (560), GPQA-D (198), and MATH-500 (500). Each seed equally averages nine model–dataset combinations; entries are mean ± population SD over seeds 0/1/2 (ddof=0). All methods use nine generation equivalents.

MethodCandidatesTarget-generation equivalentsMacro accuracy (%)Macro Pass (%)
BoN-99956.16 ± 0.3273.24 ± 0.21
A: BoN → EMO7960.09 ± 0.5678.32 ± 0.20
Full EMO5961.15 ± 0.3478.66 ± 0.57
B: EMO → BoN7961.65 ± 0.0780.10 ± 0.23

A improves accuracy and Pass over BoN-9 in all nine combinations. Accuracy gains are +3.93 pp [3.10,4.77] for A and +5.49 pp [4.19,6.77] for B; B−A is +1.56 pp [0.46,2.65]. Paired 95% CIs resample question clusters within tasks, retaining all seeds/models.

B's four new samples alone gain 6.36 pp macro accuracy over four initial-context samples [4.96,7.77]. Adaptation remains charged. This supports adaptation and subsequent sampling as complementary operations.

Reviewer_XYHLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EMO, a training-free inference framework that alternates between question-specific context optimization and token-level logits steering. Feedback from an external judge is used to revise the system context. During subsequent response generation, the next-token distributions under the revised and preceding contexts are compared at the same generated prefix. This contrast is transformed through the softmax Jacobian to construct a steering direction, which is then subject to probability-based constraints before being applied to the logits. The resulting response becomes the basis for feedback in the next context update, and the judge selects the final answer from all generated candidates. The paper interprets this process in logit space and provides a theoretical result showing that, under specified assumptions, a single round of updates reduces the KL divergence to the target distribution.

Critical strengths

1. The paper coherently connects feedback-driven context adaptation with decoding control and provides supporting empirical evidence.

EMO constructs its steering signal from the actual change in the output distribution induced by a question-specific context revision, and then uses the resulting response to guide subsequent optimization. This establishes a concrete connection between natural-language feedback and token-level intervention. The evaluation across eight models is also reasonably broad. In particular, on MMLU-Pro, all eight models improve over context-only optimization, with an average gain of 3.76 percentage points.

2. The analysis distinguishes the ability to generate correct candidates from the ability to select them.

The paper reports both judge-selected accuracy and oracle coverage. Oracle coverage is the fraction of questions for which the candidate pool contains at least one correct response. This distinction helps assess whether performance improvements arise from better candidate generation or better answer selection.

Critical weaknesses

The value of steering relative to its additional computational cost is not sufficiently established.

The comparison against context-only optimization is useful, but it matches the number of optimization rounds and candidates rather than computation. With four refinement rounds, EMO uses nine target-model generation equivalents, whereas context-only optimization uses five. These results alone do not establish whether allocating additional resources to steering is more beneficial than increasing the number of context-refinement rounds or exploring multiple refinement trajectories. The Best-of-N comparison partially addresses this issue. However, as the authors acknowledge, it excludes feedback and context-rewriting costs, differences in response length, and end-to-end latency. Comparisons against a stronger context-only baseline that uses the additional resources, or against a simpler method that directly amplifies the logit difference between contexts, would make a more convincing case for the proposed mechanism. The current evidence supports the usefulness of the full system more strongly than the practical benefits of the additional steering module.

Questions for authors

1. Clarification and scoring validation

How many mathematical predictions are marked correct solely by the substring fallback? Could you audit these cases and report results using only strict numerical and symbolic equivalence checks? Please also clarify precisely what “removing decimal points” means in the GSM8K scoring implementation and whether distinct numerical values could become identical after normalization.

2. Additional evidence on reproducibility

Could you repeat the EMO–TextGrad comparison using paired runs that control randomness in feedback and context rewriting, and report uncertainty in the accuracy differences?

3. Additional evidence on computational value

How do the results change when EMO’s additional computational resources are instead allocated to more context-refinement rounds or multiple refinement trajectories? Reporting target-model token usage, judge and context-rewriting model usage, and end-to-end latency would help assess whether the performance improvements justify the additional resources.

11 comments
AuthorsOpenReview ↗

Overview of Our Responses to Weaknesses and Questions

Thank you for recognizing our methodology of connecting context adaptation with decoding control under EM-like framework and the breadth of our evaluation. Below, we address each concern raised in the Weaknesses and Questions sections.

We organize your cost-related weakness into four points (W1–W4) for reference:

  • (W1) Matching computation. The original comparison with context-only optimization matches the number of optimization rounds and candidates, but not the computational budget.
  • (W2) Allocating additional resources. Does allocating additional resources to steering offer greater benefits than increasing context-refinement rounds or exploring multiple refinement trajectories? (Addressed together with Q3.)
  • (W3) Accounting for inference costs. The Best-of-N comparison requires fuller accounting of feedback generation, context rewriting, response-length differences, and end-to-end latency. (Addressed together with the cost-reporting requests in Q3.)
  • (W4) Validating the steering mechanism. How does the proposed steering mechanism compare with a simpler method that directly amplifies the logit difference between contexts?

We also address your Questions as follows:

  • (Q1) Clarification and scoring validation. We examine substring-only acceptances, report numerical and symbolic equivalence checks, and clarify whether GSM8K decimal normalization can map distinct numerical values to the same representation. (Addressed in the joint response to Q1&Q2.)
  • (Q2) Additional evidence on reproducibility. We clarify the decoding configurations, report paired runs controlling randomness in feedback and context rewriting, and quantify uncertainty in the EMO–TextGrad accuracy differences. (Addressed in the joint response to Q1&Q2.)
  • (Q3) Additional evidence on computational value. We examine alternative allocations of EMO’s additional resources and report target-model and auxiliary-model usage. (Addressed jointly with W2 and W3; end-to-end runtime measurements are reported under W1.)
AuthorsOpenReview ↗

Response for W1: Comparison Design and Steering Overhead

You propose to distinguish matching optimization rounds and candidate counts from matching computation. Our original comparison fixes the target model, judge, context-optimization procedure, refinement rounds, and candidate count to evaluate the incremental effect of steering. With four refinement rounds, both methods produce five candidates, whereas EMO uses nine nominal target-model generation equivalents and TextGrad uses five. We supplement this comparison with the following latency measurements. And that explains the rationality of our comparison between EMO and Textgrad.

1. Batched execution of the two context streams.

At each decoding step, EMO evaluates the old-context and new-context streams in a single batched forward pass. The streams maintain separate KV caches and compute logits at the same generated prefix. Steering selects one next token, which is appended to both streams. Thus, they share one generated token sequence, producing one candidate per refinement round. At that point, the latency of EMO and Textgrad should be comparable.

Both streams are charged in FLOPs accounting. Batched execution means that generation-equivalent ratios do not directly determine latency ratios.

2. Single-round target latency under fixed and natural output lengths.

For the fixed-length evaluation, we used 16 MMLU-Pro questions, BF16 precision, and greedy decoding on an RTX A6000. We repeated Round 1 (R1) response generation three times per question–method pair, fixing the output length at 128 tokens. Both methods used the same GPU for each model, with warm-up and alternating execution order.

Target modelTextGrad latency (s)EMO latency (s)Latency ratio (EMO / TG)Paired 95% CI
Llama-3.1-8B3.5113.8391.094[1.068, 1.124]
Qwen3-8B4.3994.5321.030[1.015, 1.046]

For the natural-length evaluation, we repeated R1 response generation five times per question–method pair, using the same questions, hardware, and BF16/greedy configuration. Each repetition followed the original stopping criteria, with a maximum output length of 8,192 tokens.

Target modelMean output tokens (TG / EMO)TextGrad latency (s)EMO latency (s)Latency ratio (EMO / TG)Paired 95% CI
Llama-3.1-8B919.38 / 961.6926.0630.331.164[0.330, 4.455]
Qwen3-8B3078.00 / 2941.8199.70102.831.031[0.813, 1.210]

Both tables measure one R1 generation, with repeated timing measurements. The fixed-length evaluation controls for output length; the natural-length evaluation includes response-length variation. Ratios use mean target-only latencies, with question-paired bootstrap intervals after averaging repetitions within each question.

3. Complete-pipeline latency and its components.

We also measured complete R0–R4 trajectories on a separate set of 10 MMLU-Pro questions (question-selection seed 20261113) with run seeds 0, 1, and 2, yielding 30 paired trials per model. Each trial generates five candidates and executes the associated auxiliary calls. Target sampling used temperature 0.6 and top-p 0.95; textual backward and context rewriting used temperature 0.7 and top-p 0.95; candidate scoring was greedy. Target generation used RTX 4090 D GPUs; the auxiliary model ran on four RTX A6000 GPUs. Output caps were identical between methods within each target: 12,288 tokens for Llama and 24,576 for Qwen. These sampled trials are separate from the greedy single-round evaluations.

For each component, we calculate the EMO/TextGrad ratio within each question–seed pair and report its geometric mean across the 30 pairs.

Target modelTrial wall-time ratio [95% CI]Target-generation time ratioAPI round-trip time ratioOutput-token ratio
Llama-3.1-8B1.067 [1.026, 1.110]1.1891.0580.988
Qwen3-8B1.117 [0.951, 1.302]1.3310.9981.048

Trial wall time covers each complete successful run, including target generation, scoring, textual backward, rewriting, and the terminal context update. API round-trip time includes queueing and network waits. The output-token ratio counts response tokens across R0–R4, with EMO’s shared sequence counted once. Confidence intervals resample the 10 questions as clusters, retaining all three seeds.

Judge concurrency was limited to one for seeds 0/1 and four for seed 2, with the same limit within each method pair. Shared-device and recovery effects are retained; these measurements characterize the recorded deployment rather than isolated server computation.

The complete token/FLOPs accounting and comparisons using more refinement rounds or multiple trajectories are presented in our response to Q3.

Taken together, these results show that the two methods have comparable per-round latency.

AuthorsOpenReview ↗

Response for W2&Q3 (Part I): Full-Pipeline Cost and Performance of EMO vs. TG

We report retrospective main-experiment FLOPs estimates and a separate 10-question study recording runtime and model usage. Part II examines alternative allocations of this computation.

1. Retrospective accounting for the main experiments.

Accounting covers eight targets and six datasets, including target generation, scoring with its returned rationale, textual backward, and context rewriting. For each call,

F^call=2Pactive(I+O),\widehat F_{\mathrm{call}}=2P_{\mathrm{active}}(I+O),

where I,OI,O are input/output tokens. Dense targets use their text-model parameter counts; Qwen3.5-122B-A10B uses approximately 10B active parameters. A steered round costs 2Pt(Iold+Inew+2O)2P_t(I_{\mathrm{old}}+I_{\mathrm{new}}+2O): both streams contribute computation, while token usage counts the shared output once. Per-query costs sum all roles and rounds before averaging.

The table covers 560 MMLU-Pro questions, all roles greedy, five candidates per method. Acc is judge-selected accuracy; Pass@5 indicates at least one correct candidate. We retain original correctness and historical selections. Costs are retrospective PFLOPs/query (101510^{15} FLOPs), including terminal context updates.

Target modelTG Acc (%)EMO Acc (%)TG Pass@5 (%)EMO Pass@5 (%)TG estimated PFLOPs/queryEMO estimated PFLOPs/queryFLOPs ratio (EMO / TG)
DeepSeek-R1-0528-Qwen3-8B66.9668.2178.2178.933.1993.6931.154
Llama-3.1-8B53.9363.9356.6174.461.0791.1621.078
Llama-3.2-3B45.7153.0448.5762.501.0511.0681.016
Qwen2.5-7B61.6165.5466.0775.890.7380.8331.127
Qwen3.5-9B63.5765.0065.0077.682.3862.7981.173
Qwen3-14B73.5775.1874.8281.251.4051.6761.192
Qwen3-32B72.3274.2973.3979.641.8872.5541.353
Qwen3-8B64.8267.5066.9675.711.5261.6921.109

Acc and Pass@5 improve for all eight models. Cost ratios are 1.016–1.353, with seven below 1.2. Llama-3.1-8B gains 10.00 pp accuracy at 1.078× estimated cost; Qwen3-8B gains 2.68 pp at 1.109×.

Saved responses are retokenized; missing auxiliary outputs are calibrated from 3,136 greedy calls. These first-order matrix-operation estimates do not fully capture attention, communication, or calibration uncertainty. They are distinct from the sampled execution study below.

2. Instrumented measurements on 10 questions.

We completed 120 EMO/TG trajectories: 10 shared MMLU-Pro questions × two targets × two methods × seeds 0/1/2. Target sampling used T=0.6, top-p=0.95; backward/rewriting used T=0.7, top-p=0.95; scoring was greedy.

We recorded forward-pass shapes, tokens, auxiliary API usage, and wall time. FLOPs use architecture-based accounting of those records, including both EMO streams and active auxiliary components. The table gives three-seed arithmetic means, averaging the same 10 questions within each seed (30 trials per target–method pair). Each trial produces five candidates and includes terminal updates; auxiliary usage sums scoring, backward, and rewriting.

Target / methodTarget input / output tokensAuxiliary input / output tokensEstimated PFLOPs/query
Llama-3.1-8B / TG2587.9 / 3986.233551.0 / 8437.00.843871
Llama-3.1-8B / EMO4551.2 / 3935.433713.2 / 9091.00.942216
Qwen3-8B / TG2893.7 / 15652.772061.2 / 8605.31.794904
Qwen3-8B / EMO4645.3 / 17378.777158.3 / 8840.52.230011

EMO's mean full-pipeline cost is 1.117× TG for Llama and 1.242× TG for Qwen, including actual response lengths and all roles. These records support the budget-matched comparisons in Part II.

Latency is directly measured. W1 reports paired geometric-mean wall-time ratios of 1.067 [1.026, 1.110] for Llama and 1.117 [0.951, 1.302] for Qwen (95% CIs), including API queueing and network waits.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions of PDF

Dear AC and reviewers,

Thank you for the careful feedback and constructive suggestions. We have revised the paper, keeping the main text within 10 pages and documenting the additional evidence in the appendices.

Main paper

  1. Hybrid experiments (§4.6; Table 4 and Figure 2). We compare BoN→EMO and EMO→BoN at matched switching points and target-generation budgets. These experiments examine whether feedback-guided refinement improves on continued sampling and whether an optimized context benefits subsequent independent samples, addressing the value of adaptation and the allocation of inference resources.

  2. Presentation and navigation. We updated the abstract, introduction, and conclusion, and added pointers from the relevant experimental discussions to the supplementary evidence.

Appendices

  1. Concrete execution examples (A.0, D.4). We trace shared context guidance, ordinary-decoding errors, and corresponding steered responses. A structured case audit addresses the proposed guidance–execution gap while distinguishing illustrative evidence from population-level frequency estimates.

  2. Complete cost accounting and resource allocation (C). We add fixed-length, natural-length, and full-pipeline timing, alongside token accounting and estimated FLOPs for all roles. Comparisons include BoN/SC budgets, additional TextGrad rounds, and multiple trajectories. Measured latency and estimated compute are distinguished, as are greedy, historical sampling, and newly instrumented experimental regimes.

  3. Mechanism controls (D.1–D.2). We add open-loop replay versus closed-loop updates, longer TextGrad trajectories, and direct-contrast versus Jacobian steering under shared contexts, masks, and tuning budgets, supplemented by no-mask, multiround, and output-length controls.

  4. Differentiator–selector disentanglement (D.3). We cross selectors on frozen candidate pools and report coverage, self-judge results, and direct strong-model solving, separating candidate-generation effects from final selection.

  5. Reproducibility and scoring (E). We clarify role-specific decoding settings and provide paired confidence intervals, sampling seeds, scoring-sensitivity analyses, and input-provenance checks. The TextGrad and ACE reference corrections are also included.

These additions clarify the mechanisms, computational trade-offs, and scope of the empirical conclusions, and make the supporting evidence easier to inspect. We thank the AC and reviewers again for helping strengthen the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.