acceptodds
Under review as a conference paper at ICLR 2027

Where Should Learning Live? Regret Analysis for Text, Parameter, and Joint Adaptation

Abstract

Language-model systems can select prompts and parameter adapters repeatedly during deployment, using feedback to guide later choices. Earlier choices may change the context or environment inherited by later configurations. We study selection among preconstructed candidates and compare cumulative loss with the best fixed prompt–parameter pair used from the beginning. This excess loss separates into three terms: selection regret within the available family, the gap from restricting that family, and the benefit or harm of the actual deployment history. Selection is evaluated using the history each candidate would have produced if used throughout, which separates candidate quality from the effects of switching. When loss functions are fixed independently of selector randomness, past choices affect loss for only a bounded number of rounds, and every candidate's own-history loss is revealed, limiting switches yields sublinear expected regret within the selected family. We also quantify the costs and additional assumptions of independent probing and deployment-only feedback. Controlled experiments show that fewer switches can either lower or raise deployed loss, depending on whether inherited histories are harmful or helpful. With Qwen3.5-9B on Spider, text-only selection has lower within-family regret than joint selection, yet produces more failures because its candidate family excludes stronger configurations.

Primary area:
learning theory
Keywords:
Regret Analysis for Text Adaptation, Regret Analysis for Parameter Adaptation, Regret Analysis for Joint Adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_EJZ4OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This work studies online selection among preconstructed prompt and parameter configurations when earlier choices may affect subsequent deployment states. It compares text-only, parameter-only, and joint adaptation by decomposing excess loss relative to the best fixed joint configuration into within-family selection regret, family restriction, and signed state effects. This work combines exponential weights with low-switch sampling to derive conditional regret guarantees and examines the additional assumptions and costs of weaker feedback. Controlled experiments and Qwen3.5-9B evaluations on Spider and SParC illustrate how candidate availability and deployment history affect performance.

Critical strengths
  1. This work separately accounts for selection regret, candidate-family restriction, and state effects, providing a useful basis for comparing different adaptation modes.

  2. By matching the marginal distributions of independent and lazy sampling, this work isolates deployment-history effects while holding expected diagonal selection loss fixed. This explains why reducing switches can either improve or worsen performance.

Critical weaknesses
  1. The Spider experiments and restricted-feedback SQL replays use one fixed evaluation order per benchmark. Because online selectors update from preceding observations, cumulative loss and the relative ranking of selectors may depend on this order. Replaying the existing Spider and cleared-history SParC loss matrices under several task orders would help establish the robustness of the reported comparisons.

  2. The sensitivity of the restricted-feedback results to the online selection learning rate remains unclear. This work compares joint and coordinatewise selection under different learning rates, but the feedback comparisons in Appendices C.5 and C.9 primarily use the theory-prescribed rate for each regime, without reporting a systematic sensitivity analysis around these settings.

Questions for authors

Please address the concerns raised in Weaknesses

4 comments

We appreciate your valuable comments. We have revised our submission in response to your review, with changes highlighted in red. Below, we provide detailed responses to your questions and concerns.

The Spider experiments and restricted-feedback SQL replays use one fixed evaluation order per benchmark. Because online selectors update from preceding observations, cumulative loss and the relative ranking of selectors may depend on this order. Replaying the existing Spider and cleared-history SParC loss matrices under several task orders would help establish the robustness of the reported comparisons.

We added task-order analyses on the original full benchmarks for the three Qwen3.5-9B seeds and extended them to Qwen3.5-9B, Qwen3.5-4B, and Ministral 3 8B, with nine adapter-training seeds per model. Each analysis compares the original order with 20 predetermined permutations; SParC permutations reorder whole dialogues while preserving their internal turns. Candidate pools, priors, learning rates, and comparators remain fixed. Section 7 summarizes the expanded results, with details in Appendix F.2.

The extension uses predefined subsets of 50 Spider questions and 50 complete SParC dialogues in each separate development and held-out partition, selected independently of model outcomes. Development SParC has 160 turns and held-out SParC has 139 turns. Every subset covers all 20 databases, with two or three questions or dialogues per database; each model/seed bank contains 24 prompt–parameter pairs. Cleared-history SParC removes earlier generated SQL while retaining the schema and user utterances. These subset results supplement the completed original full-benchmark analysis.

Full-diagonal exponential weights (EW) observes each candidate's loss on its own history and updates selection probabilities from preceding losses. Follow-the-leader (FTL) chooses the candidate with the lowest cumulative preceding loss; development-fixed selection chooses by development loss and freezes that candidate. The table reports exact expected joint-family held-out failure counts averaged across nine training seeds. Ranges summarize the 20 permuted-order means.

ModelBenchmarkEW, original orderEW, permutation rangeFTL, permutation range
Qwen3.5-9BSpider15.50415.118–15.61614.793–17.323
Qwen3.5-4BSpider17.44717.054–17.43516.129–18.798
Ministral 3 8BSpider14.89914.735–15.04113.889–15.288
Qwen3.5-9BCleared SParC51.88051.516–52.15251.210–54.851
Qwen3.5-4BCleared SParC54.72154.592–55.15953.116–54.989
Ministral 3 8BCleared SParC58.61458.451–58.72656.775–58.670

EW has a narrower permutation range than FTL in all six model/benchmark comparisons. FTL changes rank relative to development-fixed selection in 7 of 20 Qwen3.5-9B Spider permutations and 2 of 20 Ministral SParC permutations.

The restricted-feedback replays also show small changes in cumulative loss. Sparse EW probes one candidate per round through an evaluator independent of deployment; deployment-only selection updates from executed losses. Across the six settings, permutation-range widths are 0.011–0.144 failures for sparse independent EW and 0.139–0.269 for deployment-only selection with block length 1. These estimates use 256 selector repetitions per cell. The ranges describe the tested orders and include Monte Carlo variation for stochastic methods.

The sensitivity of the restricted-feedback results to the online selection learning rate remains unclear. This work compares joint and coordinatewise selection under different learning rates, but the feedback comparisons in Appendices C.5 and C.9 primarily use the theory-prescribed rate for each regime, without reporting a systematic sensitivity analysis around these settings.

We added a systematic learning-rate sweep for full- and restricted-feedback methods and comparisons at shared numerical rates. We also report development-tuned full EW. Section 7 summarizes the findings, and Appendix F.3 reports the complete grids, tuning protocol, and uncertainty estimates. The theoretical bounds retain the learning rate η\eta explicitly; the sweeps examine sensitivity around the rates that balance those bounds.

We swept multipliers {1/8,1/4,1/2,1,2,4,8}\{1/8,1/4,1/2,1,2,4,8\}, with a separate zero-rate diagnostic, for the full- and restricted-feedback methods. The SQL analysis uses the three models, nine seeds, and predefined subsets described above. The table gives mean joint-family held-out failures across nine seeds; each arrow compares a method's prescribed rate with eight times that rate. Sparse EW draws each deployment choice afresh from its current probabilities, and deployment-only selection uses block length 1. For the last column, each bank's EW multiplier minimizes average development loss over the original order and four development permutations. We freeze the multiplier and rate formula, then evaluate the numerical rate at the held-out horizon.

ModelBenchmarkFull EW: prescribed → ×8Sparse EW: prescribed → ×8Deployment-only: prescribed → ×8Development-tuned full EW
Qwen3.5-9BSpider15.504 → 15.75514.894 → 14.89914.880 → 14.86515.173
Qwen3.5-4BSpider17.447 → 16.93616.724 → 16.72916.732 → 16.78116.761
Ministral 3 8BSpider14.899 → 14.41916.719 → 16.65816.605 → 16.50714.539
Qwen3.5-9BCleared SParC51.880 → 52.48951.920 → 51.87251.963 → 51.70652.185
Qwen3.5-4BCleared SParC54.721 → 53.78357.707 → 57.40357.831 → 56.91154.079
Ministral 3 8BCleared SParC58.614 → 58.25363.779 → 63.08763.793 → 62.53058.234

Development tuning lowers mean held-out loss in five of six settings; Qwen3.5-9B cleared SParC is the exception. This evaluation uses only development outcomes to choose the multiplier, while the held-out sweep describes sensitivity.

Shared numerical rates separate the effect of feedback from the rate prescription. On Qwen3.5-9B Spider, full EW, sparse EW, and deployment-only selection yield 14.935, 14.894, and 14.880 failures at the sparse-diagonal rate, versus 15.504, 14.899, and 14.865 at the full-diagonal rate. Rate choice therefore explains part of the observed separation. At the sparse-rate anchor, the conditional Monte Carlo standard errors of the sparse and deployment-only means are 0.005 and 0.037 failures, respectively; the full EW expectation is exact.

Remark: Please let us know if you have any remaining concerns or questions. If we have addressed your concerns, we would appreciate it if you could reconsider your score. Thank you again for your constructive feedback.

Thanks for the response. I will maintain my positive score.

Thank you for your acknowledgment and positive evaluation of our work!

Reviewer_h8NQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper studies online selection among pre-constructed prompt–adapter configurations when earlier choices can affect later outcomes through persistent state. It decomposes excess deployed loss, relative to the best fixed joint configuration, into diagonal selection regret, family restriction, and a signed state effect. It provides prior-based regret bounds for exponential weights, a separation between joint and coordinatewise selection, and policy-regret bounds for lazy EW under bounded influence and counterfactual feedback, together with rates under weaker feedback. Experiments on synthetic tables and on Spider and SParC with Qwen3.5-9B illustrate each component.

Critical strengths
  1. Clear conceptual lens. The decomposition shows that low regret within the available family, convergence to the optimal choice, and infrequent switching do not by themselves imply low deployed loss.
  2. Solid theoretical grounding. The theory is rigorous and well scoped, with explicit assumptions, and covers several feedback regimes (full counterfactual feedback, independent probes, and deployment-only feedback).
  3. Relevant insight. The separation between joint and coordinatewise selection is relevant to practical methods that alternate between prompt and weight optimization.
  4. Transparent reporting. The paper includes decomposition checks and uncertainty estimates, and openly reports negative findings, such as EW not being the best selector on Spider.
Critical weaknesses
  1. Mismatch between the title and the content. The paper only partially answers the question posed in its title: it studies which candidate to select from a fixed pool rather than where adaptation should occur. The Spider results therefore depend on the constructed candidates rather than on an intrinsic difference between text and parameters. The parameter–joint gap nearly disappears at equal candidate counts (K=4K = 4), and the remaining text gap reflects that text candidates use base weights while the adapters are trained on construction data from the same benchmarks.

  2. Limited theoretical contribution. The analysis relies heavily on existing tools (prior-weighted EW, Shrinking Dartboard, policy regret under bounded memory, mini-batching, and the T2/3T^{2/3} lower bound), and Proposition 1 is close to an accounting identity. It would help to distinguish the paper's own theoretical contributions more concretely.

  3. Strong assumptions relative to the motivating setting. The diagonal oracle requires every candidate's own-history loss at every round, which would require costly cloning or simulation in stateful systems. The motivating persistent-workspace coding agent is unlikely to satisfy bounded influence and is not evaluated. Under deployment-only feedback, the rate degrades to T2/3T^{2/3} and the causal-reaction term is not controlled.

  4. Limited empirical evidence for the state-related claims. Spider is stateless, and the SParC state effect is only about 0.2 pp (roughly 1.5 MCSE), so the switching results rely mainly on synthetic tables designed for this purpose. On Spider, a development-selected fixed candidate and follow-the-leader also outperform EW. The study uses one model, two related datasets, and three seeds.

Questions for authors
  1. How can the diagonal oracle be obtained in a realistic stateful system, and what is its computational cost relative to deployment?
  2. Beyond comparisons that depend on the constructed candidates, can the analysis offer more practical guidance on whether adaptation should occur in text or in parameters?
  3. Given that a development-selected fixed candidate and follow-the-leader perform better on Spider, what is the practical benefit of adopting EW or lazy EW in deployment?
9 comments

We appreciate your valuable comments. We have revised our submission in response to your review, with changes highlighted in red. Below, we provide detailed responses to your questions and concerns.

Mismatch between the title and the content. The paper only partially answers the question posed in its title: it studies which candidate to select from a fixed pool rather than where adaptation should occur.

We changed the title to “Regret Analysis for Text, Parameter, and Joint Selection in Language Models”. Throughout the manuscript, selection means choosing among preconstructed configurations; adaptation describes the broader process, including candidate construction.

Text-only selection fixes the parameters, parameter-only selection fixes the text, and joint selection can change both. We compare their cumulative deployed loss with the best fixed prompt–parameter pair, used throughout evaluation on its own generated history. This scope is now explicit in the title, abstract, and introduction.

The Spider results therefore depend on the constructed candidates rather than on an intrinsic difference between text and parameters. The parameter–joint gap nearly disappears at equal candidate counts (K=4K=4), and the remaining text gap reflects that text candidates use base weights while the adapters are trained on construction data from the same benchmarks.

We added the suggested development-selected comparison in Section 7 and Appendix F.4: retain the base configuration and select three others with the lowest development loss, with deterministic tie-breaking. Each pool is frozen before held-out evaluation. The pools share the same candidate count, uniform prior, exponential weights (EW) learning rate, and full-joint comparator within each model/seed. The parameter pool already contains exactly four candidates.

The table reports mean expected held-out failures on the 50-question Spider subset across nine training seeds. “Joint, exhaustive” averages all four-candidate joint pools containing the base configuration:

ModelParameter, K=4K=4Joint, exhaustive K=4K=4 averageJoint, development-selected K=4K=4
Qwen3.5-4B16.27716.25516.104
Qwen3.5-9B15.40615.51515.848
Ministral 3 8B15.22816.02714.218

Development-selected joint pools have lower mean loss for Qwen3.5-4B and Ministral. For Qwen3.5-9B, the joint pool has lower selection regret than the parameter pool (1.293 versus 1.518 failures), but larger family restriction (1.778 versus 1.111), yielding higher total loss. Selection regret measures performance relative to the best candidate within a pool; family restriction measures that pool's best-candidate gap to the full joint bank. The comparison therefore separates effective selection from candidate quality at equal pool size.

We also added prompt construction under a declared budget (Section 7; Appendix F.7). For the three completed Ministral construction seeds, local reflective search produces three prompts that join the base prompt in a four-candidate pool at base weights. Each bank receives a 2.798–2.822-hour cap equal to the recorded optimization time of its three adapters; prompt generation, reflection, and serial SQL scoring count toward this cap. Prompt search draws from a pool of 11,735 examples with audited executable references, whereas adapter training used 11,752. Prompts are frozen before development selection and held-out evaluation. On the same 50 Spider questions, mean failures across these three seeds are:

Candidate family, K=4K=4Development-fixed failuresEW failures
Handwritten text25.00021.060
Optimized text17.00017.784
Parameters12.66714.599

The primary development-fixed comparison reduces text failures from 25 to 17. Its optimized-text minus parameter gap is 4.333 ± 4.509 failures; the corresponding EW gap is 3.185 ± 1.982 (mean ± standard deviation of paired seed differences). Construction improves the available text candidates and narrows the gap, while the parameter pool retains lower mean loss under this protocol. We will share the Qwen3.5-4B and Qwen3.5-9B construction results once those runs complete.

Limited theoretical contribution. The analysis relies heavily on existing tools (prior-weighted EW, Shrinking Dartboard, policy regret under bounded memory, mini-batching, and the T2/3T^{2/3} lower bound), and Proposition 1 is close to an accounting identity. It would help to distinguish the paper's own theoretical contributions more concretely.

The abstract and introduction now foreground our joint-versus-coordinatewise separation results, and Section 3 explains their proof mechanisms. Joint sampling draws a complete pair from the EW distribution; the specified coordinatewise sampler instead updates each coordinate once, conditionally on the other, starting from the preceding pair.

The construction uses full loss feedback, a uniform prior, and two choices per coordinate: a suboptimal pair has loss Δ\Delta, changing either coordinate alone raises loss to Δ+b\Delta+b, and changing both reaches zero loss, where Δ,b>0\Delta,b>0 and Δ+b≤1\Delta+b\leq1. Over TT rounds, joint sampling has bounded expected regret while the one-sweep sampler has linear expected regret at a fixed positive learning rate. For Δ>b\Delta>b, a decreasing-rate schedule gives O(1)O(1) versus Ω(T)\Omega(\sqrt{T}) expected regret even though both eventually remain optimal almost surely. The proofs lower-bound persistent trapping and the trapping tail, and establish eventual escape and finitely many departures from the optimum. Thus, a shared target distribution and eventual optimal selection can coexist with different cumulative regret.

Proposition 1 supplies the common-comparator accounting: deployed excess loss equals diagonal selection regret, family restriction, and signed state effect. Diagonal loss evaluates each candidate on its own fixed-candidate history; the signed state effect is the difference between actual deployed loss and this diagonal evaluation. We use established EW and Shrinking Dartboard tools for the state bounds, and established mini-batching for the T2/3T^{2/3} deployment-only rate with an explicit causal-reaction term. The revision distinguishes these ingredients from the separation proofs.

Strong assumptions relative to the motivating setting. The diagonal oracle requires every candidate's own-history loss at every round, which would require costly cloning or simulation in stateful systems.

We clarified the evaluation mechanism required by each guarantee in Section 5. Full diagonal feedback evaluates every candidate on its own trajectory from the common initial condition. Sparse probing requests fewer diagonal observations through an autonomous evaluator independent of deployment. Deployment-only learning uses executed block losses and requires no diagonal evaluator; its guarantee includes a causal-reaction term, controlled under complete-system finite influence.

Our recursive-SParC experiment implements full feedback by generating each fixed candidate's own history under the same dialogue resets. These losses update the selector; separately generated mixed histories measure deployed loss. This protocol distinguishes feedback access from bounded influence: resets limit how long earlier choices affect outcomes, while the separate trajectories supply the oracle. The cost comparison is detailed in the answer for How can the diagonal oracle be obtained in a realistic stateful system, and what is its computational cost relative to deployment?

The motivating persistent-workspace coding agent is unlikely to satisfy bounded influence and is not evaluated.

We replaced the coding-agent example with the recursive-SParC setting evaluated in Sections 6 and 7 (Appendices D.8 and F.5). At each dialogue turn, a selector chooses a preconstructed prompt–parameter pair to generate Structured Query Language (SQL) queries. Earlier generated queries remain in context, so switching can change the inherited history. Generated history resets between dialogues while EW statistics continue. The fixed comparator uses one pair throughout, follows the same resets, and generates its own query history.

Fixed questions and schemas, bounded dialogue length, and complete generated-history resets justify finite influence in this evaluated protocol. Separate fixed-candidate trajectories supply diagonal feedback; mixed-history losses measure state effects without updating EW. The motivating example now matches both the experiment and the assumptions used for its guarantee.

Under deployment-only feedback, the rate degrades to T2/3T^{2/3} and the causal-reaction term is not controlled.

The causal-reaction term is controlled under finite influence of the complete system, and we have added direct measurements on recursive SParC. For KK candidates over TT rounds, let bb be the deployment block length, η>0\eta>0 the learning rate, J=⌊T/b⌋J=\lfloor T/b\rfloor, and r=T−bJr=T-bJ. Let QQ be a fixed comparator distribution independent of learner randomness and μ\mu a positive prior. Its complexity is the Kullback–Leibler divergence CQ=KL(Q∥μ)C_Q=\mathrm{KL}(Q\Vert\mu). Losses lie in [0,1][0,1], and causal responses are driven by external randomness independent of the learner's private draws. Expected policy regret against a candidate drawn from QQ and used throughout on its own history is bounded by the following:

bCQη+ηbKJ2+Γb(Q)+r.\frac{bC_Q}{\eta}+\frac{\eta bKJ}{2}+\Gamma_b(Q)+r.

The first two terms control selection from observed block losses. The causal-reaction term Γb(Q)\Gamma_b(Q) compares deploying the comparator after the learner's prefixes with deploying it from the beginning. If choices cease to affect the complete causal system after mm rounds, then ∣Γb(Q)∣≤min⁡{m,b}(J−1)|\Gamma_b(Q)|\leq\min\{m,b\}(J-1). Balancing the terms yields the stated T2/3T^{2/3} rate for fixed positive mm, KK, and CQC_Q and sufficiently large TT. This connects available feedback to the conditions for sublinear policy regret; the exponent follows established mini-batching.

For the empirical measurement, a deployment-only learner generates its own history, and isolated continuations of the best fixed joint comparator evaluate reaction without influencing the learner. On 50 held-out dialogues (139 turns), the normalized estimates 100Γ^b/(b⌊139/b⌋)100\widehat{\Gamma}_b/(b\lfloor139/b\rfloor) for b=1,2,4,8b=1,2,4,8 are respectively 1.9301.930, 0.6270.627, 1.9321.932, and 1.0531.053 percentage points (pp), averaged over nine Qwen3.5-9B training seeds. Conditional Monte Carlo standard errors (MCSEs) of these means are 0.0900.090, 0.0650.065, 0.0510.051, and 0.0450.045 pp; training-seed standard deviations (SDs) are 0.4510.451, 0.3660.366, 0.5150.515, and 0.4640.464 pp. Fixed repetition budgets are 30 for each of two seeds and 41 for each of the remaining seven. Achieved within-seed MCSEs are 0.095–0.350 pp. Cleared-history controls give exactly zero reaction. Section 5 gives the guarantee, and Appendix F.6 reports these measurements and achieved precision.

Comments on the paper

14 comments

We deeply appreciate the reviewers' constructive feedback. We have revised our manuscript in response to the reviewers’ feedback and highlighted the changes in red. The summary of our revision is described as follows.

  • Title, Abstract, and Section 1: Clarified selection scope and motivation, in response to Reviewer h8NQ.
  • Section 3 and Appendix A: Clarified and summarized theoretical contributions, in response to Reviewers h1RY and h8NQ.
  • Section 4: Added practical guidance, in response to Reviewer h8NQ.
  • Section 5: Clarified feedback requirements and evaluation costs, in response to Reviewer h8NQ.
  • Section 6: Clarified controlled state experiments, in response to Reviewer h1RY.
  • Section 7 and Appendices F.2–F.3: Added task-order and learning-rate analyses, in response to Reviewers EJZ4 and h1RY.
  • Section 7 and Appendices F.4 and F.7: Added candidate-pool and prompt-construction comparisons, in response to Reviewers h1RY and h8NQ.
  • Section 7 and Appendices F.5–F.6 and F.8: Expanded state evaluations, rate tuning, and causal-reaction measurements, in response to Reviewers h1RY and h8NQ.
  • Section 7: Added a SQL summary table, in response to Reviewer h1RY.

In addition, we have provided responses to each reviewer's comments. We hope our revision and answers resolve the reviewers' concerns and questions. If you feel our responses address the issues raised, we would appreciate it if the scores could be reconsidered.

We appreciate your valuable comments. We have revised our submission in response to your review, with changes highlighted in red. Below, we provide detailed responses to your questions and concerns.

The experimental setup is not balanced, and several conclusions are built into the synthetic setting. The text candidates are six hand-written prompts, while the parameter candidates are LoRA adapters trained on in-domain data. The restriction gap may therefore mainly reflect unequal effort in constructing the candidates. Whether fewer switches help or hurt follows directly from the sign of λ\lambda in the synthetic loss dt+λItd_{t} + \lambda I_{t}.

The original text–parameter gap depends on the constructed prompt and adapter banks. Family restriction measures the gap to the best joint candidate, while selection regret measures how well the selector uses its available bank. We have tested additional prompt construction in Appendix F.7; the completed Ministral results and construction-time comparison are reported in our construction-effort answer for Does the restriction gap remain when the text candidates receive comparable effort?

Section 6 now presents the synthetic study explicitly as a controlled test of the signed-state mechanism. Diagonal loss evaluates each candidate on the history it would generate if used throughout. We hold these losses and per-round selection marginals fixed while varying the state coefficient λ\lambda across harmful, beneficial, and zero-state conditions. The sign is controlled by construction; equal expected diagonal loss isolates how temporal coupling changes the state contribution to deployed loss. Recursive SParC supplies the complementary empirical test, where the direction and magnitude of the state effect emerge from generated histories.

The only experiment with actual state carryover (SParC, lines 421–424) shows a small effect, close to the noise level, and covers only one model and one task.

We have extended the state-carryover evaluation to Qwen3.5-4B, Qwen3.5-9B, and Ministral 3 8B, with nine adapter-training seeds per model and 24 configurations per bank: six prompts paired with four parameter configurations. The additional checks use 50 Spider questions and 50 complete SParC dialogues in each of two disjoint development and held-out subsets. SParC contains 160 development turns and 139 held-out turns. Predetermined hashes of dataset identifiers select two or three questions/dialogues from each of all 20 databases, without using model outcomes; every model and seed uses the same subsets. These checks supplement the original full-benchmark results.

Recursive SParC retains earlier generated Structured Query Language (SQL) queries within dialogues and resets that history between dialogues. Both selectors use exponential weights (EW), updated from fixed candidates' own-history losses, with accumulated feedback retained across dialogues. Independent sampling draws afresh each turn; lazy sampling couples successive draws to reduce switches while preserving each turn's selection probabilities. The table reports paired lazy-minus-independent failure-rate contrasts in percentage points (pp) and expected switches per 139-turn sequence:

ModelLoss contrast, mean ± seed SD (pp)Contrast MCSE (pp)Independent switchesLazy switches
Qwen3.5-4B−0.230 ± 0.5820.151131.6934.335
Qwen3.5-9B+0.530 ± 0.5730.152131.9803.988
Ministral 3 8B+0.345 ± 0.4590.198131.1474.357

Each bank uses 16 paired selector repetitions. SD is the standard deviation across the nine training seeds; MCSE is the Monte Carlo standard error of the reported mean contrast, conditional on the evaluated banks and tasks. Switching falls sharply, while the estimated loss differences remain modest and have different signs. Clearing earlier generated queries makes the expected loss contrast exactly zero for all three models. Both history conditions use the same prescribed stateful learning rate, η=0.0858328\eta=0.0858328. This comparison isolates a signed history effect at equal expected diagonal selection loss. The separate deployment-only analysis below measures how a fixed comparator reacts to the learner's history.

The separate development-tuned recursive selection evaluation uses 64 paired repetitions per Qwen bank and 80 per Ministral bank; every reported model-mean contrast achieves conditional MCSE below 0.1 pp. At development-selected rates, lazy coupling reduces expected switches by 83–95%. The tuned-rate answer below and Appendix F.8 give the results; these estimates remain separate from the original 16-repetition comparison above.

The theoretical learning rate η\eta is very small, making EW close to uniform sampling. The independent versus lazy comparison is therefore largely uniform versus lazy-uniform sampling. On Spider, the simple baseline outperforms EW in every family (table 4), suggesting that online selection adds little here.

The independent/lazy comparison isolates temporal coupling at the same marginal selection quality. When both receive the same diagonal losses fixed independently of selection randomness, they have identical expected diagonal loss for any common fixed η\eta. Their deployed-loss difference therefore measures the signed effect of inherited history. This conclusion applies across the learning-rate range, including the near-uniform limit.

The prescribed rate balances a worst-case bound, and the added rate sweep includes a zero-rate diagnostic to distinguish learning from uniform sampling. On the original full Spider benchmark, joint EW has 315.024 mean held-out failures versus 326.306 for uniform selection across three Qwen3.5-9B training seeds; development-fixed selection and follow-the-leader (FTL) have 309.333 and 310.990 failures. Development-fixed selection retains the candidate with lowest development loss; FTL selects the candidate with lowest cumulative preceding loss. On the additional 50-question subset, nine-seed joint EW/uniform means are 17.447/16.699 for Qwen3.5-4B, 15.504/14.870 for Qwen3.5-9B, and 14.899/17.009 for Ministral 3 8B.

Development-fixed selection has lower mean loss than prescribed EW in all six model/benchmark subset settings, making it a strong deployment option. EW supplies a diagonal selection-regret guarantee against fixed comparators without identifying the best candidate beforehand; lazy coupling adds switching control. Section 7 and Appendix F.2 now state this distinction directly. The rate sweep and development-only tuning below quantify how EW's performance changes with the bank, loss sequence, and rate.

Does the restriction gap remain when the text candidates receive comparable effort?

We have tested this directly through the prompt construction under a declared budget. The completed Ministral 3 8B comparison uses three construction seeds and four candidates per family: the base prompt plus three optimized prompts at base weights, a development-selected handwritten-prompt pool, and the existing parameter pool. We construct the optimized prompts through local reflective population search. Each bank receives a 2.798–2.822-hour construction cap equal to its three adapters' recorded optimization time. The cap charges prompt generation, reflection, and serial SQL scoring. It provides a declared time-budget comparison. Prompt search draws from a pool of 11,735 examples with audited executable references, while adapter training used the original 11,752 examples.

Prompts are frozen before development selection and evaluation on the same 50 held-out Spider questions. The primary selector fixes the candidate with lowest development loss; the secondary comparison uses full-information EW with a common uniform-prior rate. Mean held-out failures across the three seeds are:

Candidate family, K=4K=4Development-fixed failuresEW failures
Handwritten text25.00021.060
Optimized text17.00017.784
Parameters12.66714.599

Prompt optimization reduces failures from 25 to 17 under fixed selection and from 21.060 to 17.784 under EW. The optimized-text minus parameter gap is 4.333 ± 4.509 failures for the primary fixed selector and 3.185 ± 1.982 for EW (mean ± paired seed SD). Thus, additional prompt construction substantially narrows the observed gap, while the parameter pool retains lower mean loss under this protocol. Section 7 summarizes these results, and Appendix F.7 gives the protocol and paired comparisons. We will share the Qwen3.5-4B and Qwen3.5-9B construction results once those runs complete.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.