acceptodds
Under review as a conference paper at ICLR 2027

Signed Residuals for Setpoint Bayesian Optimisation: Why an Active-Inference Acquisition Collapses to Greedy

Abstract

Setpoint problems, such as holding a physiological reading at a clinical target while the response drifts, are usually given to Bayesian optimisation by folding the residual into a distance to be maximised. We show that this encoding, rather than the acquisition function, decides whether a preference-based acquisition can work. Our case study is BOBA (Kelly et al., 2025), an active-inference acquisition function for dynamic BO. We prove that its pragmatic term orders candidates by the posterior mean whenever the preferred outcome lies above every prediction, which holds on 94.6% of recorded states, and that its epistemic term is too small to change the decision, so the method acts greedily. Folding guarantees this condition for every target, because it places the preferred outcome at the supremum of the objective. Keeping the residual signed restores the preference term’s ordering, raising the share of states whose candidates bracket the target from 5.0% to 57.7% on a clinical digital twin and from 5.6% to 84.5% on fourteen synthetic setpoint problems. Once the encoding is fixed, the epistemic term no longer helps: in none of eight synthetic cells is it significantly beneficial, and a two-line proximity rule (GREEDY-H) matches or beats every active-inference variant we test. On a digital twin of closed-loop anaesthesia built from 50 patient-specific models fitted to surgical recordings, the proximity rule beats a fixed clinical protocol at every budget, although a forgetting-kernel baseline overtakes it at longer budgets. Where the target is unreachable, arms with an information term degrade 1.9× in location error against 1.1× for greedy and proximity rules. Redirecting BOBA’s epistemic term at the optimum (BOBA-OPT) improves on BOBA but not on GP-UCB.

Primary area:
optimization
Keywords:
Bayesian Optimization, Active Inference, Digital twins, Acquisition Functions

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WfXsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The article considers the very recent BOBA method (Kelly et al 2025, only available on openreview and arXiv) for dynamic Bayesian optimisation where there is time-dependence in the target function. The application that is being considered is where a controller wants to adjust x over time to maintain f(x,t) at a fixed predetermined level f*. A natural way to encode this objective is to maximise -|f(x,t)-f*|. The article makes the following main points:

  1. BOBA as "published" does not correctly balance exploration and exploitation due to a scale mismatch.
  2. BOBA cannot optimise sensibly when the target is the largest (or smallest) possible value that could be achieved.
  3. A modification to BOBA is an improvement
  4. Optimising -(f(x,t)-f*) (i.e. not taking the absolute value) results in better performance. The article also presents a digital twin methodology for controlling the anaesthetic state of a patient through surgery, and runs some experiments against this digital twin (as well as standard BO benchmarks). The results are rather inconclusive.
Critical strengths

The dynamic BO setting is interesting and understudied

Taking on the building of a digital twin based on surgical data, and releasing methodology to allow reproduction of this approach, is valuable

Critical weaknesses

BOBA has, so far as I can tell, not a single citation yet, so it is a bizarre start point for an article.

It is not clear if the scale mismatch reported from the original BOBA "publication" is endemic or simply related to the scale of your experiments

That BOBA, one particular method, cannot optimise well when the target is at the extremal range should surely suggest you try a different BO approach, rather than change the thing you try to optimise.

I cannot ascertain why optimising the signed difference can work. Surely you are now just optimising f(x,t) instead of targeting f*?

Questions for authors

What is going on in the GP and acquisition function? The description lacks precision, so I really struggled to work out what was going on.

In Prop 1, I struggle to see any situation to which it applies. sigma_t will of course vary across x (although I am not sure exactly what you mean by sigma_t since you never define it). Optimising any non-greedy acquisition function will need to trade off mu_t(x) and sigma_t(x).

What is actually the solution here? It reads as if you have tried something, and seen some inconclusive results, and submitted it. That doesn't make for an ICLR paper.

1 comment

Thank you for your comemnts. We have rewritten the parts that were hard to follow.

Why start from BOBA?

  • The finding concerns how a known target is presented to any preference-based acquisition; BOBA is the worked example.
  • The argument covers any preference that is unimodal in the prediction (Sec. 1, Remark 1).

Is the scale mismatch endemic?

  • Without normalisation, yes. R grows quadratically and I only logarithmically, so R's range is a median 3.6x10^3 times larger, and across the published beta range the information term changes only 0.6-15% of decisions.
  • With normalisation the information term decides, but the preference term still carries no target ordering (Sec. 2.2, Sec. 3.2).

Why change what is optimised rather than the method?

  • Folding is the standard way to give Bayesian optimisation a target, and the standard maximisers we compare (GP-UCB, EI, AES) all run on the folded objective.
  • Composite Bayesian optimisation (Astudillo & Frazier, 2019; Uhrenholt & Jensen, 2019) already models the underlying output and applies the distance afterwards, and finds this works better than modelling the distance directly.
  • We show why this matters for preference-based acquisitions. The encoding is a modelling choice that any acquisition can use (Sec. 4).

Optimising the signed difference just optimises f. It does not. The GP models h = g - y*, and GREEDY-H queries the candidate whose predicted h is closest to zero, approaching the target from either side. Fig. 1 shows one state under both encodings (Sec. 4).

The GP and acquisition were unclear. Sec. 2.1 now describes the loop step by step, Sec. 2.2 the acquisition, and App. B the implementation.

sigma_t was undefined, and Prop. 1 never applies because sigma varies.

  • sigma_t is the GP predictive standard deviation (Sec. 2.1), now referenced in the proposition.
  • Remark 1 states when varying sigma can change the ranking.
  • On 6,049 live states, the ranking matches the posterior mean (median rho = +0.997, Sec. 3.2).

What is the solution? Model the signed residual, query the candidate predicted closest to target, and use a forgetting surrogate when the response drifts. On the twin, this approach:

  • beats the clinical protocol and an open-loop replay of clinician dosing;
  • comes within 0.65 BIS of a tuned PI controller's median regret;
  • is, with BOBA-LS on the same surrogate, the only method that keeps every patient's mean noise-free deviation within the clinical band (Sec. 6, Sec. 8).
Reviewer_MJDWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates why BOBA, an active-inference acquisition function for dynamic Bayesian optimization, often behaves like greedy exploitation in setpoint problems. It shows that folding a setpoint objective into a negative absolute-distance function places the target at the objective's upper bound, making BOBA's pragmatic term monotonic in the posterior mean under specified conditions. The authors propose retaining signed residuals and introduce a simple proximity-based acquisition rule (GREEDY-H). Experiments on synthetic benchmarks and a patient-specific anesthesia digital twin demonstrate substantially improved target bracketing and competitive performance against more complex acquisition functions, although the optimization benefits are primarily observed in one-dimensional problems.

Critical strengths

Clear and empirically supported diagnosis. Propositions 1–2 explain how objective encoding can make BOBA's pragmatic term effectively greedy. The analysis is supported by extensive diagnostics, including the degeneracy condition holding in 94.6% of recorded states, a median Spearman correlation of 0.997 on non-flat states, and direct measurements of the epistemic term's contribution to candidate selection.

Careful empirical investigation. The paper uses factorial experiments to distinguish the effects of objective encoding and information-seeking, reports confidence intervals and multiple-testing corrections, and examines sensitivity to surrogate configurations. The authors transparently document experimental limitations and negative results, including settings where their proposed methods underperform existing baselines.

Substantial application-oriented benchmark. The anesthesia digital twin incorporates patient-specific pharmacokinetic/pharmacodynamic models, temporal drift, observation noise, sensor dropout, and unreachable-target scenarios. It provides a useful benchmark for dynamic setpoint optimization and demonstrates that simple proximity-based acquisition can compete with more complex approaches.

Critical weaknesses

However, the theoretical contribution is relatively elementary; the greedy-collapse claim depends on the configuration, and the benefits of signed residuals largely apply only to one-dimensional problems. More importantly, the encoding experiments confound preference geometry with observation-noise modeling, while unverified surrogate configurations weaken the synthetic comparisons. The clinical benchmark also lacks independent counterfactual validation and important practical baselines.

Questions for authors

Propositions 1–2 follow directly from the Gaussian KL expression and assume constant predictive variance across candidates. Moreover, the observed collapse depends strongly on the weighting convention: under BOBA[pub], the full acquisition agrees with the pragmatic term on all 521 checked live states, whereas agreement falls to 4.95% under BOBA[wm] (Appendix A). The pragmatic term itself selects the highest-mean candidate in only 78.7% of live BOBA[pub] states. Thus, the headline claim requires important qualifications regarding the weighting convention and the distinction between approximate and exact greedy behavior. Can you elaborate on that?

Although signed residuals improve target bracketing, this does not consistently translate into better optimization. On the full-dimensional synthetic suite, signed encoding has a significantly negative main effect at T = 30 under version A (−0.201 SD, Table 11), while other full-dimensional comparisons do not establish significant improvements. Furthermore, folding observations after adding Gaussian noise produces non-Gaussian, input-dependent errors that violate the GP's assumed observation model. At the target, the folded observations have bias (-\mathbb{E}|\epsilon|\approx-0.8\sigma_n), approximately −2.7 BIS at the reported median noise level. This magnitude exceeds the reported +0.804 BIS encoding effect in Table 10, although the quantities are not directly equivalent. Consequently, the factorial experiments cannot cleanly attribute the observed performance differences to preference geometry rather than surrogate misspecification. Given the negative or inconclusive full-dimensional results, should we restrict the claimed optimization benefit to point-level-set problems?

1 comment

Thank you for these points as they sharpened the paper.

The greedy claim needs qualifying for the weighting and for being only approximate:

  • The preference term collapses under every weighting.
  • Unnormalised, the full acquisition agrees with the preference term on 85-99% of live states across the published weights.
  • The preference term picks the highest-mean candidate on 78.7% of those states, so at beta=1 the full acquisition is greedy on about 79%: approximately, not exactly.
  • Normalised or weight-matched, agreement falls to 3-5%, and the method explores without regard to the target (Sec. 3.2, App. A).

The propositions assume constant predictive variance:

  • Remark 1 states when varying sigma can change the ranking, and shows that both propositions hold for any preference that increases in |mu - Y*| at fixed sigma.
  • Empirically, on 6,049 live states the ranking matches the posterior mean (median rho = +0.997).

Should the optimisation benefit be restricted to point level sets? Yes. It is now restricted throughout. The bracketing result remains general.

Folding after noise confounds preference geometry with a biased, non-Gaussian likelihood:

  • We reran the folded arms with the noise added after folding (App. L, Table 22).
    • On noise-free regret, the drifting encoding effect is +0.978 BIS with noise before folding and +1.067 after.
    • On observed-reading regret it falls from +0.804 to +0.418, because that scorer then reads zero-mean noise for the folded arms but the absolute value of a noisy reading for the signed ones, which favours the folded arms.
    • The effect therefore does not come from the likelihood.
  • A second part of your point stands, and App. L now says so. The folded surface has a kink at the target that a Matern GP cannot fit, while the signed surface is smooth there. GREEDY-F and GREEDY-H differ only in the encoding, so this misfit is part of what the encoding does, not a confound.
  • Your bias calculation also applies to our observed-reading scorer, |BIS_obs - 50|: at the target its expectation is about 0.8 sigma_n, a median 2.7 BIS. We therefore report noise-free regret alongside it (App. B), and will make noise-free regret the primary twin metric at camera-ready.

Unverified surrogate configurations weaken the synthetic comparisons.

  • We re-ran GP-UCB and EI with the surrogate recorded (1,600 sessions). One of 32 means moved by more than two standard errors of the difference.
  • The 160 unconstrained BOBA[wm] sessions are excluded, and every column is marked recorded, unrecorded or mixed (Apps. B-C).
  • BOBA-OPT's seven-of-eight win over BOBA[wm] is at T=120, where the control's surrogate was not recorded. On the T=30 sessions with a recorded matched surrogate, it wins on 2 of 6 testable benchmarks and loses on none. Secs. 1 and 3.4 now state that the control's surrogate is unrecorded, and App. B gives the T=30 result.

There is no counterfactual validation or practical baseline.

  • Observational records contain only the clinicians' own dosing, so they cannot validate counterfactuals.
  • What they can test is prediction under that dosing. Forecasting the last 30% of each record from the first 70% gives a median R2 of 0.27-0.36, and beats the segment mean on 34-36 of 50 patients (Sec. 5, App. F).
  • We also added a PI controller and an open-loop clinician replay (Sec. 6, App. M).
Reviewer_worjOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies setpoint problems. The point is to hold an observable at a known target while the response drifts and argues that folding the residual into a distance determines whether a preference-based acquisition can work at all. Its case study is BOBA as an active inference acquisition for dynamic Bayesian optimisation. They show that the Gaussian KL preference term orders candidates by the posterior mean whenever the preferred outcome lies above every prediction, that this condition holds on 94.6% of recorded synthetic states, and that the epistemic term is about five orders of magnitude smaller at the published weighting, so the method behaves greedily. Because folding places the target at the supremum of the objective, the condition is a consequence of the encoding. Keeping the residual signed restores a non-monotone preference term. It then builds a digital twin of closed loop propofol anaesthesia from 50 patient specific models fitted to VitalDB recordings, and evaluates the field on it. The signed encoding raises candidate bracketing from 5.0 to 57.7 percent on the twin and from 5.6 to 84.5 percent on fourteen synthetic problems.

Critical strengths

The diagnostic holds and generalises beyond the case study. Prop1 follows from the sign of the quadratic term in the Gaussian KL over a candidate set with constant predictive variance, and Prop2 identifies bracketing as the condition that a preference term needs in order to order anything. Empirical work then shows that folding makes bracketing rare for an algebraic reason, not because of a particular benchmark: the target sits at the supremum of the objective for every choice of setpoint. The claim is supported by 6k live states with a median Spearman correlation of 0.997. They also check the mirror case and the case where the target is placed inside the observed range.

The reporting is unusually honest. They state that their own corrected acquisition, BOBA-OPT, improves on BOBA but not on GP-UCB, that the information term is never significantly beneficial once the encoding is fixed, that a two-line proximity rule matches every active-inference variant, and that no method keeps its worst-case patient inside the clinically adequate band. It also reports that an earlier configuration made BOBA-OPT look strongest because its surrogate was unconstrained, and that matching the surrogate removed most of the advantage.

Critical weaknesses

The abstract is not checked against Table 2. The abstract says GREEDY-H "matches or beats every active-inference variant we test", while Table 2 gives BOBA-LS lower regret (8.198 v 8.937) and a higher on-target share (0.672 v 0.655). The summary was not checked against the results. The authors may need to review AI's output more carefully.

The abstract claims more than the experiments establish. For example "This encoding, rather than the acquisition function, decides whether a preference-based acquisition can work" is a causal ordering that Table 2 does not support without the paper's auxiliary explanation. BOBA-OPT runs on the folded objective and obtains regret 8.835 against 8.937 for the signed proximity rule, with distance 0.1558 against 0.1492. The two are statistically tied on location error at T=30. The body explains that the epistemic term substitutes for the preference term the encoding disabled, and the +0.545 BIS gain from adding information to the folded twin supports that reading. But the abstract does not say this, and a reader will come away thinking the acquisition function does not matter.

A related issue affects [collapses to greedy] at the published unnormalised weighting the method is greedy, but under the weight-matched convention the paper also runs, about 95 percent of 3,411 live decisions change. The collapse comes from the weighting convention. The acquisition itself is fine.

Evidence for the fix is narrower than the framing suggests. On the synthetic suite the signed encoding's main effect is significantly negative in one cell and null in three, and the paper itself restricts the benefit to point level sets and calls the multi-dimensional case untested. On the twin, the improvement in drifting location error is +0.035 with a CI clear of zero, which is real but small. The rule behind the recommendation is compared against a fixed concentration of 1.713. A feedback controller or a replayed clinician policy is not tested. The twin has no monitor lag. The action is an effect-site concentration, so syringe and effect-site equilibration are already removed. The BIS measurement delay that would hurt a myopic rule is also still missing. This means the central practical claim is tested on the plant most favourable to it.

Questions for authors
  1. Can you show the Table 1 bracketing rates on an identical state set for both encodings, and state which arms contribute to the 48,400 and 66,000 states? If the paired contrast is smaller than 5.0 to 57.7, by how much?
  2. Does your BOBA arm reproduce any published result from Kelly2025? Without that, the collapse could come from the reimplementation.
  3. What happens to GREEDY-H's advantage if the twin includes a first order monitor lag of 10 to 60 seconds and a sample-and-hold control interval?
  4. Why is the deployable baseline a fixed concentration? A PI controller or the recorded clinician's infusion replayed through the twin would be the relevant comparison. The records contain the infusion history.
1 comment

Thank you for your review. Each point led to a change.

The abstract was not checked against Table 2: sorry for this mistake. Among the shared-surrogate arms, BOBA-LS has the lowest median regret, but its paired difference from GREEDY-H is not significant (-0.151 BIS), and the paper now says the two are tied (Abstract, Sec. 6.2). We have since checked every number in the text against its table. Twin regret is scored on observed readings unless marked noise-free, and key paired differences state whether they are per-patient medians.

The claim that the encoding, not the acquisition, decides was stronger than the results support: The claim is now narrower, the encoding decides whether the preference term can rank candidates. On the folded objective, the epistemic term partly substitutes for the preference term (+0.545 BIS, Sec. 4).

The greedy collapse depends on the weighting convention: Agreed. We separate three things (Sec. 3.2, App. A).

  • The preference term's collapse holds under every weighting.
  • Unnormalised, the full acquisition picks the preference term's candidate on 99.4% of live states at beta=1 and 85.2% at beta=32. The preference term picks the highest-mean candidate on 78.7% of these states, so BOBA is approximately, not exactly, greedy.
  • Normalised or weight-matched, the information term decides: the full acquisition agrees with the preference term on 3.1% and 4.95% of live states. However, it still ignores the target.

The evidence for the fix is narrower than the framing: The regret claim is now restricted to point level sets, and the regret effect (+0.804 BIS) is reported next to location error (Sec. 4).

Bracketing should be compared on identical states: Refitted on identical query histories, bracketing rises from 7.96% to 52.22% on the twin and from 17.08% to 83.05% on the synthetic suite (Table 13). That is about 8 and 13 points smaller than the unpaired figures. App. D lists which arms produced the states.

The collapse could come from the reimplementation:

  • We ran the original implementation of Kelly et al. (2025), obtained from its authors, at its published weights, changing only its output directory and device mapping.
  • The collapse appears inside it. The condition holds on 98.1% of 960 states, and the full acquisition picks the preference term's candidate on 37 of 39 live unnormalised states and 2 of 78 normalised ones (App. A, Table 7).
  • On 200 recorded states its acquisition values match our pipeline exactly (App. B).
  • That implementation's stored outputs reproduce published values, e.g. Griewank 4.29 ± 0.98 (Table 1 of Kelly et al., 2025; App. A). Our own runs use a different budget, so we do not claim to reproduce a published table with them.

The twin lacks monitor lag:

  • We added a first-order monitor lag of 10-60 s, with the dose held between decisions.
  • GREEDY-H's noise-free regret rises only at T=120, by a median per-patient 0.99 BIS at 60 s, and it stays tied with BOBA-LS and BOBA-OPT (Sec. 6.4, App. M).

A fixed concentration is not the relevant baseline: We added both baselines you suggested (Table 2, Sec. 6, Apps. M-N):

  • Clinician replay. Each patient's recorded infusion is replayed open-loop through the twin. GREEDY-H beats it by 1.3-2.0 BIS at every budget.
  • A PI controller, tuned on 65 held-out patients.
    • It has the lowest median regret. GREEDY-H on a forgetting surrogate comes within 0.16-0.50 BIS of it on observed readings, and 0.62-0.65 on the noise-free surface (Sec. 6.2).
    • Tuned for the median patient, its worst patient averages 25.4 BIS from target at T=30 on the noise-free surface.
    • Retuned for the worst patient at each budget, it matches or beats the forgetting-surrogate arms on median regret but still leaves 1 to 5 of 50 patients outside the band, with a worst patient of 11.4 to 19.0 BIS.
  • GREEDY-H and BOBA-LS on the forgetting surrogate keep every patient's mean noise-free deviation within 10 BIS at every budget, with worst patients of 6.3 to 9.2 BIS.
  • The encoding is what keeps the tail inside the band: GREEDY-F on the same surrogate leaves 5 to 13 of 50 patients outside it (Sec. 6.4, Table 24).

We hope these changes are sufficient and look forward to hearing back.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.