- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Given a physical system that may be in one of k states , the goal is to find the state by sequentially choosing candidates to pass to the system and observing response
.
Sequential Bayesian Experimental Design solves this problem by assuming a prior over , and for every possible candidate in the candidate set using the statistical model as a likelihood to build a posterior . This posterior can then be used to predict expected information gain (EIG) of observing for a hypothetically chosen , then the input with highest EIG is chosen for evaluation, a true response is observed, the posterior over can be updated, and the process repeats. In my understanding, this is a somewhat standard procedure in BED and active learning and EIG is a standard acquisition function to quantify the hypothetical benefit of observing an output for a given input.
This work takes inspiration from BED-LLM that applies the above recipe to generating questions to elicit user answers in order to learn latent user preference , the LLM proposes questions, EIG quantifies the benefit of each question, the best question is then given to the user. The BED-LLM use case applies when candidate and response are strings.
This work considers a setting much closer to standard BED where and are standard mathematical variables, explicitly is a point from a given discrete candidate set (hence it may be categorical/integer/continuous variable) and is a continuous scalar or vector response.
For applications that have a large candidate set , evaluating the EIG for all possible input points may be expensive. Secondly, some applications may also provide text feedback along with a response, such as error messages, logs, constraint violations.
This paper considers how to use an “LLM-in-the-loop” that may alleviate both of these issues. An LLM has text in (prompt) and text out (answer), while the physical system inputs and outputs are numerical. Hence application specific adapters must be constructed, in this case a “serializer” maps from data history and a list of “intents” to text prompt and a “compiler” from text answer to a subset of candidates .
The BED pipeline proceeds as normal, instead of evaluating EIG for all candidates, we use the the LLM to filter to a subset of candidates and EIG is evaluated only for the subset. If I understand correctly, the contribution of PhysGround-BED is as an acquisition function optimizer in an otherwise standard BED pipeline.
Extensive experiments comparing PhysGround-BED to evaluating EIG across the unfiltered set of candidates (Full Physics BED baseline) shows that the filtering does not hinder performance, and in fact the filtering does drastically reduce the candidate set.
Critical strengths
- integration of LLMs into BED act as an efficient AF optimizer
- the benefits over exhaustive evaluation and error mesage incorporation are clearly demonstrated in experiments.
Critical weaknesses
Narrative from BED-LLM
- If I understand correctly, BED-LLM uses the LLM for candidate generation as candidates are text.
- PhysGround-BED uses the LLM for candidate filtering and saving on EIG evaluations,
- in contrast with the prior work, this is not a string based setting suited to LLMs and hence adapters are required. The paper thus needs to justify why traditional BED methods should include LLMs
- I understand that the adapters + LLM are serving the purpose of warm start for EIG optimization (or any acquisition function, AF, optimization). Hence other forms are AF optimization are the baselines for experiments.
Two Main Advantages have weak Evidence
1. Reducing EIG evalautions
- Is running an LLM (typically multi-GPU workload required 10-100GB memory) cheaper than full EIG evaluation over a finite candidate set in these applications? Is it cheaper than running any other more intelligent form of EIG optimizer?
- If the candidates are points in a continuous space and naive brute force is slow, one may use multi-start gradient ascent (grads via reparamerization trick) to find and just evaluate EIG for all points in the neighbourhood of the argmax.
- if the candidates are categorical and EIG is estimated MC sampling, one may use off-the-shelf multi-armed bandits to allocated MC samples to promising candidates and ignore poor ones.
- Regarding the first advantage of reducing EIG evaluations, the paper does presents evidence that placing an LLM in the improves over brute force, but does not compare to any other more efficient method nor does it account for the (significant) extra memory/FLOPs of using an LLM.
2. Error message Incorporation
The second advantage LLMs provide is error message feedback that can be incorporated into the filtering of the candidate set. The paper does provide an experiment that shows a nice clear advantage. However the difficulty of the problem and hence the value of this contribution to ICLR is not clear.
The compile component of the pipeline acts as the adapter from text based LLM output to filtered candidate set. I presume this must be domain specific implementation. What do intents look like? How are they mapped to candidates? Can a basic message parser and some quick heuristics achieve the same filtering of invalid candidates? Do I need a full LLM?
(Presumably prior work along these lines would have fallen into the field of "constrained optimization" with unknown constraints and for benchmarks in such communities, error messages are simply turned into binary feedback for the optimizer, i.e. message parsing and incorporation may be a solved problem. If not text based contrains in constrained optimization may be a fruitful research direction)
The internals of Validate$_d(\epsilon, \mathcal{X}$) and are not clear, does this work solve a difficult problem? Is the problem implementation/engineering or is it algorithmic/scientific? I struggle to judge if this contribution is suitable for ICLR and if suitable whether it is sufficient for acceptance.
Limitations are not listed
- a user must write an application specific adapters: serializer , Validate() and
- the computational cost of autoregressive token generation for an LLM is very high, only applications where EIG is very expensive really benefit from compute savings. (This is not the case if a cloud provider is hosting the LLM, in which case this is harness engineering and arguably less suitable for ICLR)
Presentation
I felt that the writing was very cluttered, below are a handful of examples that led to this paper feeling unnecessarily hard to read for me
-
Many sentences contains lists often of items that have not been introduced or defined clearly. For example Lines 201-206 list many many components of all the applications which is rather unhelpful for non-experts in these applications. In general there are a lot of lists.
-
Many of the objects and quantities being discussed are prefixed by many adjectives/descriptors.
-
Details about the applications appear to be spread out throughout the paper. A clear definition would be much more helpful: for each setting, what are , , and their dimensions? what is ? what are typed semantics/intents? Ideally this should be described once directly after the problem formulation where all variables (including intents) are first defined, or at the start of the experiments/benchmarks whereafter the applications will be used. Currently such information seems to be scattered throughout the paper. Ideally method sections focus only on the mathematical formulation specifically avoiding application specifics (all paper sections follow the single responsibility principle, makes it far easier to read).
-
I may be misunderstanding but terminology appears to be from probabilistic programming languages but may not be standard for BED or more general statistics/LLMs (prior works [1,2] do not use such terminology either). As one option, perhaps the following terminology may broaden the audience of the paper increasing reach and adoption (though many other options may suffice too)
- "Catalogue": candidate set
- "Acquisitions": candidates
- "Compiler": filter, subset selector
- "Compiled acquisitions", "registered candidates": filtered candidate set
- "Backend model": likelihood, statistical model
- "Typed X/Y/X" : a prespecified list of X/Y/Z
Minor comments:
-
Equation (1) is never referenced, can it be removed?
-
Definition one seems overly formal in this more engineering-based paper, can it just be any other equation?
-
Perhaps emphasise the goal of the algorithm more (I missed it the first few times) on line 121 can be replaced
121: "we study finite-hypothesis closed-set identification over a horizon , with terminal prediction θ = arg maxθ∈Θd πT (θ)."
"The goal is to identify the true current state, , by sequentially choosing $x\in\mathcal{X}, observing and using the model $p_d() to learn about ."
Questions for authors
- what is the difficulty in incorporating error messages in the LLM?
- there are far more efficient ways to optimize EIG than exhaustive evaluation, the only baselines so far, they need to be included as baselines to make a convincing case for introducing an LLM as an alternative
- continuos : multi-start gradient ascent with early stopping when diverging too far from the given candidate set
- categorical : simple UCB multi-armed bandit