- Inconsistent experimental controls in the main factorial design.
The released structured-exploration-supp-min/run.py disables the comprehension quiz for the baseline (wu_bare) but enables it for the other three factorial conditions. The quiz answers are subsequently included in decision contexts (structured-exploration-supp-min/wu1d.py). This introduces an uncontrolled prompt difference into the central R×M comparison (Section 2.2), weakening the causal attribution of the reported interaction. Matched experimental conditions are needed to establish the claimed effect reliably.
- Mismatch between the primary metric definition and implementation.
Equation (1) defines uncertainty percentile as qt=100(rt(at)−1)/(∣Ut∣−1), using average ranks for ties. However, structured-exploration-supp-min/uncertainty_percentile.py (lines 18–20) uses np.searchsorted(v, sig[c-1])/len(v), implementing neither convention. Independent recalculation produces different numerical results, although the positive interaction remains. The reported estimates and confidence intervals should be reconciled with the stated metric definition.
- Inconsistent action handling between code and saved trajectories.
The saved trajectories, including structured-exploration-supp-min/out/wu_bare_deepseek.jsonl, contain invalid position-zero actions (16 occurrences in the DeepSeek baseline), despite the released fallback implementation enforcing valid positions from 1 to 30 (agents.py; wu1d.py). Such actions can also enter subsequent posterior calculations through obs[c] = r (uncertainty_percentile.py). This raises concerns about implementation-version consistency and the reliability of the evaluation pipeline.
- Inconsistencies in statistical analysis and reproducibility.
Figure 2 reports seed-paired bootstrap confidence intervals, whereas structured-exploration-supp-min/figs/main_data.py independently resamples factorial cells. A separate script, structured-exploration-supp-min/reviewer2_checks.py, implements paired resampling, leaving the authoritative analysis pipeline unclear. Some intermediate data required by the supplementary analysis scripts are also unavailable, preventing complete independent verification.
- Superadditive exploration does not establish superadditive task performance.
The central positive interaction is demonstrated primarily through uncertainty percentile rather than task utility. The best-found-value interaction is approximately −1.1 percentage points for DeepSeek (Table A11), while the two-dimensional interaction is not statistically distinguishable from zero (Table A13). The GPT-5.6 diagnostic also shows reduced per-step reward (Table A18). Thus, the evidence supports increased exploration extent, but not necessarily improved exploration efficiency or task performance.
- The proposed representational bottleneck is not sufficiently isolated.
All 30 candidate positions are already available in the primary task. The coverage representation primarily reorganizes and emphasizes unexplored locations rather than introducing previously unavailable alternatives. Moreover, sentence rendering alone produces a substantial improvement over a structured list (+12.9 points, Table A17). These findings leave prompt salience and linguistic framing as plausible alternative explanations for the claimed mechanism.
- Limited novelty and generalizability beyond structured search.
Externally organized histories and explicit exploration targets have already been investigated, notably by Krishnamurthy et al. (2024) [1] and Lu et al. (2025) [2]. The present contribution is primarily a factorial behavioral analysis rather than a new exploration algorithm. Furthermore, the experiments rely on finite, enumerable action spaces, leaving the broader implications for open-ended LLM agents insufficiently established.
[1] Akshay Krishnamurthy, Keegan Harris, Dylan J. Foster, Cyril Zhang, and Aleksandrs Slivkins. Can large language models explore in-context? Advances in Neural Information Processing Systems, 37, 2024.
[2] Cong Lu, Shengran Hu, and Jeff Clune. Intelligent go-explore: Standing on the shoulders of giant foundation models. In International Conference on Learning Representations (ICLR), 2025.
Response to Reviewer mCiC
Thank you for this careful and constructive review. Your two suggestions, to say more clearly what the work adds and to test an agent that can call an optimizer, shaped much of the revision, and the second turned out to be one of the most informative new experiments.
On what the findings add to work on state representation and exploration (W1). We agree that the submission did not make this clear, and we have rewritten the introduction and related work around it (Sections 1 and 4). Work on state representation has shown that summaries of past experience can help LLM agents explore, and count-based methods in reinforcement learning already use coverage to direct exploration. What remained open is which part of such a remedy does the work. The evidence comes mostly from bandits, where every option is already named, and from systems evaluated as a whole; Krishnamurthy et al. (2024) note that their prompt modifications appeared to interact in ways that were hard to attribute. Our design pulls the two parts apart in a task where the agent must infer from its history where it has not yet looked. There they turn out to be complements. Motivation alone leaves search local (in two models it makes agents try more new positions, but not farther ones), and it widens search only once the untouched regions are named. A direct instruction to try unvisited positions shows the same pattern: it adds new choices right next to known ones without widening search (Section 3.3, Figure 3). Among named bandit arms, by contrast, motivation alone is enough. So the contribution is less that coverage helps than when an LLM agent actually uses coverage information that is already in its context.
On whether this is a general decision-making effect (W2). We think this is a fair point, and we cannot rule it out, since we did not give the interventions to people. What we can say is that the starting point differs. In the same task, humans search far more widely without any help and find better options than random search with the same number of tries, while baseline agents stay next to what they have already tried and do worse than that benchmark (Sections 3.1 and 3.2). Because the human data come from a visual interface, we treat this comparison as descriptive. We have taken your comment about the framing to heart: the revision presents the work as a question about what to put in an agent's context, and keeps the human data as a reference rather than as a claim about cognition.
On tool-using agents (W3). We added the comparison you proposed. In two new conditions, the agent may ask for a GP-UCB recommendation, computed from its own observations, before every choice and at no cost in points (Section 3.3, Table A10). Having the optimizer available did not make the problem go away, because agents rarely asked for it. With the original instructions, DeepSeek, Qwen and GPT-4.1-mini requested advice on about 19%, 13% and 1% of choices. With coverage and motivation, requests rose to about 58%, 38% and 3%, and adding the tool raised the reward of that condition in all three models. In this more complete agent, then, whether useful information gets used still depends on what the context gives the agent. GPT-4.1-mini almost never called the tool, so we do not read its reward change as an effect of the optimizer.
Thank you again for these suggestions. If the revised introduction still leaves the contribution unclear, we would be grateful to know where it falls short.