- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces WITNESS, an interactive 2D grid-based environment for studying hidden-rule discovery in language models. The framework contains WitnessBench for evaluation, WitnessGym for RL training, and WitnessForge for generating and validating new games. The authors evaluate 18 proprietary and open-weight models under a shared interaction setting, and also use rule-access and observation-interface experiments to study whether the main difficulty comes from rule discovery or rule execution. The paper further trains Qwen3.8-27B with RL and reports improvements on unseen rules, private test games, and several external reasoning benchmarks.
Critical strengths
- The benchmark provides a clean and controlled setting for studying interactive rule discovery.
- The split between held-out compositions and held-out rule primitives is useful for evaluating different types of generalization.
- The paper evaluates many frontier and open-weight models under a shared interaction framework.
- It is also interesting that the paper goes beyond benchmarking and studies whether RL training can transfer to unseen rules and external tasks.
Critical weaknesses
- WITNESS is still a quite abstract 2D ASCII environment, so its connection to real-world scientific discovery is somewhat indirect.
- The positive RL results are mainly demonstrated on Qwen3.8-27B, while Qwen3.5-9B does not improve under the same setting, so the generality of the RL conclusion is still unclear.
- The number of independent games and rules is relatively limited, although each game contains multiple levels and seeds.
- The benchmark score is quite sensitive to the observation format and agent configuration, so it may partly measure the whole agent system rather than only the reasoning ability of the base model.
- The average transfer gain on external benchmarks is positive, but the improvement is not equally convincing on every individual benchmark.
Questions for authors
See Weakness
5 comments
Response to Reviewer 2esR (2/2)
W3. The number of independent games and rules is relatively limited.
At its current size the benchmark already spreads the 18 models over a wide range of scores, and since submission we have added more private games to check that this holds as the set grows. The submitted version has 47 games (22 training, 8 validation, 17 private test) with 760 verified levels, and the generation pipeline (Section 2.3) is built to extend it. This is within the range of comparable game benchmarks and training suites, which report 70 games for DiG-Bench (21 public) [6], 6 games for lmgame-Bench [7], and 3 training games for SPIRAL [8], and each WITNESS game adds a multi-level curriculum, giving 760 verified levels and 1,700 level slots per model.
Since submission we added 9 private test games, enlarging the private set from 17 to 26, and evaluated 11 models (listed in the new Table 14) on all 26 games in a single set of runs with an updated version of the canonical harness (differences listed in the caption of Table 14; model settings in Table 16). Both rankings below come from these same runs, so the comparison is between the 17-game and 26-game sets rather than between harness versions. The ranking on the 26-game set matches the ranking on the 17-game set (Spearman = 1.00 for OAE-L20 and 0.99 for RHAE-L5); per-model scores on the new games are in the new Table 14. On rules versus levels, Table 6 shows that gains on trained layouts carry over to unseen layouts of the same rules, which is why the benchmark invests in distinct rules rather than in more levels per rule. The upshot is that the benchmark already discriminates between models, and the enlarged private set confirms the rankings.
[6] Ruairidh M. Battleday et al. DiG-Bench: Discovery in Games. arXiv:2608.12593, 2026.
[7] Lanxiang Hu et al. lmgame-Bench: How Good Are LLMs at Playing Games? ICLR 2026. arXiv:2505.15146.
[8] Bo Liu et al. SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning. ICLR 2026. arXiv:2506.24119.
W4. Scores are sensitive to observation format and agent configuration.
This is a point we share, and Section 3.2 is where the paper measures it: a benchmark score reflects the model together with its harness, and we wanted to know by how much. Replacing the ASCII board with the raw grid changes scores by different amounts across models and reverses the order of Opus-4.8 and Kimi-K3, and adding code execution (Prime-Agent) changes scores further. That is why every model comparison in the paper is made under one canonical harness, so the harness is held fixed across the 18 models, and why Section 3.2 quantifies the dependence. Section 3.2 now states this reading explicitly: the benchmark score is a score of the model under the canonical harness, not a harness-free measure of the base model. In short, the benchmark gives a controlled comparison across models under one harness, and the harness dependence is measured and reported.
W5. The external transfer gain is uneven across benchmarks.
We expected the gain to vary across benchmarks, which is why the paper reports a confidence interval for each one rather than only the mean. The four benchmarks were chosen to probe different aspects of discovery: inferring an unseen encoding from examples (CipherBank), discovering a physical law through self-designed experiments (PhysGym), choosing tests that eliminate competing hypotheses (WILT), and seeking tests that could falsify one's own hypothesis (Failing-to-Falsify). At step 20, all four point estimates are positive for both training seeds, and the mean gain is +4.1 (95% CI [+2.1, +6.2]) for the first seed and +3.8 (95% CI [+1.7, +5.8]) for the second. The intervals differ in width because the benchmarks differ in size and variance (786 items for CipherBank against 80 instances for Failing-to-Falsify), and we read them as showing which aspects transfer more clearly rather than as a uniform effect. Overall, the direction of transfer is consistent across benchmarks and seeds; its size differs by benchmark, and the paper reports it that way.
Follow-up on our response
Dear Reviewer 2esR,
Thank you again for your review. We would be grateful to know whether our response and the revised PDF address your concerns, in particular, the 9B-model result in Figure 9 and the external-benchmark transfer in Appendix B.4. We are happy to clarify anything before the discussion closes.
Response to Reviewer 2esR (1/2)
Thank you for the detailed review. Our answers are below; all changes in the revised PDF are highlighted in blue.
W1. The environment is abstract and its connection to real-world scientific discovery is indirect.
We see the connection to scientific discovery in the interaction loop that WITNESS exercises rather than in the domain itself: an agent proposes a hypothesis about hidden rules, picks an action to test it, and revises the hypothesis from the outcome. Whether what is learned in this loop carries over to other domains is what the external transfer experiment tests. The model trained on WITNESS improves on PhysGym (discovering a physical law through self-designed experiments, +3.4), WILT (choosing tests that eliminate competing hypotheses, +2.3), Failing-to-Falsify (seeking evidence against one's own hypothesis, +6.1), and CipherBank (inferring an unseen encoding, +4.8), and the gain holds for a second training seed (+3.8, 95% CI [+1.7, +5.8]). Following Reviewer Yngb's suggestion, the abstract now describes this as transfer to external benchmarks of rule inference and hypothesis testing. To summarize, the connection to scientific discovery runs through the hypothesis-testing loop, and the external transfer results are the evidence for it.
W2. The positive RL results are mainly on Qwen3.8-27B; Qwen3.5-9B does not improve.
In RL, whether a task is learnable depends on the current policy: tasks that a model solves never or always provide no learning signal, and which tasks fall in between shifts as the same model trains [1, 2, 3, 4, 5]. The 9B result is one we chose to include rather than leave out, and a new experiment now helps explain it. Qwen3.8 has no 9B release, so we took the fine-tuned Qwen3.5-9B (on data distilled from Qwen3.8) and trained it with the same RL recipe. This model (Qwen3.8-9B-Distill, new Figure 9) is more learnable from the same RL task recipe than Qwen3.5-9B: maximum gains of +5.29, +0.85 for validation RHAE-L5 and test RHAE-L5, respectively. Whether a model learns from these games therefore depends on its base training recipe as well as its size, and the caption of Figure 6 now describes the comparison in these terms. How far the RL result extends across model families is a question we flag in the revised Conclusion.
[1] Minqi Jiang, Edward Grefenstette, Tim Rocktäschel. Prioritized Level Replay. ICML 2021. arXiv:2010.03934.
[2] Tambet Matiisen, Avital Oliver, Taco Cohen, John Schulman. Teacher-Student Curriculum Learning. IEEE Transactions on Neural Networks and Learning Systems, 2020. arXiv:1707.00183.
[3] Thomas Foster, Anya Sims, Johannes Forkel, Mattie Fellows, Jakob Foerster. Learning to Reason at the Frontier of Learnability. arXiv:2502.12272, 2025.
[4] Qiying Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476, 2025.
[5] Taiwei Shi, Yiyang Wu, Linxin Song, Tianyi Zhou, Jieyu Zhao. Efficient Reinforcement Finetuning via Adaptive Curriculum Learning. arXiv:2504.05520, 2025.