- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks how an agent can reuse experience across repeated exploration without discarding action order, prerequisites, and failure context, or freezing an incorrect interpretation. BISE uses two memory layers: an episode-indexed gist stores ordered experience from one attempt, while a source-linked schema page aggregates and revises reusable rules across attempts. It evaluates six Jericho games and 211 ScienceWorld task variations. On GLM-5.3, the mean score is higher than ReAct and reproduced EvoTest, but the ablations suggest that the main benefit comes from ordered gists. But the schema page’s additional performance contribution remains unclear.
Critical strengths
- The problem and solution line up. The paper does not merely ask for “better memory”; it separates preserving ordered experience from revising accumulated knowledge, and assigns those functions to the gist archive and schema page.
- The experience unit is concrete and auditable. Episode gists retain action order, prerequisites, outcomes, and failure context. Source links and trajectory audits let the reader trace how advice was formed.
Critical weaknesses
- Only one layer of the two-layer solution has stable performance support. The ordered-gist direction is supported by an ablation, but gist-only is nearly as strong as full BISE. The schema page’s added value is mainly shown through organization and traceability, not a stable task-performance gain.
- The ScienceWorld memory result lacks a key control, and the release materials are incomplete. Without a repeated-attempt no-memory system, the improvement cannot be separated from gains caused by the task or the actor’s repeated attempts. The selected cases also do not estimate the rate of wrong advice, overgeneralization, or false rules. The paper says that supplementary material contains scripts and snapshots, but gives no anonymous source link for independently checking the implementation and audit process.
Questions for authors
- Can you add a same-backbone, same-step-budget, same-attempt-count repeated-attempt no-memory control on ScienceWorld and report paired results for each task variation?
- Can you repeat the de-temporalized gist, verbatim log, gist-only, and full-BISE comparisons across multiple seeds, with paired confidence intervals or at least run-level distributions?
- Can you quantify incorrect or unverified rules in the schema page and gists, wrong retrieval advice, and their consequences, while reporting token, curator-call, and inference costs rather than only environment interaction budgets? Please provide accessible anonymous links to the source code, run scripts, and audit data during review.
2 comments
Response to Reviewer gPJg [Part 2]
Q3. Incorrect advice and consequences
We have not estimated population-level rates of incorrect rules in pages or gists, wrong retrievals, or harmful advice. The available action-level evidence traces a corrected rule and an unsupported generalization to their subsequent use.
In Library run 1, attempt 4 executes ask technician about gates (+2), examine printers (+5), and give librarian printout (+2), then leaves with the original book. The curator replaces the copying rule before attempt 5 retrieves the updated page and repeats the successful procedure. In Zork 3, the page records contrasting teleport outcomes but also overgeneralizes that every non-“I” indicator phase is fatal. Attempts 7–10 reach 6/7 without recorded deaths; those executions do not test the universal claim. The records therefore show both a correction and a rule that remains unvalidated.
A separate source-traceability screen covers 463 entries in 18 final primary pages. Of these, 435 have source-attempt markers and 28 do not. Among the marked entries, 371 meet a 40% content-word overlap threshold with the cited gists and 64 fall below it. These are lexical source-support categories, not correctness labels; auxiliary files and gist correctness are outside this screen.
Q3. Token use and curator calls
BISE records 1.73±0.08M tokens per attempt, comprising 1.60M actor and 0.13M curator tokens, with 6.7 usage-bearing curator calls. Curation accounts for 7.6% of the total. Gist-only records 1.88±0.09M tokens and 6.9 curator calls. The ReAct actor records 1.22M tokens, making BISE's recorded total about 42% higher than this actor-only control.
Counts include resumed runs and are normalized by 60 planned attempts per run. Failure logs without usage are excluded from token sums, so these are recorded inference costs rather than complete billing totals. ReAct's unused curation has incomplete logs and is excluded. Comparable EvoTest usage, monetary cost, and wall-clock latency are unavailable.
The supplementary material includes anonymized per-response token records and run-level summaries.
Response to Reviewer gPJg [Part 1]
W1. What does the schema page add?
Full BISE and gist-only achieve similar scores across three runs (55.7±2.1 and 55.0±2.7), while BISE records 7.6% fewer tokens per attempt: 1.73±0.08M versus 1.88±0.09M, including actor and curator usage. The final knowledge files contain 4,685 words, compared with 17,281 in the episode archive, summed over six games and averaged across runs. The page provides a compact view; the archive is still retained.
The Library trace shows how this view is revised. After attempt 4 succeeds with the original book, the curator replaces the page's incorrect copying rule with the technician–printer–librarian procedure. Attempt 5 retrieves the updated page and repeats that procedure. This directly documents the page's role in updating advice while preserving the earlier episode record.
W2 / Q1. ScienceWorld beyond retrying
On the same 63 initially imperfect task–variation pairs, the complete memory procedure reaches 89.5±1.8 versus 83.0±2.2 for memoryless retries over four attempts and three runs per arm, a 6.6-point difference. Mean solved counts are 50.7 versus 39.0. Both arms allow 100 actor rounds and 100 environment steps per attempt; the memory arm also allows 15 curator rounds. The selection attempt contributes neither scores nor memories to these restarted runs.
Mean cumulative-best scores across attempts are 67.5 → 81.4 → 86.3 → 89.5 with memory and 71.5 → 80.2 → 81.8 → 83.0 without. Task-aligned differences between arm means are positive for 29 pairs, zero for 25, and negative for 9. Runs are the replication unit because variations can share memory within a run; the task differences are descriptive.
Scores use environment records for the requested task and variation. Missing or wrong-task records add no verified progress, earlier verified best scores carry forward, and all 63 pairs remain in the denominator. We retain the recorded early stops, including stops without a verified solve. Both arms configure the same service alias; the archive does not pin the served revision. The comparison evaluates the complete procedures under a common interaction budget. No-memory run 1 has four attempts; the five-attempt comparison uses runs 2 and 3 in both arms and gives 91.3±1.0 versus 82.9±2.1. The original 211-pair result is a separate within-system evaluation.
Q2. Repeated representation comparisons
All five representations now have three independent sequential runs, each containing six games and ten memory-sharing attempts per game, capped at 110 steps per attempt. Individual run scores and mean ± s.d. are:
Statistics use unrounded scores. We report run distributions because the records do not identify matched seeds across conditions. Atomization has the largest mean loss, 11.5 points. The de-temporalized condition changes the summary instructions and retrieval-score presentation as well as order: content and length are not fixed. Its 3.8-point gap compares that configuration with BISE rather than isolating event order.