acceptodds
Under review as a conference paper at ICLR 2027

Textual Planning with Explicit Latent Transitions

11 views

Abstract

Planning requires a transition model that predicts how each action changes the current state. When a large language model (LLM) plays this role, every next state is generated token by token, which makes searching over many possible futures slow and expensive. Existing alternatives either still query an LLM at every step or require a symbolic model of the domain. We propose EmbedPlan, a transition model built on frozen text embeddings: it embeds natural language descriptions of the state and the action with a frozen LLM, predicts the embedding of the next state with a lightweight learned network, and returns the closest real state. Because this network can be trained on top of any encoder, EmbedPlan also provides a controlled way to compare text representations for learning transitions. We evaluate it on 9 classical planning domains, under six settings that hold out progressively more of the data, from transitions to entire domains, and against baselines ranging from predicting no change to learning symbolic action rules. On planning problems seen during training, EmbedPlan almost always ranks the true next state among its top five guesses, still does so for most queries even when every observed state is a candidate, and retains 92–99% of its single-step accuracy when predicting several steps ahead from its own outputs. Given the same candidate states as GPT-5.4, it picks the true next state more often while taking about 0.17 ms per transition with cached embeddings. Accuracy is lower on unseen problems and near chance on unseen domains, and the controlled comparison traces this limit to the state representation rather than to the learned transition.

Primary area:
applications to robotics, autonomy, planning
Keywords:
textual planning, transition prediction, latent transitions, classical planning, language models, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2SFNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EmbedPlan, a lightweight transition model for text-described planning states. A frozen language encoder represents the current state and action, learned projection heads map them to a 128-dimensional latent space, and a small learned transition network predicts the embedding of the successor state. The predicted vector is then mapped back to an actual state using nearest-neighbor retrieval.

The motivation is to replace expensive autoregressive LLM calls during planning search with a cheap latent transition operation.

Critical strengths

The central idea—whether an expensive pretrained language representation can be used once and then support a cheap learned dynamics model is interesting. The separation between a frozen semantic encoder and a lightweight transition function is conceptually clean and potentially useful for systems that repeatedly plan in the same environment. The comparison against character n-grams, bag-of-words, bag-of-literals, offsets, random representations, and lifted STRIPS is useful. The result that very simple sparse text features outperform the 70B LLM representation on held-out problems is striking and potentially an important diagnostic observation.
The fact-order experiment is also informative: a semantically meaningless reordering can move the BGE embedding more than an actual action transition does. This provides concrete evidence rather than merely asserting that pretrained embeddings have the "wrong geometry."

Critical weaknesses

A concern I have is how that pool is obtained for unseen problems.

In the Extrapolation protocol, each query is evaluated against the true successor plus distractors drawn from the same held-out problem. Thus the model is effectively asked to identify the successor among states already available from the test problem, rather than to construct a successor whose representation may not previously exist.

This distinction seems important for the intended use as a cheap transition model. For a genuinely new planning problem, it is unclear how the system obtains an enumerable bank of states that both covers the problem and already contains the correct future state. The issue is therefore less candidate size than candidate coverage: the reported Hit@k evaluates ranking conditional on the ground-truth successor being present, but does not measure the probability that such a candidate would be available in the first place.

Questions for authors

In an unseen problem, where does the candidate state bank come from? In particular, can EmbedPlan predict transitions when the correct successor has not previously been enumerated, or is the intended setting one in which the reachable state set is already available?

4 comments
AuthorsOpenReview ↗

Comment 1 of 2: Candidate generation and coverage

Thank you for recognizing the clean separation between the frozen semantic encoder and the lightweight transition model, and for highlighting the sparse-feature comparison and fact-order experiment as important diagnostic findings. We appreciate that you see the potential value of this approach for systems that repeatedly plan in the same environment.

Your question about candidate coverage is important because it determines where these findings can be applied. We agree with this concern and have already revised the manuscript to clarify the distinction (Section 3).

Q1/W1. Where do candidate successors come from, and does Hit@k measure coverage?

Hit@k measures how well EmbedPlan ranks the correct successor, conditional on that successor being in the candidate pool. It does not measure candidate generation or coverage.

The original submission stated this assumption in Section 3, the limitations, and Appendix B.4. We agree that it deserved more prominence, and have now added a dedicated paragraph, "Candidate-pool construction and coverage" (p.5).

The distinction depends on the evaluation setting.

1. Observed problems: the pool contains states collected offline in the benchmark domain.

In Interpolation, training and test transitions come from the same planning problems. The standard evaluation includes the true successor and 127 distractors.

We also evaluate ranking against every state observed in the benchmark domain (Table 2), rather than only 128 candidates: 46,205 states on Ferry and 13,373 on Logistics. The true successor is included by construction, with no distractor sampling in this full-pool setting.

The results show both the capability and its limitation. Full-pool Hit@5 is 86.9% on Ferry and 74.9% on Logistics, while Hit@1 falls to 35.2% and 32.6%. We report this decrease explicitly.

The states were collected offline for the benchmark. We do not claim that an equivalent pool would automatically be available in deployment.

2. Unseen problems: the evaluation pool is constructed from the held-out problem, not generated by EmbedPlan.

In Extrapolation, the model is trained on other problems from the same domain. Each evaluation query includes the true successor and 127 distractors drawn from the held-out problem (Table 1).

These distractors are challenging because they share the problem's objects and often differ from the target by only a few facts (Appendix B.4).

The resulting 54.6% mean Hit@5 across nine domains (Table 5) measures generalization of transition ranking to new problem configurations. It does not establish the coverage of an independently generated candidate pool.

3. Can EmbedPlan return a successor absent from the pool?

No, not as a textual state under the evaluated retrieval formulation.

The network predicts a next-state embedding, but retrieving a discrete textual successor requires an available candidate. An external proposer or a collection of previously visited states could supply candidates, but neither integration has been evaluated here.

We also test whether intermediate retrieval can be avoided. On observed Ferry problems, step-wise Hit@1 falls from 94.2% with per-step snapping to 68.3% without it, using a 1,000-state evaluation pool (Table 4). This shows that latent-only rollout is not currently a complete alternative.

4. What do we know about the coverage of a candidate generator?

Table 3 reports that GPT-5.4 includes the true successor among its top five generated states for 56% of Ferry queries and 100% of Logistics queries. These numbers measure the generator alone, not a combined generator-and-EmbedPlan system.

We now report this distinction explicitly in Section 3 (p.5), and state that such a combined pipeline has not been evaluated.

The boundary is therefore clear: EmbedPlan learns to rank successor states, but does not solve the separate problem of making the correct successor available for retrieval. This limitation is now explicit in the revised manuscript.

AuthorsOpenReview ↗

Comment 2/2: Scientific contribution, manuscript revisions, and reassessment

Does the candidate-coverage limitation undermine the transition-learning results?

The limitation affects practical applicability, but does not invalidate the controlled transition-ranking experiments or the scientific findings they support.

As you highlighted in your strengths, the comparison with simple sparse representations and the fact-order experiment provide important diagnostic evidence. These findings do not depend on how candidates would be generated in a deployed system.

In Table 7, we compare representations using the same transition head, training data, and candidate protocol. On unseen problems across Ferry, Logistics, and Goldminer, character n-grams reach 79.4% Hit@5, compared with 56.8% for Llama-3.3-70B embeddings. Because the candidate pool is identical, coverage cannot explain this difference.

The fact-order experiment provides an explanation for the representation gap: reordering semantically identical state facts moves BGE-M3 embeddings 1.13x further than applying an actual action (Appendix N). This identifies a concrete limitation of frozen text representations rather than merely observing lower accuracy.

Appendix G.2 further shows that learning action-conditioned dynamics matters. Removing the action-disambiguation loss reduces mean Hit@5 on unseen problems from 54.6% to 35.3%, and action Acc@5 from 16.2% to 4.8%.

These are controlled findings about learned transitions and representations. Candidate coverage limits their direct application to a complete planner, but does not undermine the comparisons or their conclusions.

We believe this distinction matters for assessing Soundness. A missing operational candidate generator is a legitimate limitation of applicability, but not evidence that the controlled transition-learning experiments are methodologically invalid.

What have we changed in the revised manuscript?

We have already incorporated the clarifications, with all changes highlighted in blue.

  • Section 3, p.4: Added an explicit sentence stating that retrieval cannot return states outside the candidate pool.
  • Section 3, p.5: Added "Candidate-pool construction and coverage", clarifying the evaluation pools, conditional ranking, generator coverage, and absence of a combined proposer-and-ranker evaluation.
  • Section 3, p.5: Included the full-pool Hit@1 results on unseen problems, 3.8% for frozen BGE-M3 and 9.3% after LoRA adaptation, averaged over three domains. These results were already reported in Section 4.3 and Appendix M.
  • Table 1: Added explanations of what is new at test time for each exposure level and clarified two protocol descriptions to improve presentation.
  • Limitations: Made the candidate-availability assumption more explicit.

These revisions directly address the coverage and presentation concerns without changing the experimental results or expanding our claims.

Our request

We agree with the practical limitation you identified, and have addressed its presentation in the revised manuscript. We believe the scientific contributions you highlighted remain valid and informative within their stated scope.

Does the revised Section 3 resolve your central concern about the interpretation and validity of the evaluation?

If so, we would appreciate a reassessment of the paper, particularly its Soundness, Presentation, and overall recommendation, in light of the revisions and the scientific findings you recognized.

If a methodological issue remains beyond the acknowledged candidate-coverage limitation, we would appreciate knowing what it is so that we can address it during the discussion.

Reviewer_2SFNOpenReview ↗

i raised my score to 3 given the rebuttal has addressed part of my concern

It's great to see authors have identified the problem and make some revisions carefully, however, the limitation might not be easily addressed given the revised content.

Thank you for reconsidering your assessment and for engaging constructively with our rebuttal. We genuinely appreciate it!

We are glad the revisions helped clarify the scope of our evaluation and the scientific contribution of our controlled experiments. We believe these findings offer valuable insights into learning transitions over frozen language representations, including where current representations can be improved.

We appreciate your thoughtful consideration and the opportunity for meaningful scientific discussion.

Reviewer_CA1GOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The authors build an explicit transition model with a lightweight neural network on an embedding of states and actions produced by a frozen LLM encoder. They find they can succeed in winnowing down the correct next state to 5 entries amongst a known set of possible next states fairly effectively with a 0.17ms transition time.

Critical strengths

There are some careful controlled experiments here and a broad reporting of results across many different conditions.

Critical weaknesses

This paper is phrased as a positive result, but it seems more sound to think of it as a negative result since correctly choosing the next state succeeds 35% in Ferry, 33% in logistics, and 3% with an out of domain problem. Furthermore, it seems there are superior alternative approaches based on n-grams as per table 7.

I expect the fundamental issue here is that LLM embeddings just aren't trained by default to have the desired information in an invariant representation. Thus as a negative result this is unsurprising. The authors experiment with training a LoRA which does help, but it only raises the out of domain perf to 9%, still very far from a useful approach. There's also significant other work along the lines of training LLMs for transition functions. One example that I'm aware of in the transformer context is the next latent transformers https://arxiv.org/abs/2511.05963 which learn a fast transition model at the token (rather than compositions of tokens) level. But, if you have a language model it's also possible to just directly produce code for state and transition operators then run that code to do the search as per https://arxiv.org/pdf/2404.11833.

If the authors want to stick with the positive-result framing, the existing evaluation seems plainly unsound because you can't effectively plan with a success rate of 1/3 per step.

Questions for authors

No questions.

5 comments
AuthorsOpenReview ↗

Comment 1/2

Thank you for recognizing the "careful controlled experiments" and the breadth of our evaluation. We agree that the weak generalization results matter. Our concern is that the review you shared here uses results from different experiments to conclude that the evaluation itself is unsound.

Are the 35%, 33%, and 3% results measuring the same capability?

No. The first two are full-pool top-1 retrieval on observed problems. The 3% result concerns unseen problems within known domains, not unseen domains.

In Table 2, Ferry's 35.2% and Logistics' 32.6% are Hit@1 when ranking against every benchmark-observed state: 46,205 and 13,373 candidates. The same full-pool rows report Hit@5 of 86.9% and 74.9%. With 128 candidates, Hit@1 is 99.0% and 96.3%. We report the full-pool deterioration precisely because the smaller-pool results do not establish unrestricted successor retrieval. Hit@5 is not a substitute for reliable top-1 planning, but it answers the stated ranking question.

The 3.8% frozen and 9.3% LoRA results come from a separate BGE-M3 experiment, averaged over three domains, at full-pool Hit@1 on unseen problems of observed domains (Section 4.3, Appendix M). They are not "out of domain" results. Unseen-domain transfer is measured in Table 5 and is near chance: 6.6% Cross-Domain and 9.2% Leave-One-Out Hit@5 over 128 candidates. The abstract explicitly reports this failure.

Does 35% full-pool Hit@1 make the transition evaluation unsound?

It rules out reliable unverified greedy planning in that setting. It does not invalidate measurements of conditional transition ranking.

The paper defines EmbedPlan as a transition component, not a complete planner (Introduction). Search integration and candidate generation are not evaluated. Table 4 separately tests short rollouts on observed problems with the model's predictions fed back: with per-step snapping and 1,000 candidates, Hit@1 is 94.2% on Ferry, 69.0% on Logistics, and 78.4% on Blocksworld. These are not full-pool or end-to-end planning results, and we do not present them as such.

We also report whole-trajectory metrics on held-out reference plans. Under teacher forcing, the true successor ranks in the top five at every step for only 22.6% of Plan-Variant trajectories and 3.3% of unseen-problem trajectories (Section 4.2). These are not successful end-to-end executions, and we do not present them as such.

The distinction is consequential for Soundness: a component can have limited usefulness for a complete planning system while its controlled experiments remain valid for the question they measure. The review points to the former but does not identify an incorrect measurement or unsupported claim establishing the latter.

We have now strengthened this boundary in the revised Section 3: retrieval cannot return a state absent from its pool, Hit@k is conditional on coverage, and an operational candidate generator is not evaluated. These edits are highlighted in blue.

We address the framing, sparse baselines, representation diagnosis, and related work in the next comment.

AuthorsOpenReview ↗

Comment 2/2

Is transfer failure being presented as a positive result?

No. The paper presents both the capability within observed problems and the failure to transfer as central findings. The abstract reports near-chance performance on unseen domains. Section 4.2 is "Where generalization stops", Section 4.3 is "What limits transfer", and the limitations rule out domain-general planning. We do not ask the reviewer to treat the failed transfer as a success. We ask that the contribution be assessed as the controlled characterization of where learned textual transitions work and fail.

If n-grams win, what is new?

Their advantage is our own highlighted diagnostic result, obtained with the head, data, and candidate pool fixed. In Table 7, character n-grams reach 79.4% Hit@5 on unseen problems across three domains versus 56.8% for frozen Llama-3.3-70B embeddings. We explicitly recommend simple representations when the state text has a stable template. Lifted STRIPS induction, given symbolic facts, reaches 99.8%, which we report as an oracle reference rather than conceal.

The value of this comparison is that it isolates the state representation as an important constraint on transfer. It does not claim that LLM embeddings outperform sparse features on these benchmarks or on untested free-form text.

Isn't the failure of LLM embedding invariance obvious?

The hypothesis may be intuitive. The experiments measure which invariances fail and how they affect transition learning. Reordering the same state's facts moves its BGE-M3 embedding 1.13 times further than an actual action (Appendix N). On unseen problems, 57.6% of Ferry and 65.5% of Logistics test transitions involve ground actions absent from training, despite sharing known action schemas (Section 4.3). These observations identify fact-order sensitivity and object-role binding as concrete obstacles, rather than simply labeling the embeddings unsuitable.

The learned head also matters. On nine-domain Extrapolation, removing the action-disambiguation loss reduces Hit@5 from 54.6% to 35.3% (Appendix G.2). This shows that learning action effects contributes to generalization within domains, although it does not solve transfer to unseen domains.

Does LoRA solve the problem, and how does this compare with prior work?

No. The 3.8% to 9.3% full-pool Hit@1 improvement on unseen problems is a preliminary BGE-M3 result, not evidence of useful out-of-domain performance. We report the low absolute accuracy and interpret adaptation as a direction to investigate.

Thought of Search is cited in the introduction, though its distinction deserves more space. It uses an LLM to construct executable successor and goal-test code for search. EmbedPlan evaluates learned vector transitions without such a validated symbolic program. Next-Latent Prediction Transformers trains a backbone with a next-latent objective, whereas our question is what frozen encoders already support. We will make these distinctions explicit in Related Work, without claiming that our method outperforms either approach.

What supports Informativeness 1 and Soundness 1?

The review recognizes careful controlled experiments, yet the stated reasons for the lowest ratings do not engage with what those experiments establish. Sparse representations outperforming a 70B encoder, fact-order perturbations exceeding real action transitions, and the measured failure of cross-domain transfer are useful findings even if the underlying intuition is familiar. A negative result can be informative without making a deployable planner.

We agree that weak full-pool top-1 accuracy and missing search integration limit practical significance. But those limitations are measured and disclosed, not evidence that the reported experiments are invalid. We have revised the candidate-pool explanation in Section 3, and will clarify the related-work distinctions.

We would appreciate reassessment of Informativeness, Soundness, and the overall recommendation.

If a decisive concern remains, could you identify the specific invalid experiment, unsupported claim, or prior result that already establishes the central diagnostic findings? We would be glad to address it during discussion.

AuthorsOpenReview ↗

Rebuttal Discussion Follow-up: Revised Manuscript and Feedback

Dear Reviewer,

As the discussion deadline approaches, we would appreciate your feedback on our detailed rebuttal.

We have addressed your concerns about the evaluation settings and experimental validity, and uploaded a revised manuscript with changes highlighted in blue.

We would sincerely appreciate your reassessment in light of these clarifications. If a decisive concern remains, we welcome the opportunity to address it.

Thank you for your time and consideration.

Reviewer_v2FJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

EmbedPlan trains a lightweight transition model on top of frozen LLM text embeddings. A frozen LLM encodes the natural-language state s and action a into vectors, a small learned "projection head + transition network" predicts the next-state embedding, and nearest-neighbor retrieval returns a real successor state that is fed back. This turns each planning step from a token-by-token LLM call into a cheap vector operation (~0.17 ms per step with cached embeddings, about 10,000× faster than GPT-5.4 generation).

Its three contributions are: (1) this modular "frozen encoder plus learned dynamics" transition component; (2) a controlled six-protocol evaluation ladder ordered by exposure (Interpolation, Plan-Variant, Extrapolation, Multi-Domain, Cross-Domain, Leave-One-Out) across 9 classical PDDL domains and 4 encoders; and (3) a systematic diagnosis.

Validation results: near-perfect within observed problems (Hit@5 99.7%), 92–99% of single-step accuracy retained over multi-step rollouts, and better successor ranking than GPT-5.4 on the same candidates. However, accuracy drops to 54.6% on unseen problems of an observed domain and to near chance (6.6%) on unseen domains. By holding the head and data fixed and varying only the state representation, the authors localize this transfer bottleneck to the frozen state representation itself (embeddings cluster by surface form and object identity, and are sensitive to fact order, rather than by abstract structural role), not to the learned transition network. They conclude that an object-aware, order-invariant state encoder is the concrete target for future transfer.

Critical strengths

(1) Lightweight alternative to per-step LLM calling, suited to short, data-rich tasks. On these benchmark domains, plans are short (mean ~2.2 steps) and transition data is abundant (~3M transitions), so freezing the LLM and training only a tiny head (<500K params) on cached embeddings turns state prediction into a ~0.17 ms vector operation (~10,000× faster than GPT-5.4), which directly attacks the main cost barrier of LLM-based planning.

(2) A controlled diagnostic, not just a benchmark number. By fixing the head and pool while varying only the state representation, the paper attributes the cross-domain failure to frozen embedding geometry rather than the learned transition, turning a negative result into an actionable target for future encoders.

Critical weaknesses

(1) The efficiency claim is benchmark-dependent and partly an artifact. The ~10,000× speedup compares cached-state-embedding retrieval against an online LLM API call, and it ignores the one-time per-domain cost (~2 h of embedding extraction per domain with Llama-3.3-70B, plus ~15 min training). Because cross-domain transfer fails, this cost is not amortized, so "lightweight" holds only for repeatedly planned domains within observed problems rather than as a general property.

(2) The core promise of a reusable transition learner does not actually hold. Accuracy is near-perfect only inside observed problem manifolds; on unseen problems Hit@5 falls to 54.6% and to near chance (6.6%) on unseen domains. The method thus behaves as per-domain memorization plus smooth interpolation, and the authors themselves attribute this to the frozen state representation, leaving the central claim (learned transferable dynamics) unproven.

Questions for authors

None.

3 comments
AuthorsOpenReview ↗

Comment 1/2

Thank you for recognizing the diagnostic value of our representation analysis and its implications for future encoders. We agree that efficiency and generalization are central to assessing this work. However, both critical weaknesses appear to evaluate broader claims than the paper makes, while overlooking measurements that explicitly address the concerns.

W1. Does the efficiency claim ignore per-domain training costs?

No. The paper explicitly quantifies these costs, states that they cannot be amortized across domains, and limits the benefit to environments used repeatedly.

The approximately two hours of Llama-3.3-70B embedding extraction and 15 minutes of head training quoted in the review come directly from Appendix B.5. That appendix also states:

"Training is a real tradeoff and is not amortized across domains, since cross-domain transfer fails."

Section 3 ("Intended use") makes the same restriction in the main text. It identifies per-domain training and estimates a break-even point of approximately 4,300 transitions against the measured autoregressive baseline with Llama-3.3-70B, or 640 with MPNet.

We therefore agree that training is unattractive for a domain encountered only once. That is precisely why the paper targets repeated planning within a domain. The criticism that these costs are ignored does not reflect the analysis already provided.

We also distinguish the latency measurements. Appendix B.5 reports 0.168 ms with cached embeddings and 18.6 ms including BGE-M3 query encoding, corresponding to 11,215x and 101x speedups against the measured 1,888 ms LLM API generation baseline. These are different inference settings, not interchangeable end-to-end planning measurements. The comparison also assumes available candidate states and contrasts retrieval with generation, so it should not be interpreted as equal-functionality planning performance.

We agree that these conditions should be more prominent and will explicitly repeat the caching, per-domain amortization, and measured-baseline qualifications alongside the efficiency claims.

W2. Is learned transferable dynamics an unproven central claim?

Successful cross-domain transfer is not our claim. Demonstrating its failure and investigating why it occurs is one of the paper's three stated contributions.

The abstract explicitly reports near-chance performance on unseen domains. Section 4.2, "Where generalization stops", measures that failure: Cross-Domain reaches 6.6% Hit@5 and Leave-One-Out reaches 9.2%, against 3.9% chance (Table 5). Section 5.1 states that EmbedPlan is a domain-specific transition component, not a domain-general planner.

The paper instead distinguishes two questions: can the model learn action-conditioned transitions within a domain, and how far do those transitions generalize?

There is evidence of learning beyond simple lookup, although extrapolation remains limited.

A tabular one-hot model trained on the same Logistics data reaches 8.3% Hit@1 within observed problems, compared with 96.3% for EmbedPlan (Section 4.3, Appendix L). This shows that simple state lookup does not explain the observed performance. It does not exclude every form of memorization.

The stronger evidence comes from unseen-problem evaluation. On Extrapolation across nine domains, the learned transition achieves 54.6% Hit@5 over 128 candidates. Removing our action-disambiguation objective, which contrasts predictions from alternative actions applied to the same state, reduces Hit@5 to 35.3% and action Acc@5 from 16.2% to 4.8% (Appendix G.2).

This ablation shows that learning action-specific effects materially improves predictions on unseen problems. It does not establish robust transfer to unseen domains, and we do not claim otherwise.

The limited extrapolation is also real. The paper reports substantial deterioration on held-out problems and very low exact whole-trajectory accuracy. These findings motivate our investigation of the representation bottleneck rather than being results we attempt to explain away.

Our disagreement is therefore not about whether cross-domain generalization fails. We agree that it does. It is about treating that documented failure as evidence against a central claim of successful cross-domain transfer that the paper explicitly disclaims.

We discuss the scientific contribution and the Informativeness assessment in the following comment.

AuthorsOpenReview ↗

Comment 2/2

Is the representation diagnosis scientifically informative?

The paper does more than observe that transfer fails. It identifies concrete representation properties associated with that failure through controlled comparisons.

Your review recognizes the representation diagnosis as an actionable direction for future encoders. We believe this is an important point when assessing Informativeness.

1. The representation, not just the transition head, substantially changes generalization.

Table 7 holds the transition architecture, training data, and candidate pool fixed while varying the state representation. On unseen problems across Ferry, Logistics, and Goldminer, character n-grams achieve 79.4% Hit@5, compared with 56.8% for Llama-3.3-70B embeddings.

The same head therefore generalizes substantially better when given a representation that preserves small factual edits. This supports the representation as a major bottleneck, although it does not establish that no other bottlenecks exist.

2. The experiments identify specific missing invariances.

Reordering a state's facts without changing its meaning moves its BGE-M3 embedding 1.13 times further than applying an actual action (Appendix N).

On unseen problems, 57.6% of Ferry and 65.5% of Logistics test transitions also involve grounded actions absent from training, although their action schemas are known (Section 4.3). The analysis points to fact-order sensitivity and object-role binding as concrete problems for frozen state representations.

These are measured properties, not simply the assertion that pretrained embeddings have the wrong geometry.

3. The results distinguish insufficient representations from insufficient action supervision.

Our transition objective explicitly contrasts the predicted outcomes of alternative actions applied to the same state. The action-loss ablation in Appendix G.2 shows that this supervision is consequential: removing it reduces unseen-problem Hit@5 by 19.3 percentage points.

Together with the fixed-head representation comparison, this helps distinguish what the learned transition can capture from information that the frozen representation fails to preserve.

The experiments do not prove that an object-aware, order-invariant encoder will solve generalization. They identify a specific, testable direction for improving it.

How should these findings affect the assessment?

We find Informativeness 1 difficult to reconcile with the review's own recognition of an actionable representation diagnosis.

We do not claim to introduce a complex transition architecture, solve domain-general planning, or outperform simple sparse features on templated states. The contribution is a lightweight action-conditioned transition component, a controlled evaluation across six protocols and nine domains, and an empirical account of where frozen language representations help and fail.

The unfavorable findings are part of that contribution. In particular, the n-gram advantage, fact-order sensitivity, and cross-domain failure establish limitations that would be easy to miss if the evaluation only reported high accuracy on familiar problems.

We fully accept that practical applicability and novelty should influence the overall recommendation. However, the two critical weaknesses in the review concern limitations already quantified or explicitly stated, and neither identifies an invalid experiment.

Our request

We would appreciate reconsideration of Informativeness and the overall recommendation based on the paper's actual claims, the controlled findings, and the cost and generalization limitations already reported.

In particular, if the representation diagnosis is considered an actionable contribution, we would appreciate understanding what justifies the lowest Informativeness assessment.

If a broader efficiency or transfer claim remains unsupported, could you identify the specific sentence? We will qualify it directly in the revised manuscript.

We welcome further discussion before the review period closes.

AuthorsOpenReview ↗

Rebuttal Discussion Follow-up: Revised Manuscript and Feedback

Dear Reviewer,

As the discussion deadline approaches, we would appreciate your feedback on our detailed rebuttal.

We have addressed your concerns about efficiency, generalization, and the scientific contribution, and uploaded a revised manuscript with changes highlighted in blue.

We would sincerely appreciate your reassessment in light of these clarifications. If a decisive concern remains, we welcome the opportunity to address it.

Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Call for reviewers to engage the rebuttal

Dear Reviewers,

As the discussion period approaches its end, we sincerely appreciate your time and consideration.

We have addressed the reviews point by point, provided supporting experimental evidence, and uploaded a revised manuscript with changes highlighted in blue.

We would particularly appreciate hearing from reviewers who have not yet responded to our rebuttal. Do our clarifications change your assessment? If decisive concerns remain, could you identify them so we can address them before the discussion closes?

We value this opportunity for scientific discussion and remain fully committed to engaging constructively.

Thank you again, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.