acceptodds
Withdrawn from ICLR 2027

Does This Action Still Explain the Task? Reverse Scoring for Diffusion Language Model Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion-based large language models (dLLMs) promise to break the sequential latency bottleneck of autoregressive agents through parallel decoding, but recent evaluations show this efficiency does not transfer to embodied agentic competence: dLLM-backed agents repeatedly fall into retry loops, re-issuing an action long after it has failed. We give a mechanistic account of this failure and a training-free remedy. We trace the retry loop to the adaptivity of masked decoding: the sampler commits the positions it is most confident about and defers the uncertain ones, and at a failure state the context already offers a confident fill for the deferred decision, i.e. the failed action itself, so the retry is committed without the failure feedback ever being confronted. We model the resulting distortion of the action distribution as a *task-blind corruption*: contextually salient actions (e.g., the action just taken) receive inflated probability by a factor that depends on the state and the action but not on the task. Under this model, we analyse an invariance proposition: the task-blind factor cancels exactly from the *reverse conditional*, i.e. the likelihood of the task given the state and a candidate action, which coincides with the task posterior of an idealized uncorrupted model. Masked dLLMs evaluate the reverse conditional natively, unlike autoregressive models, by masking the task tokens and denoising, at the cost of a few parallel passes per candidate. We instantiate the rule as *Reflect Reverse* and evaluate it on four multi-turn embodied benchmarks, where it improves task success and progression rates over forward-scoring baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion LLM, training free, embodied decision making

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dJqMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  1. The author summarizes/defines a problem existing for DLM agents (task-blind corruption).
  2. The author proposes a solution to help with this problem in agent tasks, and the results are good.
Critical strengths
  1. I like the analysis of the problem. The author offers a clear, testable explanation for a known failure. This paper offers a concrete cause: confidence-first decoding copies the failed action from the history before the open “what now?” decision is made.

  2. The proposed idea is simple and effective.

Critical weaknesses
  1. Simple baseline alternatives are missing (the gain can be due to an overly simplified baseline): random pick after removing duplicates, majority vote / self-consistency, simply banning the action that just failed, etc.

  2. The problem setting (background) should be expanded. A lot of content in section 1 should be in section 2. The reading of paragraph 1 is painful. Not natural at all. A broken cross-reference. The Intro says “Section 5 develops the mechanism”, but Section 5 is Related Work, and the mechanism gets only a few sentences there. In 4.2, the sentence “SR improves on ALFWorld, ScienceWorld, and BabyAI…” appears twice in a row.

  3. Temperature or diversity effects. Retries depend strongly on temperature, yet only two temperatures are tested. A higher-temperature One Pass might close much of the gap. Also, given the low success rate of the task, the gain can be hugely contributed to diversity caused by randomness.

Questions for authors
  1. Dream models? It seems only the LLaDA family is utilized, which is a bit unnatural to me (not critical, though).

  2. Inherently, I feel like this makes a natural connection to the nature of decoding for dLMs. Has the author considered different remasking strategies? Although I understand this is a higher-level setting instead of a decoding strategy paper.

  3. Refer to weaknesses 1/3.

Reviewer_bPWZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper focuses on the problem that masked dLLM agents keep re-issuing an action that has just failed. The explanation offered is that masked dLLMs decodes the tokens they are most certain about first, and at a failure state the certain thing to write is the action already sitting in the context, so the model copies it instead of weighing the alternatives. To solve the problem, the paper proposed Reflect Reverse that asks how well the situation plus a candidate action explains the task, which a masked model can estimate by masking out the task description and checking how confidently it fills it back in. A short theory section argues that a bias which ignores the task cancels out of this reverse score and bounds how far the corrected choice can be from an ideal one. The method is evaluated on benchmarks including ALFWorld, ScienceWorld, BabyAI and Jericho using LLaDA backbone.

Critical strengths
  • The paper focuses on an interesting question of dLLM agents
Critical weaknesses
  • Lack of evidence that reverse scoring beats a trivial control. The paper's own mechanism implies the fix: draw the same number of candidates, drop repeats of the failed action, vote. The control, self-consistency over the candidates, and an AR agent under the same protocol are all missing. There is no latency reported. A no-repeat control might likely match the method at no cost.
  • The theory does not cover the algorithm. Prop. 3.1 and Thm. 3.1 concern the exact reverse conditional under the assumption of coherent forward and revrese conditionals which masked models violate and the paper fails to checks. The algorithm instead computes a Monte Carlo estimate from randomly masked task tokens from Eq. 2, and the gap to the exact conditional is only assumed bounded. The mechanism contradicts its own evidence: retries are blamed on unmasking order, but Fig. 1 uses a likelihood estimate with no unmasking order, and the order ablation is missing. The state check uses an alternative chosen by Qwen, so Qwen preferring it is by construction.
Questions for authors

How does the proposed method work for other dLLM families in addition to LLaDA, e.g., the Dream family?

Reviewer_fx7JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies repeated-action loops in masked diffusion language model agents and proposes Reflect Reverse, a training-free action-selection method. It generates and deduplicates candidate actions, scores each candidate by reconstructing masked task-description tokens conditioned on the interaction history and the candidate, and samples an action using a softmax over these scores. Experiments with LLaDA-8B-Instruct and iLLaDA-8B-Instruct on four interactive benchmarks report improvements in success and progress over One Pass in most configurations and lower repetition in all evaluated configurations.

Critical strengths
  • A practical, training-free selection rule with useful empirical gains. Reflect Reverse uses the backbone's existing masked prediction capability, without a separately trained critic or candidate execution in the environment.
Critical weaknesses
  • The performance–compute tradeoff introduced by additional sampling is insufficiently characterized. Reflect Reverse adds candidate-action sampling and multiple random-mask scoring passes per candidate, but the paper does not report how performance changes with the number of candidate actions or reflection samples. The experiments use a generation attempt limit of 20 and 64 or 128 reflection samples per candidate, without a sampling-budget ablation. It is therefore unclear how much sampling is needed to obtain the reported gains, whether performance saturates at a smaller budget, and how additional computation translates into improved task success.
Questions for authors
  • Could you report a pass@N-like candidate-coverage or oracle-selection metric on the same sampled action sets, together with Reflect Reverse's gap to that reference? This would help distinguish failures to generate useful actions from failures to select them, and quantify the remaining headroom for candidate selection.
1 comment

Thank you for your helpful feedback. I plan to withdraw the submission, but before doing so, could you please clarify what you mean by an “oracle-selection metric”? Your clarification would help me improve the paper for my next submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.