acceptodds
Under review as a conference paper at ICLR 2027

Decoder Silencing: Cloned GUI Agents Know When to Ask, and Greedy Decoding Discards It

Abstract

A GUI agent that sends messages, deletes files or makes payments should ask before acting on an instruction it cannot resolve. Policies cloned from demonstrations routinely ask on no such instruction, and the field reads a ask-rate as a behaviour the supervision failed to teach. We measure the probability such a policy puts on asking with one teacher-forced forward pass per held-out state, no rollouts and no outcome labels. Across reruns of a single recipe, with nothing varied that the trainer reads, it spans to . Greedy decoding compares each against one half, so a run holding substantial ask-mass and one holding almost none deploy at the same ask-rate. We call the first case decoder silencing: the behaviour is present in the belief and discarded at the decoder, not missing from the policy. The distinction is actionable: only a silenced behaviour can be repaired without retraining, and the probe says before deployment which runs a decode-time threshold will repair. We test this on AmbiGUI, a testbed we release where ambiguity comes from environment state rather than instruction text and every instance is verified on both sides: the question resolves it, guessing causes an irreversible action. On a pre-registered set it repairs about half of them, cutting irreversible wrong actions and raising task success, though a minority of runs come out worse on each. On AndroidWorld the probe is blind before training, when the tiers do not separate, and names what the agent does after. There, a backbone trained on whole episodes and deployed live on the emulator asks on every ambiguous instance and finishes of them with no irreversible error. No arm completes a task when its demonstrations stop at the decision rather than the finished state. Put plainly, a policy that never asks has usually been silenced rather than left untaught — so a ask-rate should be measured before it is believed.

Primary area:
foundation or frontier models, including LLMs
Keywords:
GUI agents, behavior cloning, clarifying questions, decoding, imitation learning, ambiguity, benchmark, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LbQDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper claimed that GUI agent when the task is underspecified, the agent gives 0-% ask-rate even with a substantial ask-mass, due to greedy decoding. The paper calls this phenomenon decoder silencing. The paper also proposed a dataset called AmbiGUI and conduct some experiments.

Critical strengths

None

Critical weaknesses
  1. This paper is extremely difficult to follow for human readers. For example, in contribution #2, "The demonstration rate decides whether the lottery exists at all." reads non-sense to me: What is the demonstration rate? What is the lottery? All of these are not obvious from the context and not explained in the following text. Similar experiences repeatedly happen during my reading of these paper. Without a significant improvement on the readability, the proper evalution of the contribution in the paper will be unlikely.
Questions for authors
  1. Can you explain again the contribution of this paper? I found it fairly difficult to understand, as a human reviewer, except the first decoder silencing part.
Reviewer_gnpBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper distinguishes failing to learn a clarification behavior from failing to express it under greedy decoding. It measures ASK-token probability on teacher-forced action prefixes and compares ambiguous with unambiguous instances. It then applies a decoding-time logit offset to encourage asking without changing the model weights. Experiments on the proposed AMBIGUI testbed show substantial variation across nominally identical training runs and improvements from thresholding on replayed episodes. Additional live AndroidWorld and reached-state evaluations examine whether the probe and intervention remain useful outside the constructed decision contexts.

Critical strengths
  1. The repeated-run experiments establish an interesting sensitivity to training arithmetic. Across 40 runs, ASK probability spans 0.0002–0.9907, while the deterministic-kernel control collapses the variation. The additional fixed-seed trainer experiments strengthen the observation that similar training configurations can produce very different deployed asking behavior.

  2. The replay intervention has measurable benefits, and the adverse results are reported. Across 24 screened checkpoints, Table 19 reports mean DAMAGE decreasing from 43.0% to 17.2% and success increasing from 54.5% to 77.5%. The paper also reports checkpoints that worsen and live experiments that fail to reproduce the improvement. This makes the empirical record more informative than a selected successful demonstration.

Critical weaknesses
  1. The cleanest evidence is the same-domain comparison in Appendix L.6: all seven AMBIGUI checkpoints distinguish the tiers on constructed trajectories, but only three do so on reached trajectories. For one checkpoint, AUROC drops from 1.000 to 0.435 on the same instances. The live results also do not demonstrate the claimed repair under successful task execution: the competent whole-episode AndroidWorld model already asks, while the silent checkpoint whose asking is restored completes no tasks. These results support a narrower finding about probabilities under constructed prefixes, but weaken the interpretation that useful clarification behavior is present in the deployed policy and merely suppressed by decoding.

  2. The preregistered separation-window test does not convincingly predict recovery under its registered transition-based outcome, and the later 0.05 mean-gap cutoff was selected after observing results. The controls in the screened campaign are also problematic: all five rejected checkpoints already ask under greedy decoding, so their zero recoveries are guaranteed by the definition of recovery. This does not test whether the screen distinguishes repairable from nonrepairable silent checkpoints. The stronger statements that the probe identifies repairable models before deployment, or that a small gap rules out decoding-time repair, therefore remain unsupported.

Questions for authors
  1. Is there a held-out, task-competent live checkpoint that is silent under greedy decoding and becomes selectively asking—with improved task outcomes—using a threshold fixed from disjoint probe states?

  2. How does the frozen screening rule perform on an independent population containing both repairable and nonrepairable silent checkpoints, rather than controls that already ask?

  3. What explains the demonstrated-versus-reached discrepancy in Table 18? Can the probe remain informative when the trajectory prefix and deployment implementation are matched?

Reviewer_9cLAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether a cloned GUI agent’s failure to ask clarifying questions reflects absent behavior or suppression by greedy decoding. It measures ASK probability at teacher-forced decision states and proposes changing the decoding threshold when ambiguous and unambiguous states are separable. AMBIGUI supplies controlled ambiguity and measurable irreversible errors. Repeated training runs show substantial variation in ASK probability, and replay experiments demonstrate improvements from threshold adjustment for some checkpoints. The paper also reports AndroidWorld experiments and substantial limitations in transferring replay results to live execution.

Critical strengths
  • The distinction between action probability and behavior under a particular decoder is diagnostically useful. The repeated-training experiments and deterministic-kernel controls provide evidence that numerical variation can produce substantially different clarification behavior near a decision boundary.
  • The paper reports negative findings and limitations in considerable detail, including unsuccessful live repair, over-asking, and checkpoints that cannot complete tasks. These disclosures help distinguish the controlled empirical observation from its practical applicability.
Critical weaknesses
  • The broad deployment interpretation exceeds the strongest evidence. Section 6 reports two cold starts that remain at 0% asking on live emulators despite reaching 100% under the threshold intervention on replay, explicitly meeting the paper’s stated falsification condition for Claim 2. The replay finding remains informative, but the title and broader claim of decoder-only repair need to reflect this limitation.
  • The probe’s interpretation as evidence that the agent “knows when to ask” is not sufficiently established outside constructed decision states. The paper reports weak separation on common instance sets or policy-reached states. Moreover, the controlled demonstration rate creates the regime of interest, while its prevalence under realistic data collection remains unestablished.
Questions for authors
  • Given the live-execution counterexamples, what precise version of Claim 2 remains supported? Should it be restricted to replay-conditioned decisions?
  • Can probe states representative of actual policy-reached states be identified before deployment? What predictive accuracy does the probe achieve on those states?
  • Under the actual action serialization, when is greedy selection equivalent to comparing ASK probability against 0.5? How does the interpretation change when probability is distributed across multiple competing tokens or action sequences?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.