TopoRerank: Bounded Structural Reranking for Fixed-Candidate GUI Action Selection
Abstract
Graphical user interface (GUI) agents may choose the wrong control even when the target is present, as semantically similar candidates receive close scores. End-to-end accuracy conflates candidate coverage, ranking, target assignment, and final selection. To isolate this decision, we propose TopoRerank under a protocol that fixes the candidate set, Stage–1 scores, target matcher, split, and evaluator. The method adds a bounded, instruction-independent residual from candidate-local accessibility-tree features, changing only close competitors. On package-disjoint AndroidControl, it improves a text-retrieval baseline by 3.62 percentage points across three seeds, with a positive package-equal interval; the gain falls to 0.53 percentage points under a stronger learned text-only ranker. An area-free nearest-center control is positive under the primary retrieval baseline but inconclusive under lexical scoring; app-disjoint GUIrilla results are positive after target-domain refitting. Overall, local hierarchy offers a conditional correction to frozen rankings, not a replacement for semantic ranking or candidate recall.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.