acceptodds
Withdrawn from ICLR 2027

FINGR: Learning Dexterous Hand Control for Real-World Rubik’s Cube Solving

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Manipulating a Rubik's Cube with one single dexterous hand is a challenging test of sustained, contact-rich control: the hand must execute successive layer turns while keeping the cube secure. Each turn requires some fingers to support the cube while others push a moving layer, release contact, and reset for the next move. To learn this coordination, we introduce (Future-supervised Interaction Network with Geometric Representations), a policy that combines finger-relative geometry with future interaction prediction. A shared point encoder expresses the cube relative to each fingertip and aggregates its points without depending on cubie indexing. Learned future tokens share the observation encoder and receive supervision for contact-force changes, layer-turn progress, and finger joint displacement at multiple time scales. The resulting representation conditions a flow policy that directly generates finger actions. On a real dexterous hand, our policy achieves around success over 300 turn attempts, compared with for the base flow policy. Integrated with grasping and table-assisted regrasping, the policy solves all ten scrambled cubes in a mean complete-system time of approximately seconds.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Dexterous Manipulation, Visuotactile Learning, Imitation Learning, Flow Matching, Task and Motion Planning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aJDVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents FINGR, a policy for real-world dexterous manipulation focused on solving a 2×2×2 Rubik’s cube with a single robotic hand. The method combines two main ideas: (1) a finger-relative geometric representation that encodes cube points relative to each fingertip, and (2) a future interaction prediction objective that supervises latent representations using multi-horizon predictions of contact-force changes, layer-turn progress, and finger joint motion. Experiments are conducted on a real robotic platform.

Critical strengths

1.The paper presents a solid real-world system with strong empirical performance. In particular, the reported 99% success rate on continuous layer turns and the successful end-to-end solving of 10 scrambled cubes suggest that the approach is robust in practice, not only in isolated demonstrations. The experimental section is also reasonably complete, with baseline comparisons and ablations on the proposed components.

  1. The finger-relative geometric representation is a sensible design choice for this task. Instead of relying only on global cube features, the policy explicitly models geometry relative to each fingertip, which fits the contact-rich nature of the manipulation problem. The ablation results indicate that this component provides a meaningful improvement over the base flow policy.
Critical weaknesses
  1. The manipulation setting is fairly constrained, which makes the scope of the contribution narrower than the paper sometimes suggests. The system relies on a largely fixed grasp where the thumb and middle finger continuously stabilize the cube, while the remaining fingers execute predefined U/L layer turns. There is no finger gaiting or substantial in-hand reconfiguration during manipulation. As a result, the learned behavior appears closer to a specialized controller for a particular cube-turning setup than a more general dexterous in-hand manipulation policy. This limitation is important because much of the claimed significance comes from its relevance to broader dexterous manipulation.

  2. The evaluation is limited to a tagged 2×2×2 cube under a carefully engineered pipeline, including marker-based reconstruction, restricted move sets, and planner-assisted table regrasps. It remains unclear whether the proposed representation and future prediction objectives would transfer to less structured settings, larger cubes, or other articulated object manipulation tasks. The paper demonstrates strong performance within its setup, but the evidence for broader applicability is limited.

Questions for authors
  1. The manipulation policy is evaluated under a fixed grasp configuration with only U/L turns. How much of the reported performance depends on this constrained setup? For example, can the same policy architecture handle settings that require significant in-hand reorientation or changing support fingers during manipulation?

  2. The paper motivates the method as improving dexterous in-hand manipulation more broadly, but all experiments are on a tagged 2×2×2 cube with marker-based reconstruction. Do the authors have any evidence that the proposed finger-relative representation and future-token objectives transfer beyond this specific task setting, e.g., to other articulated objects or less constrained cube manipulation setups?

  3. The complete system relies on planner-assisted table regrasps when the current grasp cannot realize the remaining moves. Could the authors clarify how often the learned policy itself fails due to grasp limitations rather than execution errors? This would help separate manipulation capability from planning constraints in the reported full-system results.

Reviewer_YWezOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a method for solving a 2x2x2 cube using a dexterous hand. The method is a composition of a planner that handles high-level planning and regrasping and a low-level policy that conditions on finger joint states, tactile sensing, and cube poses to execute cube layer turns. The main components of the low-level policy are the relative finger-cube state encoding (rather than global) and auxiliary future state prediction (on contact force changes, layer turn progress and finger joint motion). The paper defines the overall method for solving the cube well and includes comparisons to other architectures for the low-level policy and includes ablations on relative finger-cube geometry and the auxiliary future-state objectives. Although the paper is well-written in terms of its method description, the paper overall lacks proper motivation for why solving this problem is important and why learnings from this task can inform other contact-rich in-hand dexterous manipulation methods. Also the relative finger-object representation for dexterous manipulation and future state prediction are existing methodologies that the authors identify in the related work. For this reason, the paper would benefit from defining the core scientific novelty it contributes.

Critical strengths

S1: The method is properly shown to increase success rates on performing the layer rotations compared to baseline architectures (Table 1). The ablations also demonstrate that the local finger-object state encoding and the auxiliary objectives both help improve success rates (table 2). The evaluation includes execution time and drops/timeouts, which helps understand the reduction in success rates across methods. The 300 attempts per method in Table 1 and the 10 full cube solves in Table 3 show the system was heavily evaluated in the real world.

Critical weaknesses

W1: The main weakness is the lack of proper motivation for why the cube solving task was picked and how their methodological contributions provide insight into in-hand dexterous manipulation. Thus while the method itself is shown to improve success rates, it’s not apparent how the learnings would broadly apply to dexterous manipulation across tasks. The authors do identify this as a source of future work in the conclusion, but do not provide a hypothesis for scenarios they expect their method to work well. I recommend adjusting the introduction to motivate the chosen task and its importance. The introduction also does not contain any citations; including citations will help contextualize the motivation of this work as well. Adjusting the introduction and related work would address this concern.

W2: It’s not clear what the particular methodological novelty is. The authors identify prior work that already does contact-based grasp refinement and hand-surface encodings, thus it’s not clear how the proposed finger-relative geometry encoding is different from existing work, beyond the brief description provided in the related work section. Also the authors identify future state prediction as a common approach (such as in WAM models), so it’s not clear if the particular future prediction terms are new. Detailing the specific differences and their importance would address this concern. Furthermore, there are no comparisons to other choices of prior finger-relative geometry encoding nor other auxiliary terms used in prior work.

W3: Limited ablations. The authors do not ablate the individual terms in the future prediction loss (contact force change, layer turn progress, and finger joint motion). An ablation with these auxiliary terms, but without the local geometry encoding is also missing. Adding these ablations would satisfy this concern.

Questions for authors

Q1: Can you provide additional details on the Base Flow Policy? My understanding is that it is the same architecture in Figure 2, but without the local geometry encoding and future state prediction. Expanding the baselines section of 4.1 would address this.

Q2: Do these method contributions for local geometry and auxiliary objectives apply to other dexterous manipulation tasks? Why was the cube layer rotation task chosen in particular?

Reviewer_mFxEOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

In this paper, a method for solving 2x2x2 Rubik’s Cube, called FINGR, is proposed. It defines task-specific geometric representations that improve the performance of a flow-based policy. The method was evaluated on a real robot and compared against common imitation learning baselines.

Critical strengths

Solving Rubik’s Cube is an interesting problem for evaluating dexterous manipulation methods.

Critical weaknesses

The method is very specific for solving the Rubik’s cube. t does not transfer to other domains, and such the novelty and contribution is very limited.

Existing methods solve the more complicated and more common Rubik’s cube (3x3x3), such as in [1].

[1] OpenAI, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, Jonas Schneider, Nikolas Tezak, Jerry Tworek, Peter Welinder, Lilian Weng, Qiming Yuan, Wojciech Zaremba, and Lei Zhang. Solving Rubik’s Cube with a Robot Hand.

Questions for authors

Could you method be used for other manipulation task? If yes, how? This matters because it indicates the impact of the proposed method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.