acceptodds
Withdrawn from ICLR 2027

HERA: Hierarchical Response-Aware Supervision Allocation for Efficient VLA Fine-Tuning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) models provide a foundation for robotic manipulation. Fine-tuning on demonstrations collected in target environments enables adaptation, but can require lengthy training. One source of this inefficiency is repeatedly training on demonstrations with little additional benefit at the current learning state. Existing data-selection methods concentrate training on useful supervision through subsets selected by behavioral structure or estimated utility. Yet fixed subsets cannot track changing utility: retained samples may become less useful, while excluded samples remain unavailable when their value grows. Our key insight is that training utility evolves at two coupled granularities: learning from one action chunk can change the value of physically related chunks, while the remaining benefits within a chunk can evolve differently, or oppositely, across its action subspaces. These effects depend on trajectory structure and the current learning state. We introduce HERA, a HiErarchical Response-Aware Supervision Allocation framework for flow-based VLA fine-tuning. A shared response estimator combines physical correspondences with the current learning state to predict changes to a fixed training-side reference objective, sharing sparse probe responses across related chunks. Dynamic chunk sampling adapts training frequencies while keeping all chunks eligible. Dynamic action-subspace weighting assigns normalized, chunk-specific weights over native action subspaces. Both allocations are jointly optimized and periodically refreshed. On LIBERO, HERA reaches 80% success with 53.6% fewer optimizer updates than LoRA, yielding a 1.93× wall-clock fine-tuning speedup including response-estimation and allocation overhead. At 20K updates, HERA improves average success rate over LoRA by 9.38 and 10.75 percentage points on LIBERO and PiPER-X, respectively.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Efficient Fine-Tuning, Adaptive Supervision Allocation, Robot Learning, Data Selection, Flow Matching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7gUeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose HERA, an efficient fine-tuning framework for flow-based vision-language-action models, aiming to improve learning efficiency by dynamically allocating training supervision resources. The core idea of this method is to allocate supervision at two interrelated granularities. At the action-chunk level, it adjusts the visitation frequencies of different training samples through dynamic sampling, and at the action-subspace level, it assigns different loss weights to the native physical action subspaces of each action chunk according to current learning needs. To coordinate these two decisions, the authors further design a trajectory-guided training response estimator that combines physical action correspondences with the current learning state of the model, uses a small number of calibration responses to predict the effects of different supervision signals on fixed training-side reference targets, and jointly optimizes the sampling distribution and subspace weights through periodic updates. Experimental results show that HERA achieves a higher average task success rate under the same number of optimization steps.

Critical strengths

This paper focuses on the important practical problem of improving VLA fine-tuning efficiency and takes a novel perspective: the learning utility of action chunks and their internal action subspaces evolves dynamically with training. Building on this perspective, the authors propose a hierarchical response-aware supervision allocation framework that dynamically and jointly adjusts the sampling frequencies of sample chunks and the supervision weights of subspaces according to the model's current learning state. Compared with the conventional practice of treating all expert demonstrations and action dimensions uniformly, this design establishes a more scientific and efficient training resource allocation mechanism. The substantial improvements in training efficiency demonstrated in the experiments strongly support the practical value of this approach.

Critical weaknesses

None

Questions for authors

1.Although HERA demonstrates promising performance on LIBERO-Long, its real-world evaluation is limited to four simple tabletop manipulation tasks. It remains unclear whether the proposed method remain effective in more challenging long-horizon tasks involving multi-stage task execution, contact-rich interactions, and accumulated execution errors. We encourage the authors to provide more empirical results to demonstrate the effectiveness of HERA on more complex multi-stage real-world manipulation tasks.

2.The simulation experiments are conducted only on LIBERO. Although the results show consistent performance improvements across its four task suites, it remains unclear whether these gains generalize to other simulation benchmarks with different task distributions, environment settings, and manipulation challenges. We encourage the authors to supplement the evaluation with experiments on other robotic manipulation benchmarks (RoboCasa or DROID) to examine the generalization ability and robustness of HERA.

Reviewer_f7vsOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

HERA aims to reduce the time needed to fine-tune a VLA model. It changes both how often each action chunk is sampled and how much weight is given to different action subspaces within that chunk. A shared response estimator predicts how candidate training updates would affect a fixed reference set. The method is evaluated using π0.5 with LoRA on LIBERO and four real-world tasks. The results are promising, but the evidence is limited to this fine-tuning setup.

Critical strengths
  • The reported speedups include response estimation and allocation costs. HERA reaches 80% LIBERO success about 1.93× faster than ordinary LoRA. The benefit continues at 90% success, where it takes about 53% of LoRA’s training time.

  • Sampling alone provides a substantial improvement, and adding action-subspace weighting improves the results further. Joint optimization also performs better than sequential allocation.

  • HERA improves average success across four tasks and works with joint-based actions as well as the Cartesian actions used in simulation.

Critical weaknesses
  • Using the same LoRA setup for all methods is a fair way to isolate the effect of supervision allocation. However, the paper does not compare against full fine-tuning or test HERA with full fine-tuning. It therefore remains unclear whether HERA plus LoRA is a better practical choice than a well-tuned full fine-tuning baseline. Matching the paper’s own LoRA peak also does not establish that HERA reaches state-of-the-art performance faster.

  • The method builds on adaptive sampling and loss weighting, so an important question is whether its specific response estimator is necessary. Removing physical correspondence features hurts performance, but this does not show whether the complete estimator is better than simpler signals, such as current losses, hidden features, or gradient alignment, under the same computation budget.

  • LIBERO uses only eight probe chunks, giving 24 calibration directions. The paper includes useful prediction diagnostics, but these cover only 24 excluded directions at each of three training snapshots. More evidence is needed to show that the probes cover the candidate pool and that predictions remain reliable between refreshes. Sensitivity to the reference set’s size, composition, and fixed flow draws would also help establish whether reducing reference loss reliably guides policy improvement.

  • The evaluation covers four LIBERO suites and one real-world setup with four tasks, all using π0.5 with LoRA. These results show that HERA works in the tested settings, but they do not establish broader generalization across datasets, policy architectures, robots, or unseen tasks. Additional benchmarks and experiments with a different backbone or embodiment would help show whether the gains extend beyond this particular setup.

  • The paper reports faster attainment of 90% LIBERO success and of its own LoRA baseline’s peak performance. This is useful evidence, but neither target is established as state of the art. It remains unclear whether HERA retains its speed advantage at higher success rates near a strong baseline’s best performance.

Questions for authors

Please see weaknesses

Reviewer_StYdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

HERA is a framework for accelerating flow-based VLA fine-tuning through joint supervision allocation. It dynamically adjusts both the sampling probabilities of action chunks and the relative weights of their native action-subspace losses. A shared response estimator combines physical trajectory correspondences with the current learning state to predict changes in a fixed training-side reference objective. These predictions guide a constrained allocation problem, while periodic refreshes and a rotating candidate pool control estimation cost. Experiments with π₀.₅ on LIBERO and real-world PiPER-X tasks show improved policy performance and reduced time to target success. Ablations examine the contributions of sampling, subspace weighting, joint optimization, physical correspondences, and refresh frequency.

Critical strengths

HERA connects subset selection, as studied by SIEVE [1] and DataMIL [2], with dynamic sampling and loss balancing, represented by InfoBatch [3] and FAMO [4]. Its distinctive contribution is jointly optimizing chunk visitation and chunk-specific subspace weights through a shared response model. The formulation captures interactions between the two allocation levels.

Experiments on LIBERO and PiPER-X show useful improvements, and the paper reports wall-clock savings with recurring estimation and allocation overhead accounted for. The ablations support the principal design choices. Implementing optimizer-response estimation and cached allocation within an ordinary fine-tuning loop also represents substantial engineering work, with no additional policy-inference procedure required.

Critical weaknesses

Important implementation and baseline details remain underspecified. The paper does not sufficiently specify the physical descriptors and prototypes, feature projections, reference readout weights, or regularization and solver settings. Selection and tuning configurations for SIEVE [1], DataMIL [2], InfoBatch [3], and FAMO [4] are also incomplete. These omissions make the method difficult to reproduce and the comparative efficiency results harder to audit.

The utility proxy needs stronger reliability validation. HERA optimizes a small, fixed training-side reference, whereas GLISTER [5] and CUPID [6] illustrate validation-based and closed-loop performance perspectives on data utility. The provided diagnostics support local response prediction, but sensitivity to reference selection, fixed flow draws, and estimation staleness remains unclear. Evaluation on a single policy backbone further limits the established scope.

[1] SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models

[2] DataMIL: Selecting Data for Robot Imitation Learning with Datamodels

[3] InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning

[4] FAMO: Fast Adaptive Multitask Optimization

[5] GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning

[6] CUPID: Curating Data your Robot Loves with Influence Functions

Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.