acceptodds
Withdrawn from ICLR 2027

Human Demonstrations and RL Data Yield Different Chunking Effectiveness: Activity Sparsity as a Key Indicator

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Action chunking has emerged as a promising technique for robotic learning, yet it remains unclear when an action-chunking policy can be more effective. Our extensive empirical studies reveal that action chunking policies trained on human demonstrations perform well on standard benchmarks with moderate chunking sizes, whereas those trained on reinforcement learning (RL)-generated data often degrade as the chunk size increases. We investigate this intriguing human-RL gap in chunking effectiveness through the lens of activity sparsity. Our findings show that human demo datasets generally exhibit higher activity sparsity than RL-generated datasets across tasks. Building on this insight, we introduce sparsity regularization into the RL data-generation process and show that the resulting datasets can improve the effectiveness of action chunking policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
action chunking, robotics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_XGifOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates why action-chunking policies trained on human demonstrations generally tolerate longer action chunks better than those trained on RL-generated data. Through experiments across several robotic manipulation tasks, the authors identify activity sparsity as a key indicator of chunking effectiveness, measured using Mean Inactive Dimensions (MID) and Hoyer sparsity. They further introduce sparsity regularization into RLPD to generate sparser trajectories and evaluate their impact on downstream diffusion policies.

Critical strengths
  1. Interesting empirical observation. The identified human–RL gap in chunking effectiveness is relevant to robotic imitation learning. The authors demonstrate this phenomenon across multiple manipulation tasks and different RL-generated datasets.
  2. Systematic analysis of data characteristics. Rather than relying solely on qualitative observations, the paper compares several structural properties and evaluates activity sparsity using two complementary metrics. The subsequent regularization experiments provide an additional attempt to validate the findings.
Critical weaknesses
  1. Limited evidence for the underlying mechanism. The paper primarily establishes correlations between activity sparsity and chunking effectiveness without explaining why sparse actions should benefit longer action chunks.
  2. Limited effectiveness of sparsity regularization. The intervention results are mixed. While L1 regularization improves performance on Can, Hoyer-inspired regularization on Square only improves relative chunking effectiveness, with the absolute success rate at (16, 8) decreasing from 42.3% to 39.3%. L1 regularization on Square significantly degrades performance despite increasing sparsity. These results weaken the practical significance of the proposed intervention.
  3. Confounding factors in the chunking evaluation. The experiments increase both the prediction horizon and execution horizon simultaneously. Consequently, it is unclear whether the observed performance degradation primarily arises from increased action-sequence modeling difficulty or reduced feedback frequency during execution.
  4. Limited generalization and experimental validation. The experiments rely on a single diffusion-policy architecture, primarily with low-dimensional observations. The absence of visual-policy and real-world robotic experiments limits the generality of the conclusions.
Questions for authors

See Weaknesses.

Reviewer_Fu7kOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper finds a phenomenon in behavioral cloning: with different data sources, different action chunk sizes perform best. It tries to study why, and uses two activity-sparsity metrics (MID and Hoyer sparsity). It then adds sparsity regularizers to RL data generation, which in some settings moderately reduces the drop in behavioral cloning's success rate at a large chunk size.

Critical strengths
  1. It is an interesting question. In my experience, the good action chunk size differs across policies, tasks, and data sources, so it is good to see someone study this.
  2. It attempts to give an explanation by analyzing activity sparsity, and compares it against four other structural properties of the data.
Critical weaknesses
  1. Scope: there could be many confounding factors, and it is unclear where this phenomenon exists. Is it only in diffusion policy, or does it also hold for ACT or other policies? Does it hold for vision-based policies instead of state-based ones? The action space likely matters a lot, but the paper only uses delta actions. What about actions relative to the chunk start, or absolute actions? What about different normalizations, and is there any normalization at all? All of this affects how general the finding is.
  2. The paper does not clearly explain why. It is unclear why the two metrics (MID and Hoyer) would make the optimal chunk size different. Therefore, the proposed regularization is also not clearly useful, and the results are mixed (e.g., L1 on Square makes the drop worse, Table 4).
Questions for authors
  1. Does the human–RL gap hold for other policy classes, such as ACT, and for vision-based policies?
  2. Does it hold for other action spaces, such as actions relative to the chunk start or absolute actions? What action normalization does the policy use, if any?
  3. What is the intuition for why activity sparsity changes the best chunk size?
Reviewer_ATXjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this work, the authors present a work about action chunking effectiveness. The primary thesis is that action chunking is differently effective based on whether the data comes from RL or human demonstrations. In particular, the authors claim that RL creates action chunks that are more activity rich than humans, which is why they require shorter chunk lengths. With this insight, they develop activity-sparsity regularizer, which, when applied, reduces the drop on policies trained on RL data when a longer action chunking is applied.

Critical strengths

it is an interesting insight to have identified that human behaviors and RL controller behave differently, and the different behaviors impact standard behavior cloning knowledge when applied to different data sources. The metrics invoked to understand activity sparsity is also interesting, although it looks mostly about sparsity over dimension and less about sparsity over time. Finally, the application of sparsity regularization over RL training algorithms also seem cool in theory.

Critical weaknesses

The paper seems on the level of something written by an AI autoresearch agent over a weekend rather than a serious research work worthy of being published at ICLR. Here are all the weaknesses that I noticed with it.

  1. Very simple sim environments contain all experiments, and the tasks on the environments are not quite diverse either.
  2. The observation that "longer action chunks hurt performance" is correct, but the work only explores one possible hypothesis on why.
  3. Sparsity over action dimensions is probably real, but it is also probably an artifact of the teleoperation interface rather than human behavior, which is not considered or discussed at all. The authors also don't make distinction of the fact that they are both planning for and executing longer horizons which are two related factors.
  4. RLPD, given that it contains human prior data, cannot be considered a purely RL-style algorithm.
  5. Robomimic-style machine generated dataset often contain planner-generated behavior, which is not considered in this work at all.
  6. Most damningly, the RLPD experiments show very mixed successes in their chosen metrics, and almost always hurt performance, which makes further comparisons moot. It is only useful if the regularization regularizes, i.e. finds another equally valid solution with some desirable properties.
Questions for authors
  1. What other concepts beyond activity sparsity could explain this effect? What other hypotheses did you explore?
  2. How may RL vs. a planner be different in this consideration?
  3. How might different teleop interfaces affect the results that you are noticing, i.e. where humans control a humanoid with a learned controller/RL based retargeting algorithm?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.