- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
This paper proposes MaskLAM, a simple modification to latent action model training that aims to reduce the influence of visual distractors. Instead of reconstructing the entire next observation, the method uses agent masks obtained from pretrained segmentation models to restrict the reconstruction loss to agent pixels, while keeping the full observation available as input. The authors compare loss masking with input masking and evaluate the approach on multiple datasets. The results show improvements in latent action prediction and downstream control performance over several baselines.
Critical strengths
The paper addresses a relevant problem in latent action learning and proposes a simple modification that requires no changes to the underlying LAM architecture. The results consistently show improvements over label-free baselines, and the comparison between loss masking and input masking provides some practical insights into the importance of preserving contextual information.
Critical weaknesses
In its current form, I find the contribution too limited for acceptance, while the writing is clear and the method works empirically well.
-
Limited novelty and heavy reliance on external spatial priors : The core idea is to use a pretrained segmentation model to identify the agent and apply the reconstruction loss only to those pixels. While effective, this seems like a fairly straightforward use of existing segmentation and masked reconstruction techniques. The method does not discover which factors are endogenous or exogenous, but instead relies on an external segmentation model and an explicit prompt identifying the agent. In this sense, the method shifts the supervision requirement from action labels to spatial priors.
-
The theoretical framing and claimed contribution seem stronger than what is actually shown : The paper describes loss masking as a "training-time intervention" based on the Ex-BMDP factorization, but the actual operation is simply restricting the reconstruction loss to selected pixels. This does not identify the endogenous/exogenous factorization itself. Proposition 1 also seems rather straightforward, which tells Bayes-optimal predictor with additional context cannot perform worse than one without it. It does not establish that the proposed unsupervised training recovers better actions or that loss masking is generally superior to input masking.
Questions for authors
-
Since MaskLAM relies on a pretrained segmentation model and prompts identifying the agent, how should we compare its supervision and computational requirements with methods that use action labels? Could the authors provide a more complete comparison of these costs?
-
Given that Proposition 1 does not establish recovery of action-relevant latent variables, what evidence supports interpreting MaskLAM as separating endogenous and exogenous dynamics, rather than simply improving action predictability through a more focused reconstruction objective? Could the authors provide a more direct analysis of whether exogenous information is actually removed from the learned latent actions?
4 comments
Response to Reviewer x5d9 (2/2)
W2/Q2: Theoretical framing and evidence (continued).
"or that loss masking is generally superior to input masking"
Section 6.1 reports that the two are statistically indistinguishable on DCS and DMW. We chose them because they are the established distractor benchmarks, and during experimentation we discovered that most of the change in both is driven by the agents themselves. This motivated the Atari experiments, where the environment moves the agent and the two variants differ (Table 21). The linear analysis now proves this case (Appendix V). A model with masked input sees only how the agent moved, so its latent action encodes the sum of the action and the motion caused by the context. A model with masked loss sees the context and can subtract its effect. For a self-actuated agent the two agree.
"what evidence supports interpreting MaskLAM as separating endogenous and exogenous dynamics, rather than simply improving action predictability through a more focused reconstruction objective? Could the authors provide a more direct analysis ...?"
It uses endogenous context. The mask covers only the agent, but the inverse model attends to the ground under the cheetah and to the floes and logs that carry the agent in Atari (Fig. 5). Hiding these regions from the model, as input masking does, lowers NR from 0.94 to 0.33 (Table 21).
It discards exogenous information. Your request for a more direct analysis led to a useful new experiment for the revision. Our datasets render every simulator state twice, with and without the video background. We infer the latent action for the same transition before and after swapping in the video background of another episode and measure how far it shifts, on a scale S where 0 means unchanged and 1 means as different as two unrelated transitions (Table C in R2).
- Replacing the background moves LAPO's latent action most of the way to an unrelated transition (S = 0.85). MaskLAM's moves by S = 0.03 with simulator masks and S = 0.13 with SAM masks.
- LAOM-Labels, trained with 128k action labels, is at S = 0.69 on average, although the background's motion is barely decodable from its latent action (last column of Table C in R2). Action labels make the action decodable, but on average they do not make the latent action invariant to the exogenous change.
MaskLAM is therefore more than a focused reconstruction objective. Its latent action depends on the endogenous part of the scene and is nearly independent of the exogenous part. The test covers distractors outside the mask, and the hard DCS setting covers changes on the agent itself (Table 17).
Q1: Supervision and compute cost.
"how should we compare its supervision and computational requirements with methods that use action labels? Could the authors provide a more complete comparison of these costs?"
We think the fair way to compare them is by what it costs to scale each kind of supervision to more data. For robots, action labels come from teleoperation, so every labeled second needs a human operator and a robot. This is why robot datasets are small and expensive. DROID took 50 data collectors 12 months for 350 hours of data [21]. This cost grows with the data, has to be paid again for every new embodiment, and has to be paid at recording time. If the ground-truth actions were not recorded, they cannot be recovered afterwards at any cost, which is the case for most existing video, for example of humans or animals. For humans, pseudo-actions could be obtained with pose estimation, but it is unclear how good a substitute they are for ground-truth actions, and running a pose estimator on every frame is likely as expensive as running a segmentation model. The supervision of MaskLAM does not grow linearly with the data. It is one coarse box per task on DCS and DMW, and a single text prompt for all of DROID. What MaskLAM needs instead is compute for the masks, which we list in Table 3. So MaskLAM trades operator and robot time for GPU time, which can be paid afterwards for any video. We agree that the segmentation model and the prompt are still a form of supervision, but a much cheaper and more scalable one than teleoperating a robot, especially for utilizing already available, unlabelled video data.
Presentation.
Your review describes the writing as clear, while the presentation score is 1. If specific parts were hard to follow, we would be grateful for pointers and will revise them.
We hope our answers address your concerns. If they do, we would be grateful if you would consider raising your score. We are happy to run any further experiments during the rebuttal discussion if time allows.
I appreciate the authors' thoughtful responses and the effort they have put into addressing my concerns. Most of my questions have been adequately resolved. I also acknowledge that the presentation score was a mistake on my part, and I have therefore raised it from 1 to 2.
However, regardless of the effort demonstrated during the rebuttal or the number of additional experiments conducted, I respectfully maintain that the core idea does not offer a sufficiently strong contribution to warrant acceptance at ICLR. Therefore, I will maintain my original overall score.
Thank you for reading our response and for correcting the presentation score. We are glad that most of your questions are resolved. Your comments led to the linear analysis (Appendix V) and the background-swap test, and both improved the paper.
We respect that we weigh the contribution differently, and we do not want to argue the point again. We only want to say briefly where we see it. Masked reconstruction targets were known in model-based RL with action and reward labels [22], and object-centric latent action learning [23] masks the input together with the target and needs action labels to select the agent slot. What had not been shown is that masking the loss alone, with the full frame as input and zero-shot masks, is enough. It matches 128k action labels in LAOM’s own training setup, and it keeps the lowest probe error under the color and camera changes for which [32] argued that segmentation would not suffice (Table 17). In a linear model we prove that the masked loss recovers the action for any distractor strength, that weight decay favors a latent action free of exogenous content, and that loss masking differs from input masking when the environment moves the agent (Appendix V). The background-swap test and the Atari experiments confirm this for the trained models.
That a single change to the loss achieves this, where other approaches add action labels [31], a flow decoder [7], slot models [23] or a vision-language model [32], is to us the result of the paper and not a shortcoming. The mask also states whose action is to be learned, which action-free video alone leaves open when several things move.
Response to Reviewer x5d9 (1/2)
We thank Reviewer x5d9 for recognizing the relevance of the problem, the fact that the method needs no architectural change, and the practical insight from comparing loss masking with input masking. The review raises three concerns, summarized here and answered below.
Tables A to E are in the comments R1 and R2 under the general comment.
W1: Novelty and reliance on spatial priors.
"a fairly straightforward use of existing segmentation and masked reconstruction techniques"
We discuss Segmentation Dreamer [22] and related masked-reconstruction methods in Section 2. As noted there, they train on action- and reward-labeled transitions, whereas MaskLAM brings target masking to action-free video, where the reconstruction target is the only signal that defines the latent action. For this setting, LAOM [31] concluded that supervision is required, and its follow-up [32] argues that "segmentation, while simple, is not enough". Prior work counters distractors with additional supervision or model components, such as action labels [31], an optical-flow decoder [7], a slot model [23], language conditioning [6], or a vision-language model [32]. We show that none of this is needed for a visible agent. With SAM masks and no action labels, MaskLAM's latents are more action-predictive than those of LAOM-Labels with 128k labels on DMW and on the hard DCS setting (Tables 1 and 17). The linear analysis backs this with theory (W2/Q2 below).
"The method does not discover which factors are endogenous or exogenous, but instead relies on an external segmentation model and an explicit prompt" / "the method shifts the supervision requirement from action labels to spatial priors"
This is the thesis of the paper. The conclusion states that the supervision can shift from action labels, which are scarce, to segmentation masks, which are cheap and available zero-shot. What the model learns beyond the mask is in W2/Q2 below.
W2/Q2: Theoretical framing and evidence.
"describes loss masking as a 'training-time intervention' based on the Ex-BMDP factorization ... This does not identify the endogenous/exogenous factorization itself."
We use the Ex-BMDP framework to describe the problem. The agent mask supplies one part of the factorization, the agent itself, and it is the only prior. Which context outside the mask matters is learned from data, which we see empirically in the results below (Fig. 5, Table 21, Table C in R2). Contribution 1 will name the mask as a spatial prior.
"Proposition 1 also seems rather straightforward ... It does not establish that the proposed unsupervised training recovers better actions"
As the paragraph after Proposition 1 states, it concerns the information available to the inverse model and does not guarantee what training recovers. Your comment motivated us to extend the linear analyses of [49, 25] from the full-frame objective to a masked objective. The analysis, which is Appendix V of the uploaded revision, shows the following under spatial separability, with no assumption on the strength of the distractor.
A simulation in the style of [49] agrees with these results. The analysis is linear and assumes a fixed mask, and it identifies the action up to a function of the current frame (as in any linear LAM). Whether the conclusions carry over to nonlinear models is what our experiments already tested.