- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies action-coordinate representations for precise positioning from offline visual trajectories. It constrains latent transitions to be linear in the action and anchors the target at the origin, allowing closed-form control instead of online planning search. The paper develops existence, representability, and conditional optimality results, and evaluates the approach on several positioning tasks, including physical systems.
Critical strengths
- The representation is closely tied to the control objective. The connection between structural assumptions, observation history, and the resulting controller gives the method a clear conceptual basis.
- The evaluation spans different positioning mechanisms, and Appendix F examines important assumptions and design choices.
Critical weaknesses
- Some headline comparisons do not isolate the action-coordinate constraint. In Table 2, the image-based Interferobot configuration uses a sharpness encoder and 800 training epochs for AC-WM, versus an autoencoder and 200 epochs for the listed world-model baselines; training duration also differs on LensAlignment. These comparisons establish performance of the configured systems, but do not attribute the entire gap to the proposed representation.
- Reward-based anchoring is an important data requirement. Page 30 specifies using the 50 frames closest to the target on random data, and the supplied implementation ranks stored objective or metric values. The relationship between these quantities and the rewards available to each baseline should be explicit for each environment. The limitations when the best available observations remain far from the target also matter.
Questions for authors
- Which main comparisons match the encoder, observation history, and training budget simultaneously? Please identify any existing results that separate these factors; a focused comparison on one representative environment would be sufficient if additional evidence is needed.
- Which quantities determine anchor ranking, reward regression, and target labels in each environment, and how do they relate to the rewards given to the baselines? Please connect the existing anchor-error analysis to the random-data setting.
4 comments
Q2
We agree that the manuscript should state this more explicitly, and the revision adds the table below to Appendix E. Every environment returns the reward as given in Appendix E.
| Environment | Reward observed (all methods) | AC-WM target labels, anchored data | AC-WM target labels, random data |
|---|---|---|---|
| BeamAlignment | frames with , i.e. weighted beam error below m | the frames of highest | |
| SpheroEnv (sim and real) | frames whose carries the success bonus, i.e. px | as on anchored data | |
| Interferobot | the frame at each episode is collected from | the frames of highest | |
| LensAlignment | the frame at each episode is collected from | the frames of highest | |
| LightTunnel | the frame at each episode is collected from | not run |
All methods, offline or online, model-based or model-free, receive this and then use it according to their design. No method receives other ground-truth quantities or evaluation metrics that depend on them (e.g. the beam error in mm or the ball distance in px). Specifically, our anchor ranking reads only and no other ground-truth information. On random data, the frames closest to the target was a wrong formulation, in fact we select the frames of highest reward in the available dataset, which is not necessarily the same. We apologize if the framing "closest to the target" was creating the impression that ground truth information was used. We have corrected this in the revised manuscript accordingly.
The same holds for the online domain. As AC-WM is designed to be an offline method, to make it working here as well, we mark the frames of its buffer within the goal tolerance and at least the of highest reward (except on SpheroEnv, see table) for the anchor mechanism. TD-MPC2 observes the same reward at every transition. The two methods differ only in how they use it: TD-MPC2 regresses it through a reward head and a discounted value, while AC-WM additionally uses its ranking to fix the origin of the latent chart. As the controller improves, the highest-reward frames move closer to the target, so the anchor sharpens with the data.
That said, we extended our anchor-error analysis to more random-data settings, specifically to
SpheroEnv to BeamAlignment and Interferobot (both using vector and image observations). The
results are in Figure 9 and Figure 32 (Appendix F.4) of the revised manuscript.
Thanks for adding the matched comparisons. That clears up my first concern. In the online experiments, do you use the shared reward alone to decide whether a frame is within the goal tolerance?
Yes, all models receice the same information: the observations, the actions and the reward . Whether a frame is within the goal tolerance is decided from this reward alone, by the threshold . The threshold is also set from the rewards in the replay buffer. No additional information enters the anchor loss term.
We thank the reviewer for the constructive feedback. The comments pointed us to some places where the presentation of our paper could be improved.
Q1
We agree: Table 2 lists the best configuration we found per method, and these potentially differ in the observation history k, the encoder used, and the training epochs.
In the original submissions, the following comparisons already match the factors mentioned by the reviewer precisely:
LightTunnel(image), all methods share k=1, the spectral encoder, and 400 epochs (now Figure 22).LensAlignment(vector), they share k=1 and 400 epochs. Figure 13 shows AC-WM and RG-WM at each k∈{1,3,5} on same encoder and training budget.For a clearer overview, we provide the following two additional comparisons with the same k, trainings budget and encoder architectures and hope this will address the reviewers concern appropriately.
LensAlignment(image)All methods using the spectral encoder and 400 training epochs:
BeamAlignment(vector)All methods using 200 epochs of training budget
Note that k=4 padded repeats the first frame to match k=4, which makes no sense for AC-WM that relies on random actions during its probing phase.
Generally, our search followed one rule across environments: Where a vector version exists, we trained large grid of configurations there and select the best one per method. On images, the encoder adds a further axes to that grid and every run is considerably more expensive, so a full search is out of reach. We therefore start each image environment from the best vector configuration and search only the factors specific to images, foremost the encoder and the window k. Moreover, by training encoders together with decoders in isolation, we made sure that the encoders are expressive enough to represent the relevant information in the images.
The reviewer's question prompted us to extend our hyperparameter search on the image-based environments to more encoders and more values of k where computationally feasible and reasonable. On two of them, we found better configurations for the world-model baselines (MPC-WM now better on
Interferobotand RG-WM better onBeamAlignment), which does not change the general picture. The revised manuscript now reports these results as well.