acceptodds
Under review as a conference paper at ICLR 2027

Learning Counterfactual Predictive Memory for Long-Horizon Aerial Exploration

Abstract

An exploration robot must remember not only where it intended to fly, but which observations would make that intention obsolete. Recent aerial planners exploit geometric passability and cached global tours, yet neither a binary frontier label nor a visit order predicts the decision consequences of a new observation. We propose counterfactual predictive memory (CPM): an action-conditioned representation of how a short sensing maneuver changes newly observed volume, frontier connectivity, and continuation cost. Paired simulator branches share the same observed history and differ in either the unobserved environment or the executed maneuver. A single learned transition law evaluates immediate motion and observation-conditioned continuation; its value discrepancy also determines which historical details can be compressed. We derive a conditional action-regret bound and evaluate the method through aerial comparisons, component ablations, and planar learning controls. In the forest comparisons, CPM reaches 94% coverage in 251.6 s with LiDAR and 507.4 s with limited-FoV depth sensing, the lowest completion times within the respective comparison groups. Component ablations reveal complementary contributions to exploration efficiency, with tradeoffs in prediction error and planning latency.

Primary area:
applications to robotics, autonomy, planning
Keywords:
representation learning, model-based reinforcement learning, predictive memory, active exploration, aerial robotics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fwdFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a learnable algorithm for aerial exploration. Their method is based on creating a compressed map representation in order to reduce computational requirements without sacrificing exploration performance. This compressed representation is built in a way that makes it similar to a map built purely from raw observations according to a learnable value function. In other words, it retains only the information that is most relevant to future actions. The proposed method is evaluated quantitively on a simulated benchmark, where its performance seems somewhat comparable to other works in terms of planning time and latency. Real world experiments are also provided for qualitative purposes.

Though there are some potentially valuables ideas, the paper seems very poorly written. What I could piece together was only after undue effort to decode and reverse engineering its content based on much better written previous work. This was the case for essentially all parts of the submission, including figures and tables (e.g., Table I and Figure 4 should have the same data, but are clearly distinct).

I believe that the paper would require extensive revision before publication, so I do not recommend acceptance at this stage.

Critical strengths

The paper brings some interesting ideas and the resulting algorithm is comparable to related work. In particular, they consider an objective that apparently tackles a potential issue in entropy based objectives: the fact that reducing map entropy may not lead to changes in future actions, so it may not be an ideal objective. This seem well motivated and effective for the problem

The submission also includes some some real world experiments, although these are only qualitative and for demonstration purposes. Depending on which reported data is corrected (Table I or Figure 4), one could also consider the performance of the method to be comparable to similar work.

Critical weaknesses

The main issue with the paper is that it is particularly difficult to read and understand, even after multiple passes at the submission and related work. This is not because the algorithms are inherently complex, but because throughout the paper, among other things, there are very strange word choices and sentences are constructed in unnatural ways. The most closely related paper according to the authors, PACE, has no such issues and could be understood with a small fraction of the effort spent in this submission.

To illustrate, consider this sentence from line 228: "We charge shadow construction and retention to compute and memory totals". This use of the word "charge" is perhaps natural on financial contexts, (e.g., a customer was charged for a product), but is not typically used in the technical context of the paper, even though in principle compute and memory requirements also incur a sort of cost. Perhaps this is why the authors used the term "compression audits", with "audit" being a similar strange word choice in the context, but not in finance.

The previous example might not be so significant in isolation, but similar issues occurred in many other parts of the writeup and I ended up compiling a sort of decoding table to approximately map what was written to what a typical reader would be able to understand if they were familiar with the foundational literature (such as the works of Kaelbling and Sutton, both prominent authors cited by the submission) and based on other sections of the paper. The "outcome law" is the "state-transition function"; "entropy bonus" may refer to "maximizing entropy reduction" as an objective, which was mentioned earlier in the text; "continuation" might be used for the "cost-to-go" or "value function" or for "rollouts" depending on the context; "observation-contingent" might be a different wording for "observation-conditioned"; "limited-FoV" is the expression for experiments using a depth camera (PACE used "Depth Cam" for example).

Structurally, the writeup might benefit from adding more conjunctions, otherwise it becomes harder to understand how one idea evolves to the next as the text starts to resembles a collection of disjoint points rather than a cohesive narrative. That would also shift the focus from describing "what was done" to explaining the "why it was done", which would further improve clarity. As another example, consider that both this submission and PACE use RMS jerk as a metric, but PACE makes sure to explain, even if briefly, that it is used to quantify the smoothness of the trajectory. A reader familiar with trajectory planning probably already knows what jerk is, but being explicit about it facilitates reading and also makes the paper more accessible to a broader audience.

As a last point about readability, figure 1 also seems constructed in a strange and confusing way. Each scene inset seems to have a slightly different scene layout in previously observed regions and it's not clear if this is an artifact or a result of the periodic scene recompression (or "compression audits" as they may be called). The description mentions a blue and an orange feedback, but I can only find a green or perhaps teal one and it's not clear what it is connected to. It is also confusing that the shadow model has the same number of edges and nodes as the compressed, when, to my understanding from the text, the shadow one should be larger. Perhaps that is the state of the compressed model before compression?

To highlight a second more technical weakness, it would seem that the main motivation of the paper is summarized in this sentence from line 87: "The central falsifiable claim is that this representation preserves decision-relevant history at lower memory cost than geometry reconstruction or a cached visit order". Yet, the authors seemingly never report the algorithm's memory footprint, only the planning latency. Upon reading PACE, it became clear that the problem relates the Traveling Salesman Problem (TSP), so my impression is that the memory-footprint is not actually the main issue, but rather the fact that large graphs lead to expensive solutions, since the TSP is NP-hard. This might explain why memory footprint was never reported, but ultimately it's the author's job to clearly motivate their work and experiments.

Questions for authors

A question important to my assessment: why is the data from Figure 4 seemingly different from Table I? For example, CPM appears to have the best time in table I, but is surpassed by many methods according to Figure 4? The caption says the data is the same

General questions and comments (not critical for the assessment)

  • Does the replanning latency affect the exploration time? In other words, is the drone effectively waiting for a decision during that time or still flying, so the latency cost gets absorbed into the flying time?
  • Related to the previous question, I think it would be interesting to include CPU/GPU utilization as another metric.
  • It is generally not recommended to use 3D bar graphs when 2D ones can convey the same information. The extra dimension adds visual noise and makes it harder to read.
Reviewer_MBcEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Counterfactual Predictive Memory (CPM) for UAV exploration in unknown environments. The key idea is to make the memory remember what each candidate move would lead to, such as how much new space it reveals and what it costs. The learned model guides both selection and memory compression by preserving action values. The paper also provides a conditional regret bound for a single decision. CPM achieves the shortest completion times in both forest sensor tracks.

Critical strengths
  1. The paper is well motivated. Asking whether a new observation would change the next decision is closer to what exploration actually needs than asking whether a frontier is passable.
  2. It does not treat uncertainty reduction as useful by default. If no possible observation would change the best follow-up action, the value of that observation is lower. This helps avoid useless actions.
  3. The experiments are solid. The authors also provide real-world demonstrations, making the results reasonably convincing.
Critical weaknesses
  1. The gains over the fastest baselines are relatively small, at about 3.2% for LiDAR and 2.3% for depth sensing.
  2. The memory efficiency claim could use more direct support. The retention ablation shows a benefit at a fixed token budget, but the paper does not show how performance changes with the memory budget or compare memory costs against geometry reconstruction and cached visit orders.
  3. The evaluation could be more complete. Although the benchmark includes forest, maze, and indoor layouts, the main quantitative results cover only forest scenes. The planar results are also mixed. DARE achieves a shorter path and higher coverage, while the other three methods have lower planning latency than CPM.
  4. Since both memories use the same prediction model and value function, their value estimates may agree even when both are inaccurate. The paper acknowledges this limitation, but its practical impact remains unclear.
Questions for authors

See the Weakness part.

Reviewer_qCgLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies autonomous exploration by quadrotor UAVs in unknown, cluttered environments and proposes Counterfactual Predictive Memory (CPM). Given an observed history, CPM constructs simulator branches that vary in hidden-world completion and candidate sensing maneuver, and learns a shared action-conditioned model to predict newly observed volume, frontier-topology events, motion costs, and observation-contingent continuation value. The predicted values are also used to guide memory compression: a compressed representation is evaluated against a shadow representation on the same feasible candidate actions, and discrepancies inform memory retention or refresh. The experimental results demonstrate the effectiveness of the proposed method.

Critical strengths
  • Paired simulator branches provide supervision for unexecuted sensing maneuvers, enabling the model to learn their potential exploration outcomes.
  • Memory retention is guided by changes in candidate-action values, directly connecting compression to downstream exploration decisions.
  • The indoor flight demonstrations show that CPM can be integrated into a UAV exploration system and executed in a closed loop, complementing the quantitative simulation results.
Critical weaknesses
  • The paper state that hidden-world completions must reproduce the observed sensor prefix, and specify the number of completions and maneuvers sampled. However, the procedure for generating these completions, the sensor-consistency tolerance, and the distribution of unobserved geometry are not specified sufficiently for independent reproduction. These choices determine the counterfactual training targets and may affect transfer to environments outside the simulator distribution.
  • Given the paper’s claim that CPM preserves useful history at lower memory cost, results across several token budgets, together with peak total memory and shadow-audit costs, are needed.
  • The paper states that its benchmark contains forest, maze, and indoor layouts, whereas the main aerial comparison in Table 1 reports only forest results. This limits the evidence for performance across different forms of clutter and topology.
  • The paper reports no quantitative completion, coverage, distance, or runtime measurements for these flights and includes no real-world baseline.
Questions for authors
  • How are hidden-world completions generated, and how is their consistency with the observed sensor history verified?
  • What is the total system memory cost, including the map, observation log, and shadow-memory audits, across different token budgets?
  • Can you report quantitative results for the stated maze and indoor layouts, together with variation across runs?
  • What quantitative outcomes were recorded in the four real-world flights, and is a comparison under the same flight conditions available?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.