acceptodds
Under review as a conference paper at ICLR 2027

Learning When to Call: Predictive Opportunity Ranking for Embodied Agents

Abstract

Embodied agents must maintain continuous control while selectively invoking expensive reasoning backends. With a limited call budget, poorly timed requests may miss useful pre-event opportunities. We focus on event-relative opportunity prediction: using causal observations to anticipate pre-onset windows in which to place reasoning requests. The Predictive Reasoning Scheduler (PRS) implements a lightweight scorer from recent state history and current bird's-eye-view context; a separate shared selector enforces budget and cooldown constraints. We evaluate offline exact-call placement on frozen trajectories using Useful-Window Coverage (UWC), the fraction of distinct event windows reached by at least one request. On CARLA Town10HD, a development-selected checkpoint covers 23 of 63 windows with 165 calls, compared with approximately 6 for Random-. The UWC@ gain is 26.71 percentage points (95% bootstrap interval: 12.64–40.97). On fresh Town10HD scenarios, a model selected using a prespecified development procedure achieves a normalized log-budget UWC-AUC gain of 0.2392 over Random-. Controlled retraining also finds that current-state inputs and broader supervision can outperform the reference design. These findings support event-relative opportunity prediction as a learning formulation for improving the temporal placement of budgeted reasoning calls.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Agents, Event Anticipation, Budgeted Reasoning Invocation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_TD59OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies when an embodied agent performing continuous control should call an expensive reasoning backend such as a VLM when it can afford only a limited number of calls. The authors formulate this as event-relative opportunity prediction: from causal observations, a scorer predicts whether each time step falls within a short window before an event onset, 0.5 s by default, and a separate shared selector chooses call times under budget and cooldown constraints. The proposed Predictive Reasoning Scheduler, or PRS, encodes the last 20 steps of state history with a GRU and the current bird's-eye view with a small CNN, and uses the summed probability of the time-to-event head as the opportunity score. Evaluation is performed offline on frozen CARLA trajectories using Useful-Window Coverage, or UWC, which is the fraction of distinct event windows reached by at least one call. The paper reports that PRS covers more windows than Random-k with the same number of calls.

Critical strengths

S1. The paper clearly formulates a practical problem. For agents that combine continuous control with an expensive reasoning backend, deciding when to call, rather than how often, is an important problem. Separating the learning of opportunity scores from allocation under budget and cooldown is a clean design that allows predictive quality alone to be compared under the same allocation rule.

S2. The metric and comparison conditions are designed fairly. UWC counts multiple calls on the same event only once, so a method cannot raise its score by clustering calls around easy events. All methods are also given the same candidate times, the same number of calls, and the same cooldown, and a known-window oracle serves as an upper bound, which makes the comparison fair.

S3. Negative results are reported transparently. The authors openly report the performance loss under online allocation and the controlled experiments showing that simpler alternatives outperform the proposed reference design.

Critical weaknesses

W1. The definition of the useful window ignores the backend's response time, and there is no evidence that the calls actually help. The target window is the 0.5 s before onset, but backends such as VLMs usually take longer than this to respond, and the authors themselves report that "longer lead times enabled timely returns." In other words, UWC counts as useful the call times at which a response cannot arrive in time, and the authors acknowledge that UWC does not measure the value of the returned answer. The results therefore show better placement under a surrogate target that ignores latency, but not that PRS makes reasoning calls more useful. Evaluation with windows shifted earlier by the latency, as well as results showing actual improvements in driving, are needed.

W2. The main results rely on an offline setting that uses hindsight, and the paper does not show that the gains survive under causal allocation. The offline selector compares the scores at all time steps of a completed trajectory before choosing calls, which is not possible in deployment. Under the evaluated causal allocator, coverage retention in the primary evaluation domains, Town10HD and Town07, drops to 0–16%, partly because the allocator commits far fewer calls than the budget, placing only 32 of the 165 budgeted calls on Town10HD. The paper therefore does not yet show a deployable allocation that preserves the offline gains. In addition, each of the 63 trajectories in the primary Town10HD cohort contains exactly one event, which makes the task closer to locating a single event in a completed drive than to scheduling calls during ordinary driving. There is also no comparison with stronger baselines such as a lead-time-adjusted risk trigger or accident anticipation models.

W3. The methodological novelty is limited, and missing details limit faithful reproduction. PRS combines a small GRU and CNN classifier with greedy top-K selection. In the authors' own controlled comparisons, broad 2 s supervision significantly outperforms the proposed objective and a model using only the current state performs better on average, which leaves the history-based design without clear support. In addition, the identity of the 17-dimensional input features, the simulation tick interval, the onset definition for each event type, the definitions of the time-to-event bins and auxiliary labels, and the causal allocation algorithm are not described, which limits faithful reproduction of the method and results.

Questions for authors

Q1. Does the advantage of PRS persist when it is evaluated with windows shifted earlier than the onset by the backend's response time?

Q2. Why does the performance of the causal allocator drop so sharply on Town10HD and Town07, and why does it commit far fewer calls than the budget? Can PRS outperform Random-k and a lead-time-adjusted risk trigger under online allocation and on long trajectories with no events or multiple events?

Q3. What are the state features, the bird's-eye-view channels, the tick interval, the event onset definitions, the time-to-event bins and auxiliary labels, and the causal allocator? If a current-state-only model and broad supervision perform better, why keep the history-based GRU as the reference design?

2 comments

Thank you for your careful review and helpful comments. We address each of your questions in turn.

Q1. Coverage under latency-shifted windows

The shifted-window analysis identifies the response lead times under which predictive scores retain a placement advantage. Section 5, Table 2, and Appendix F.2 keep calls and window width fixed while shifting windows earlier, using onset as a reference deadline. With 165 calls per method and a 0.30 s shift, PRS covers 22/63 events, versus a mean of 6.35/63 for Random-k and 8/63 for lead-time TTC. The paired gains have 95% intervals of [3.09, 29.3] and [3.44, 26.3] events, respectively. At the original driving pipeline's 1.33 s p95, Table 2 instead shows higher coverage for TTC. The advantage therefore depends on the required lead time.

New Figure 1 and Section 3.3 connect these comparisons to reasoning assistance by distinguishing event onset, evidence availability, the decision deadline, and response latency. Feasible calls require relevant evidence and enough time for a response to arrive by the deadline. UWC evaluates request placement, while these additional conditions specify when a request could support a decision.

Q2. Online allocation, budget use, and event distributions

PRS produces scores from current and past observations, while the allocator handles budget and spacing. Revised Section 6.2 clarifies that the evaluated sequential allocator knows trajectory duration but does not read future scores. On Town10HD, per-trajectory flooring reduces the nominal allowance of 165 calls to a ceiling of 126, and 32 calls are realized. Flooring explains the change in the ceiling, while the remaining 94 unused calls concern sequential commitment.

Appendix D.2 controls for call quantity by comparing offline and sequential placement at matched realized counts. Their UWC values are 0.0159 and 0 on Town10HD, and 0.2846 and 0.0488 on Town07. These comparisons separate budget utilization from placement, showing that call quantity alone does not explain the online coverage difference.

Sections 3.1 and 3.2 allocate a shared budget per trajectory, so multiple events compete for the same calls and spacing capacity. The budget does not reset for each event. Section 5, Table 3, and Appendix F.3 add offline comparisons on Town05 and Town06 with more dispersed onsets, where PRS exceeds elapsed-time and route-progress rules. These results complement the single-event cohort. The requested online comparison against Random-k and lead-time TTC on long trajectories with no events or multiple events is not included in the current evaluation.

Q3. Method definitions and the reference design

State features describe vehicle motion, control, and surrounding risks, while BEV provides spatial context. Section 4.1 explains their use. Appendix A.2 lists all 17 state inputs in stored order, their provenance, and the packed semantic BEV representation of size 3 x 64 x 64. It specifies conversion to floating point and division by 255, and distinguishes the training-manifest clock from acquisition intervals and evaluation cadence.

Sections 4.1 and 4.2 define the call score as the sum of the first two TTE probabilities, jointly covering the 0.5 s before onset. Phase, horizon, and urgency heads provide auxiliary supervision without entering the score. Appendix A.1 explains recorded event assignments, zero window labels for off-event examples, and event-wise coverage. Appendix A.2 distinguishes evaluation timing categories from training-head classes. These descriptions specify the deployed scoring interval, recorded-label usage, and evaluation conventions.

Section 4.3 and Appendix A.5 specify the shared offline selection rule. Section 6.2 and Appendix D.2 describe sequential commitment and its information conditions.

We retain the GRU and BEV encoder as a lightweight reference implementation for studying opportunity prediction under shared calling constraints. Keeping the checkpoint fixed also lets the latency analysis examine existing calls without retraining effects. Section 6.1 and Appendix C report that current-state inputs and broader supervision can match or exceed the reference results. Current observations can support future-window prediction, and broader training targets can improve ranking within the same evaluation window. These findings identify useful design choices within the framework. The contribution is event-relative opportunity prediction and controlled temporal placement, with inputs, encoders, and supervision treated as implementation choices.

Thank you for the detailed response and the revisions. The new figure and section on timing make the problem much clearer, and the added details on the inputs, labels, and allocator address my reproducibility concern. The new towns with more spread-out events are also a welcome addition.

However, my main concern remains. The latency-shifted results show that PRS only wins at short lead times, and at the measured latency of your original pipeline, the TTC rule covers more windows. The closed-loop test also shows no driving benefit, so I still do not see evidence that better placement actually helps the agent. The online results are also weak. On Town10HD, the allocator uses only 32 of the 126 available calls without a clear reason, and on Town07 it falls well behind offline placement even with the same number of calls.

I appreciate that these results are reported openly, but since this gap goes to the core of the paper's motivation, I will keep my score.

Reviewer_zXRoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to place a limited number of reasoning requests before upcoming events. A compact model uses recent state observations and a current bird's-eye view to predict suitable request times. A shared selector ranks candidate request times by their scores across completed trajectories, subject to per-trajectory quotas and minimum spacing between requests. Useful-Window Coverage measures the fraction of event windows containing at least one request. Experiments on recorded CARLA trajectories, including fresh scenarios, show improved offline placement over the evaluated baselines. Further studies examine the effects of input history, supervision, online allocation, and backend response timing.

Critical strengths

The fixed-budget evaluation allows a fair comparison of request placement methods. Methods share recorded trajectories, independently defined event windows, candidate times, request allowances, and minimum spacing between requests. The score-based methods also share the selection rule. These controls make the coverage differences interpretable. The fresh-scenario evaluation provides additional evidence beyond the development settings.

The controlled learning studies distinguish changes to a fixed model's inputs from retraining with different inputs and objectives. This distinction matters when assessing the value of history. The authors report strong performance from current-state inputs and broader supervision, along with limitations in online allocation and backend response timing. These results provide useful guidance about which aspects of the approach have empirical support.

Critical weaknesses

The practical meaning of the target window remains the main concern. A request inside the prescribed interval receives credit regardless of whether its answer arrives in time to help the controller. Appendix E reports backend response times longer than the primary window and limited success in obtaining relevant answers before onset. Appendix D also shows substantial loss of coverage when requests are committed online in the harder evaluation domains. The authors state the offline scope clearly. The significance of the result for reasoning assistance still depends on a stronger connection between window coverage and usable responses.

The advance over established event prediction and adaptive invocation needs stronger comparative support. Zhao et al. study future accident timing, and AdaDrive learns when to activate an LLM in a driving system. UWC is also closely related to the existence-based component of range recall studied by Tatbul et al., applied here to request times and pre-event windows. The controlled placement protocol appears to be the clearest distinction. The paper's own matched studies favor current-state inputs or broader supervision over important choices in the reference design. Including these stronger alternatives in the fresh-scenario evaluation under equal model-selection effort would clarify what the proposed formulation adds.

Questions for authors

The relationship between event onset, backend response time, and the controller's decision deadline needs clarification. Evidence that requests committed online lead to useful responses or better decisions under matched realized call counts would materially affect the assessment. Further detail on the allocator's budget use and the reported closed-loop outcome would help explain the loss of coverage and practical benefit. The comparative assessment would benefit from fresh-scenario results for the current-state and broad-supervision predictors under the same model-selection procedure. The exact Town07 selection rule and the corresponding seed-level records are also needed to assess the reliability of the selected model. The method description should specify the complete state inputs, timing categories, auxiliary labels, event-onset and event-assignment rules, and online allocation policy. These details are needed to reproduce the comparisons and determine which observations were available before onset.

2 comments

Thank you for your constructive comments. We have summarized the issues you raised and respond to each in turn.

Q1. Event onset, response latency, and decision deadlines

UWC evaluates the placement of calls within predefined pre-onset windows. We have added Figure 1 and Section 3.3 to distinguish event onset as an annotation reference from the controller's decision deadline. A feasible call requires relevant evidence at invocation and enough time for the response to arrive by that deadline.

We also assess latency sensitivity by shifting evaluation windows earlier while keeping the call sets and window width fixed, using onset as a reference deadline. Section 5 and Appendix F.2 describe this analysis, with results summarized in Table 2. With 165 calls and a 0.30 s shift, PRS covers 22/63 events, compared with a mean of 6.35/63 for Random-k and 8/63 for lead-time TTC. This placement advantage is conditional on the required lead time, and Table 2 reports how coverage and method rankings change across the full sweep.

Q2. Online allocation, budget use, and decision outcomes

PRS supplies scores from current and past observations, while the allocator applies budget and spacing constraints. In Section 6.2, we distinguish Town10HD's nominal allowance of 165 calls, the 126-call ceiling after per-trajectory flooring, and 32 realized calls. Appendix D.2 compares offline and sequential allocation at matched realized counts. This accounting separates budget utilization from temporal placement without attributing the entire coverage difference to call quantity.

Appendix E also summarizes an equal-call content pilot with a content-usefulness AUC gain of 0.008887 over Current-only and a reported interval of [0, 0.024742]. This positive point estimate provides preliminary content-level evidence under a 30 s batch timeout. The pilot is separate from online decision evaluation. Section 7.1 and Appendix E summarize the closed-loop outcome, which has not established a driving benefit. The supported contribution concerns predictive call placement, with response content and control outcomes evaluated separately.

Q3. Fresh-scenario comparisons under matched model selection

Our contribution is event-relative opportunity prediction and placement evaluation under shared call constraints. Section 6.1 and Appendix C report controlled comparisons in which current-state inputs and broader supervision match or exceed the reference results. These findings identify effective design choices within the formulation.

The fresh-scenario evaluation in Section 6 and Appendix B.2 concerns the selected reference model. The requested fresh-scenario comparison of current-state and broad-supervision predictors under the same selection procedure is not included. We therefore distinguish the controlled learning comparisons from fresh-scenario confirmation and present the GRU as a reference implementation.

Q4. Town07 model selection and seed-level records

We have specified the selection order in Appendix A.3 and reported the results for all ten seeds in Table 5. For each seed, Town03 selects the checkpoint by 2% UWC, useful-call precision, narrow-window frame average precision, and earlier epoch, in that order. Town07 then ranks the ten checkpoints by normalized five-budget log-budget UWC-AUC, median route-cluster AUC, 2% UWC, 2% useful-call precision, and smaller seed ID. The first four Town07 criteria are maximized.

Selection precedes fresh Town10HD evaluation. The selected checkpoint is seed 1649629920 at epoch 24. Its fresh-scenario AUC is 0.4047, with a gain of 0.2392 over Random-k. Table 5 also lists the remaining candidates, allowing the selected result to be read alongside seed-level variation.

Q5. Method definitions and reproducibility

Section 4.1 describes how structured state history and current BEV form the opportunity score. Appendix A.2 lists all 17 state inputs in stored order, their provenance, the packed semantic BEV representation and preprocessing, and the training-manifest and acquisition-clock conventions.

Sections 4.1 and 4.2 define the score as the sum of the first two TTE probabilities, jointly covering the 0.5 s before onset. Phase, horizon, and urgency heads provide auxiliary supervision but do not enter the call score. Appendix A.1 explains how we use recorded event assignments, zero-window labels for off-event examples, and event-wise evaluation. Appendix A.2 distinguishes evaluation timing categories from the training-head classes.

Section 4.3 and Appendix A.5 specify offline score-ranked selection. Section 6.2 and Appendix D.2 describe sequential commitment using current and past information and remaining budget, explicitly stating that the evaluated allocator knows the trajectory duration but does not read future scores.

Reviewer_zXRoOpenReview ↗

Follow-up on the authors’ response

Thank you for the response and revisions. Figure 1 and Section 3.3 clarify the timing of evidence availability, event onset, and the decision deadline. The latency-shifted evaluation, more complete budget accounting, and Town07 selection rule with ten-seed results address several of my earlier questions. The added clock controls and experiments with more dispersed event onsets are also helpful.

My main concern is still the comparison with other methods. The new results show an advantage for PRS placement at a 0.30 s lead time, but the ranking changes at longer delays. At the original driving pipeline's roughly 1.33 s p95 latency, those same PRS calls cover no windows, compared with 17 for TTC. The content pilot is a useful first step, but it does not yet show that assistance arrives in time or improves driving decisions.

I understand that the paper focuses on offline placement. Even within that scope, the requested comparison on fresh scenarios against stronger current-state and broad-supervision predictors remains important, especially since a simple elapsed-time rule achieves full coverage in both Town10HD cohorts. The development-town results are useful, but do not fully address this concern.

The revisions make the paper's contribution and limitations clearer. I am keeping my score at 2 because the remaining gaps in the comparisons and empirical evaluation still limit the strength of the contribution.

Reviewer_7ZVGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper is about how to properly invoke expensive reasoning backends. Calling the backend too early may provide insufficient information, while calling it too late may leave insufficient time to get feedback. In short, it is an important question. I understand that the authors propose the Predictive Reasoning Scheduler (PRS), which predicts event-relative opportunities for invoking a reasoning backend. It uses recent state history and bird's-eye-view (BEV) information to predict better calling times. There is also a separate selector to determine when to make calls under a limited budget and cooldown constraints.

The authors evaluate their method using autonomous driving trajectories collected in the CARLA simulator. They use Useful-Window Coverage (UWC) as their metric to measure whether reasoning calls are placed within predefined time windows before events occur. This paper includes comparisons of different model architectures, training objectives, and online allocation strategies. However, I found that the experimental results show that PRS achieves higher offline window coverage than several baseline methods, while the online experimental results are less convincing.

I believe that deciding when to invoke reasoning backends is an important problem because agents need to receive results in time to make their decisions. However, I am not convinced that the proposed approach addresses this practical question. Although the paper includes some experiments on backend responses, these results do not convincingly demonstrate the practical effectiveness of the proposed approach.

Critical strengths

This work is about a meaningful and practical problem. Deciding when to invoke expensive reasoning backends is very important for embodied agents. The idea of using event-relative predictions to find better invocation opportunities is useful. If this mechanism works well, it could help many agents use their reasoning backends more efficiently and get useful feedback in time.

The authors conducted many experiments and reported their results in great detail. Besides including comparisons with different baselines, model architectures, training objectives and online allocation strategies. They also provided detailed experimental settings, including random seeds and training configurations. Even though some experimental settings are repeated in different sections, which makes the paper a little difficult to follow. Maybe transferring some tables in formal content is better.

Critical weaknesses

I think the biggest problem is that the proposed evaluation metric cannot demonstrate whether the reasoning calls are actually useful. They use UWC to measure whether calls are placed within predefined time windows before event occur. However, making a call at the right time does not mean the returned result is useful. I found that the backend experiments didn’t provide convincing results. In Appendix E, the calls made 0.5 seconds and 1 second before the event could not return results in time. Although the calls made 2 seconds before the event returned in time, none of them produced results that were both timely and correct.

Another important problem is the difference between offline and online performance. I understand that the proposed model only uses current and past observations to predict invocation opportunities. However, the offline selector uses scores from the entire recorded trajectory to select the calling times. I think this is not appropriate for a practical online system, because the agent cannot know the scores of future observations in advance. The online experimental results also raise serious concerns. As reported in Appendix D.2, the offline UWC on Town10HD is 0.3651, while the causal allocator achieves a UWC of 0. This means that the evaluated online allocation method fails to cover any useful windows in this scenario. I think this is a very important problem. The paper aims to find better invocation times for embodied agents, but the main improvements are achieved in offline experiments. The online results are much less convincing.

Questions for authors
  1. What practical problem does the proposed method actually solve?

I understand that this paper aims to find better invocation times for reasoning backends. However, I still don't understand what specific problems these calls are supposed to solve. For example, in autonomous driving scenarios, why does an agent need additional reasoning when an event is about to occur? How do the authors determine whether an event actually requires reasoning? I think predicting an upcoming event and deciding when to invoke a reasoning backend are not exactly the same problem. Could the authors explain this relationship?

  1. How can the proposed metric demonstrate the actual effectiveness of reasoning calls?

The authors use UWC to measure whether calls are placed within predefined event-relative windows. However, I don't understand why placing a call within these windows means that the call is useful. In Appendix E, calls made 0.5 seconds and 1 second before the event cannot return results in time. Even when calls made 2 seconds before the event return in time, none of them produce results that are both timely and correct. Also, the online results are much less convincing than the offline results. I am not convinced that higher UWC means better reasoning or better decisions. Could the authors provide evidence that the proposed method can actually help agents make better decisions?

  1. Can the proposed method be generalized to other embodied agents?

The paper presents this problem as a general problem for embodied agents. However, the experiments mainly use autonomous driving scenarios in CARLA with a fixed base policy. I think the current evidence is insufficient to support such a broad claim. Could the authors explain whether this method can work with other embodied agents or different control policies? What evidence supports its generalization beyond the evaluated scenarios?

2 comments

Thank you for your careful review and helpful comments. We address each point in turn.

Q1. Practical motivation and the role of event prediction

Our paper studies how to select reasoning-call times under a limited budget. To clarify the motivation for this work, we have added a pedestrian-crossing example to the Introduction. As shown in Figure 1, the base policy continuously controls the vehicle, while additional reasoning can be used to analyze the pedestrian's motion and crossing intent. Appropriate call timing depends on both the availability of sufficient evidence and whether the response can arrive in time to inform a decision.

Event prediction provides a temporal reference for invocation. We treat pre-onset windows as potential calling opportunities. PRS predicts these opportunities from available observations, and a shared selector schedules calls under fixed per-trajectory quotas and minimum-spacing constraints. We compare the coverage of distinct event windows at matched call counts to assess how the predicted scores affect temporal placement.

Q2. Window coverage, response usefulness, and online allocation

To clarify the connection between call placement and reasoning assistance, we have added Section 3.3 to the revised manuscript. We specify that relevant evidence should be available when a call is issued and that the response should arrive before the controller's decision deadline. We also add latency-sensitivity analysis in Section 5, Table 2, and Appendix F.2, respectively. We keep the model, call sets, and window width fixed while shifting the evaluation windows earlier to examine coverage under different response lead-time requirements.

As shown in our results, with 165 calls per method and a 0.30 s shift, PRS covers 22/63 events, compared with a mean of 6.35/63 for Random-k and 8/63 for the lead-time-adjusted TTC rule. This comparison shows that predictive scores can preserve a placement advantage when additional response lead time is required. The advantage is conditional on the specified lead time, and Table 2 reports how coverage and method rankings change across the full sweep.

Appendix E also reports an equal-call backend content pilot that checks window placement, event predictions, and action recommendations against the evaluation rubric. PRS has a normalized answer-content coverage AUC gain of 0.008887 over Current-only, with a 95% confidence interval of [0, 0.024742]. This positive point estimate provides preliminary evidence connecting call placement with answer-content coverage. The pilot uses a 30 s batch timeout, so it evaluates content coverage separately from decision-time response availability. Our claim concerns the contribution of timing prediction to fixed-budget invocation, with downstream driving outcomes assessed separately in Appendix E and Section 7.1.

In a practical online system, PRS can produce an opportunity score at each candidate call time using current and past observations. An online allocator can then determine whether to issue a call based on the score, remaining budget, and minimum-spacing constraint. Our primary experiments evaluate offline ranking over completed trajectories under matched call counts and selection rules. Section 6.2 and Appendix D.2 are used to describe the evaluated sequential allocator’s information conditions, budget use, and comparisons at matched realized call counts.

Q3. Applicability to other embodied agents

PRS is intended for embodied systems that continuously perceive and act under a limited reasoning-call budget. The base policy continuously controls the agent, PRS predicts pre-onset calling opportunities from current and past observations, and a selector schedules calls under budget and minimum-spacing constraints. Applying this formulation to another embodied task involves adapting the observation inputs, target events, and calling windows, and training a corresponding opportunity predictor.

Our CARLA experiments provide a concrete implementation and evaluation of this framework in autonomous driving and can serve as a methodological reference for embodied tasks with similar reasoning-call constraints.

Thank you for your detailed response and the additional experiments. I appreciate the authors' efforts to clarify the motivation and provide more results.

However, I am still not convinced that the main concerns have been addressed. The new experiments show better event-window coverage when the evaluation windows are shifted earlier, but higher coverage does not necessarily mean that the reasoning calls are useful. The results still do not clearly demonstrate that the backend can provide timely and correct feedback or improve the agent's decisions. The additional content pilot also provides only limited evidence of practical benefits.

I also remain concerned about the gap between offline and online performance, as well as the limited evaluation for broader embodied-agent scenarios.

Overall, I think the paper addresses a meaningful problem, but the evidence is still not sufficient to support its broader claims. Therefore, I will maintain my original rating.

Comments on the paper

1 comment

Dear Reviewers,

I hope this message finds you well.

I would like to remind you that we are currently in the rebuttal phase of the paper review process. During this stage, authors may respond to the comments and feedback you provided. If you receive any responses from the authors, please consider providing your feedback to ensure a smooth review process.

Thank you for your hard work and dedication throughout this review process. Your insights are invaluable in enhancing the quality of the submissions, and we truly appreciate your involvement.

If you have any questions or need further assistance, please do not hesitate to reach out.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.