acceptodds
Under review as a conference paper at ICLR 2027

VERA: Verifiable Feasibility Representations with Counterfactual Credit for Constrained Multi-Agent Control

Abstract

Constrained multi-agent control requires more than predicting rewarding actions: an action can cease to be executable as contact windows, shared capacity, and deadlines change. We introduce VERA, a centralized-training, decentralized-execution framework that separates feasibility estimation from credit assignment. Each actor predicts a five-dimensional verifiable feasibility representation (VFR). After an action is proposed, exact action-conditioned margins available only during training supervise that representation, while a counterfactual group-relative advantage (CGRA) ranks candidate representation–action pairs. Execution uses one actor pass and no privileged state. In a dynamic space–air–ground integrated network (SAGIN), VERA obtains success with coverage violation, within points of a privileged-mask reference. With rewards matched over ten paired seeds, VERA improves success over the strongest baseline by points () and reduces violation by points (). A ten-seed factorial attributes a – point gain to CGRA across handcrafted, learned, random, and latent representations; evaluation on seven unseen topologies preserves a – point advantage over multi-agent proximal policy optimization. From 10 to 40 users, success remains –, and VFR adds only ms to a central processing unit (CPU) actor step. Cross-domain tests further identify the governing condition: counterfactual credit succeeds when candidate scores respect shared constraints and fails under incompatible reward geometries. These results establish action-conditioned feasibility as an auditable training interface and counterfactual credit as a geometry-dependent optimization mechanism.

Primary area:
reinforcement learning
Keywords:
Multi-agent reinforcement learning, constrained reinforcement learning, centralized training with decentralized execution, counterfactual credit assignment, verifiable feasibility representations, space-air-ground integrated networks, safe resource allocation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1ETUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies constrained multi-agent control and proposes a method called VERA. The method exploits an asymmetry between training and execution: during training, the system can compute the exact feasibility of an action after it is proposed, whereas during execution it cannot. VERA introduces two training-time mechanisms: VFR (verifiable feasibility representation, a representation supervised by feasibility margins) and CGRA (counterfactual group-relative advantage, which uses a one-sided what-if evaluator to evaluate multiple representation–action candidates and applies group-relative normalization). In SAGIN experiments, under reward-matched conditions, VERA’s success rate is about 9 percentage points higher than the strongest reward-matched baseline in the paper (AB-MAPPO-R), while substantially reducing coverage violations. The experiments show that the performance gain comes mainly from CGRA, while the semantic content of the representation itself has little effect; in addition, in two other domains, CGRA runs into problems such as reward hacking and training collapse.

Critical strengths
  1. The main strength of this paper is its solid analysis of where the performance gain comes from. With extensive comparisons, comprehensive ablations, and rigorous statistical tests, readers can clearly see which factors drive the improvement. In addition, the clear separation between training and execution ensures very low deployment overhead.

  2. The authors honestly report unfavorable results and failure cases in the main text, and this honesty deserves credit.

Critical weaknesses
  1. The performance gain is not disentangled from the use of extra information: CGRA has access to what-if outcomes of unexecuted representation–action candidates, and VERA receives an additional validity reward, neither of which is available to the baselines.

  2. The gain from CGRA is observed only in a simplified custom simulator and fails in the other two domains; its applicability criterion is derived post hoc, yet the title and abstract imply general applicability, which is an overclaim.

Questions for authors
  1. Can the baselines be given the same extra information, i.e., access to what-if outcomes of unexecuted representation–action candidates and the validity reward? If not, how can the gain be attributed to VERA rather than to the extra information?

  2. How does VERA compare empirically to COMA and difference rewards, and what distinguishes CGRA from them conceptually?

  3. Can the applicability criterion for CGRA be stated before the experiments and tested on a held-out domain? Otherwise, would the authors revise the title and abstract to reflect the limited scope?

  4. Please specify the candidate sampling method, the candidate scoring function, and the hyperparameter settings.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 1ETU

We thank the reviewer for the constructive report and for recognizing the mechanism analysis and the honest reporting of negative results. The four questions identify the right pressure points: extra information, comparison with existing counterfactual methods, scope, and implementation details. We answer each in turn, and we agree with the scope criticism.

Before the point-by-point reply, we summarize the common question across the three reviews: whether the gain comes from extra simulator access rather than from the mechanism. We ran the comparison at fixed information. M1 and M3 draw and score the same G = 8 candidates per agent per step (160 evaluations per step) and receive the same VFR supervision and the same validity and certificate terms in the score; M3 additionally uses those scores as a within-group baseline. In the released code, with all four fixes active, over five seeds × 1,000 episodes (last 500), M1 reaches 38.64 ± 1.95% SR and M3 reaches 52.82 ± 0.77%, a paired difference of +14.18 points (p = 7 × 10⁻⁵, 5/5 seeds positive). M1 is only +1.72 points above M0, which uses no candidate scores (p = 0.44).

Three code states are kept separate. S1 is the submitted artifact (fixed bandwidth denominator, EMA peers, what-if certificate target, COV over all users). S2 is the bandwidth-only retrain (proportional bandwidth, exact bandwidth-demand peers, EMA CPU, what-if target, COV over all users); only this fix changes the executed physics, so only this one was fully retrained. S3 is the released code used for every new result: proportional bandwidth, exact CPU and bandwidth peer context from the registered candidate 0, VFR target from the realized step(), and active-user COV. The S3 headline pair is the check that the remaining training-signal fixes do not erase the result.

W1/Q1 — the gain is not disentangled from extra information

The fixed-information ladder separates them. M1 receives the same 160 queries, the same what-if rewards and certificates, and the same VFR supervision as M3, including the validity term −λ_c · viol(a_g, ĉ_i); it differs only in that the scores are not used as a group baseline. M1 is statistically indistinguishable from M0, which uses 20 queries and no scores (+1.72, p = 0.44). Adding the group baseline at the same 160 queries gives +14.18 points (p = 7 × 10⁻⁵, 5/5). Permuting the scores within the group returns to the M0 level (−2.66 vs. M0, p = 0.53), and a global scale instead of the per-group scale changes nothing (−0.10, p = 0.93). The validity reward is therefore present in the control arm; what the control lacks is the within-group comparison.

A baseline can obtain the same information by paying the same 8× simulator cost. What it cannot obtain for free is the comparison of its own candidate outcomes within the group. The M1 arm is exactly the "same extra information" baseline requested in this question.

AuthorsOpenReview ↗

Response to Reviewer 1ETU — Part 2

Continued from Part 1. This part covers Q2 (COMA and difference rewards), W2/Q3 (scope and overclaiming), and Q4 (sampling, scoring, and hyperparameters).

Q2 — comparison with COMA and difference rewards

Our original COMA baseline implemented the baseline on the wrong variable: it held the agent's own executed action fixed and sampled peers. The COMA paper defines the baseline by marginalizing the agent's own action while holding peers at their executed actions. We reimplemented it accordingly — centralized Q(s, a_1 … a_K, i), M = 4 samples of the agent's own action, Q − b z-scored and blended exactly as CGRA, with G = 1 so that the advantage source is the only difference — and reran five seeds under the corrected physics.

ArmSR (%)COV (%)Local share
COMA (corrected, M = 4)53.30 ± 0.020.00100% from episode 30
CGRA (= M3, S2)53.65 ± 0.822.01 ± 0.1794.9%
All-local control53.31 ± 0.210.00100%

The corrected baseline does not fail, but it does not learn to offload either. Coverage violations fall to exactly zero by episode 30 and stay there in all seeds; an instrumented replay shows the local share rising from 9.6% to 100.0% at episode 30; the converged SR matches the never-offload control to 0.01 points. The critic explains it: after correct marginalization, mean Q and mean b agree to seven decimals (143.594685 vs. 143.594688), so the advantage is numerically zero and its z-score carries no action-dependent signal. COMA converges to the conservative local policy — far better than the earlier incorrect run, but not evidence of selective offloading. CGRA reaches a comparable SR while offloading on 5.1% of decisions and without training a critic.

On difference rewards: a difference-reward baseline requires a per-agent difference function D_i = R − R_(−i), which in turn requires a decomposition of the global reward. The coverage term here is a shared constraint, not a separable sum of agent contributions, so we did not construct such a baseline and do not claim an empirical comparison. Conceptually, CGRA estimates the counterfactual from the policy's own candidates rather than from a designed reward decomposition, which is what allows it to use the simulator's per-candidate rewards and certificates directly. We state this as a conceptual distinction, not as an empirical result.

W2/Q3 — scope, post-hoc criterion, and overclaiming

We agree with the criticism. The gain is established in SAGIN and does not transfer to the two stress domains; the criterion in Section 5 was induced from those failures rather than stated in advance; and the earlier abstract and conclusion implied broader applicability than the evidence supports.

The revision rewrites the abstract and conclusion to scope the claim to constrained offloading of this class, states the criterion as an explicit, testable hypothesis before the experiments, and keeps the title unchanged because it already names the constrained setting. A held-out-domain test is the next step and is not claimed here. The all-local control further narrows the performance claim: in this benchmark, most of CGRA's benefit is avoiding harmful offloads rather than beating a local-only policy. Never offloading scores 53.31 ± 0.21% SR with 0.00% COV; selective offloading adds +3.33 points (p = 0.115). The revision states this bound explicitly.

Q4 — candidate sampling, scoring, and hyperparameters

  • Sampling: G = 8 i.i.d. samples from π_θ(· | o_i) per agent per step, drawn in one batched pass; no top-k, no reranking; candidate 0 is executed.
  • Decoding: each candidate maps to a node n_g, an offload ratio ρ_g, and a bandwidth request b_g.
  • Scoring: r_exec(g) is the RLVR execution reward (success, latency, weighted energy, coverage violation); c*(g) is the five-dimensional certificate; r_total(g) = r_exec(g) + λ_v · [ −(1/5) || ĉ_i − c*(g) ||₁ ] + λ_c · [ −viol(a_g, ĉ_i) ], with λ_v = 0.5 and λ_c = 0.1.
  • Advantage: A_i = clip( (r_total(0) − mean_g r_total) / (std_g r_total + 10⁻⁸), −3, 3 ), blended as 0.5 · GAE + 0.5 · A_i.
  • Training: five seeds × 1,000 episodes, last 500 episodes as the reported window; the constrained baseline uses 3,000 episodes at the matched budget.
  • Configuration: full hyperparameter snapshots are in the updated artifact under rebuttal_evidence/configs/ (experiment_configs.json, mechanism_ladder.yaml), with per-seed CSVs and the recomputation script in the same directory.

Appendix A.2 gives the same definitions and the pseudo-code; the implementation is in grpo_trainer.py (_grpo_group_step) and env/sagin_env.py (evaluate_candidates).

Thank you for the detailed responses and for the information-matched ladder, the released-code rerun, the corrected COMA run, Algorithm 1 and Table 14, which address much of my review. Numbers refer to the revised PDF. Three points remain open for me; none of them needs a new experiment.

  1. W1/Q1 (where the gain comes from). The what-if half is answered within VERA: at the same 160 queries the within-group baseline adds +14.18 points on the released code (p. 15), and M1 shows no detected gain over M0 (Table 9). Still open: (a) p. 7 and p. 19 say the arms share a reward, but not whether M0 and M1 carry Eq. 6's λ_v and λ_c terms. Your reply's "validity term" −λ_c·viol is Eq. 5's consistency term, so the λ_v term (Q1's validity reward) remains open. (b) Table 9 and p. 7 call M2 a "global baseline", but p. 5 and your reply treat M3−M2 as a change of scale. If M2 uses a global baseline, M2 ≈ M3 (54.35 vs 54.25) does not single out the within-group baseline, as the abstract does. Which is meant? (c) Table 5 places the ladder behind the abstract's +1.72 and +15.20 under the submitted code S1, and only M1/M3 were rerun. The Table 9 caption could say so and the abstract quote the released-code +14.18.

  2. W2/Q3 (scope of the claims). Thank you for agreeing on scope and for the all-local bound, new abstract ending and narrowed conclusion. The abstract's five-seed 55.33/0.45 and Table 2's S5 block have no all-local reference, though the control's 53.31% SR exceeds every non-privileged baseline there. Could both show it? The p. 2 heading now says "stated before the experiments", while your reply says the criterion was induced from the failures. My Q3 did say "before the experiments", but I meant before a test that could refute it. A heading matching your reply would settle this.

  3. Q2 (COMA). A.9 infers a zero advantage from equal per-seed means of Q and b, but since b averages Q over the agent's own action under the policy, Q − b has zero mean in expectation for any Q. The "numerically zero" sentence could rest on the behavioral evidence.

Minor: a table of λ_v, λ_c, λ_g and the PPO settings (Eq. 5 also lacks the 1/5 of Algorithm 1, line 9); a citation for difference rewards in A.9; and a budget and code state per caption, since the full method's COV is 0.45 (Table 2), 2.33 (Table 7), 2.78 (Table 12) and 2.37 (Table 13).

Thank you again.

Reviewer_1N9EOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VERA, a centralized-training, decentralized-execution framework for constrained multi-agent control. VERA combines a five-dimensional verifiable feasibility representation (VFR), supervised using simulator-computed feasibility targets after action selection, with a counterfactual group-relative advantage (CGRA) that scores candidate representation–action pairs while holding peers’ requests fixed. The verifier and candidate scoring are used only during training, while execution requires a single decentralized actor pass. The method is evaluated primarily in a space air ground integrated network (SAGIN) simulator using reward-matched baselines, extensive ablations and factorial experiments, topology and scale generalization, and additional cross-domain tests. The authors report improved success and constraint compliance over MAPPO-based baselines in SAGIN, with the experiments identifying CGRA as the main performance driver while also documenting its failures in resource-allocation and force/contact settings.

Critical strengths
  1. Careful experimental design and mechanism attribution. The paper uses reward-matched baselines, paired-seed analyses, a seven-arm mechanism ladder, and a representation × CGRA factorial. In particular, CGRA improves success by roughly 14–16 points across all four representation types. This makes the paper’s conclusions unusually checkable and helps identify which components actually drive the reported improvements.
  2. Thorough evaluation of generalization and failure modes. The paper evaluates topology and scale shifts as well as additional control domains, and openly reports cases where CGRA fails, including resource-allocation and force/contact settings. These negative results help define the limitations of the proposed mechanism and are scientifically useful even independently of the positive SAGIN results.
Critical weaknesses
  1. The importance of the semantic VFR is not convincingly established by the paper’s own ablations. Random supervised targets nearly match the handcrafted VFR when CGRA is active (Table 9), removing or permuting individual VFR coordinates has little effect (Table 8), and zeroing the VFR input at inference leaves success essentially unchanged (Table 16). Moreover, in the retrained mechanism ladder, feeding VFR into the action head provides no detected benefit over auxiliary VFR supervision alone (Table 7). These results suggest that auxiliary supervision may be useful, but do not establish that the physically meaningful VFR semantics or their direct use by the policy drive the gains. This weakens a central part of the paper’s framing around verifiable feasibility representations.
  2. The contribution of CGRA is not sufficiently isolated from its privileged training-time candidate evaluation. CGRA evaluates eight alternative representation–action candidates using simulator-side information unavailable to the deployed policy, while the exact candidate score (r_i^{(g)}) is not fully specified. The paper does not compare against a baseline given the same candidate-evaluation budget or against established counterfactual credit approaches under comparable information. This is particularly important because removing CGRA reduces success from 54.57% to 40.12% and increases coverage violation from 2.33% to 62.23% (Table 7). Without such controls, it is difficult to determine whether the improvement comes from the proposed credit-assignment mechanism itself or from the additional training-time candidate evaluations.
Questions for authors
  1. Could the authors provide the complete definition of the CGRA candidate score (r_i^{(g)}), including exactly which simulator-side quantities and feasibility information are used to evaluate each candidate?
  2. Since CGRA is the dominant contributor to performance, how does it compare with a baseline given the same eight candidate evaluations and the same privileged training-time information, but using a simpler selection/credit rule or an established counterfactual credit method? This would help determine whether the gains arise from the proposed group-relative advantage formulation or from access to additional candidate evaluations.
  3. Could the authors report the distribution of execution choices (e.g., local, LEO, and UAV) for VERA and the main baselines? VERA’s very low energy consumption together with its relatively high latency raises the possibility that it learns a substantially different offloading strategy; understanding this would affect how I interpret its near-zero coverage violation.
  4. Since random supervised targets nearly match the handcrafted VFR in performance and compliance (Table 9), what evidence demonstrates that the physically grounded VFR provides greater auditability than random-target or latent representations? For example, does the signed-margin calibration reported in Table 11 degrade when the representation is not physically grounded? Evidence of such a distinction would affect my assessment of the VFR contribution.
4 comments
AuthorsOpenReview ↗

Response to Reviewer 1N9E

We thank the reviewer for the detailed reading and for the two central weaknesses, which identify the right questions: whether the semantic VFR is doing anything, and whether CGRA is separable from privileged candidate evaluation. Both are now addressed with new controls and a corrected baseline. We also appreciate the positive assessment of the experimental design and of the negative results.

Before the point-by-point reply, we summarize the common question across the three reviews: whether the gain comes from extra simulator access rather than from the mechanism. We ran the comparison at fixed information. M1 and M3 draw and score the same G = 8 candidates per agent per step (160 evaluations per step) and receive the same VFR supervision; M3 additionally uses those scores as a within-group baseline. In the released code, with all four fixes active, over five seeds × 1,000 episodes (last 500), M1 reaches 38.64 ± 1.95% SR and M3 reaches 52.82 ± 0.77%, a paired difference of +14.18 points (p = 7 × 10⁻⁵, 5/5 seeds positive). M1 is only +1.72 points above M0, which uses no candidate scores (p = 0.44): the queries alone do not produce the gain.

Three code states are kept separate. S1 is the submitted artifact (fixed bandwidth denominator, EMA peers, what-if certificate target, COV over all users) and produced the original ladder and Tables 7/9. S2 is the bandwidth-only retrain (proportional bandwidth, exact bandwidth-demand peers, EMA CPU, what-if target, COV over all users); only this fix changes the executed physics, so only this one was fully retrained. S3 is the released code used for every new result: proportional bandwidth, exact CPU and bandwidth peer context from the registered candidate 0, VFR target from the realized step(), and active-user COV. The S3 headline pair above is the check that the remaining training-signal fixes do not erase the result.

Q1 — complete definition of the CGRA candidate score

For agent i, the actor emits G = 8 i.i.d. samples from π_θ(· | o_i) in one batched pass. There is no top-k filtering and no reranking; candidate 0 is executed. Each candidate decodes to a node n_g, an offload ratio ρ_g, and a bandwidth request b_g, and is scored by env.evaluate_candidates under the registered same-slot peers.

  • r_exec(g) is the RLVR execution reward: success, latency, weighted energy, and coverage violation, with the same weights as the executed step.
  • c*(g) is the five-dimensional certificate computed by the simulator for that candidate.
  • r_total(g) = r_exec(g) + λ_v · [ −(1/5) || ĉ_i − c*(g) ||₁ ] + λ_c · [ −viol(a_g, ĉ_i) ], with λ_v = 0.5 and λ_c = 0.1.
  • The group-relative advantage is A_i = clip( (r_total(0) − mean_g r_total) / (std_g r_total + 10⁻⁸), −3, 3 ), blended as 0.5 · GAE + 0.5 · A_i.

Appendix A.2 gives the same definition and the pseudo-code. The implementation is in grpo_trainer.py (_grpo_group_step) and env/sagin_env.py (evaluate_candidates). All simulator-side quantities used for scoring are listed above; no deployed-time signal is used at execution.

AuthorsOpenReview ↗

Response to Reviewer 1N9E — Part 2

Continued from Part 1. This part covers W1/Q4 (VFR necessity and the audit), Q2 (simpler rules and COMA), Q3 (execution distribution), and W2 (decoupling from privileged evaluation).

W1/Q4 — is the VFR necessary, and is Table 11 degenerate?

We agree that the ablations do not make VFR a source of performance, and the revision no longer claims that. VFR is framed as a dense supervision interface that makes the feasibility estimate auditable, not as a performance driver.

The audit must use AUC(ĉ), not Table 11. Table 11 bins decisions by the true margin, for which AUC(c*) = 1.000 in every arm by construction; it therefore audits the certificate, not the learned head. On 20,000 decisions per arm, only semantically grounded heads rank violations above chance:

HeadVFR MAEAUC(ĉ)corr(ĉ, c*)
Handcrafted0.0770.926 ± 0.028+0.84
Learned0.0800.938 ± 0.047+0.88
Random0.6260.499 ± 0.204+0.00
Latent-5D1.0270.487 ± 0.052−0.02

AUC(c*) = 1.000 for all four rows. This is the evidence that physical grounding buys auditability; it does not buy success rate, and we do not claim that it does.

Q2 — comparison with simpler rules and established counterfactual methods

M2 and M4 are exactly those controls. M2 keeps the same candidate scores but uses a global scale, and is indistinguishable from CGRA (−0.10 points, p = 0.93). M4 permutes the scores within each group and collapses to the no-score level (−2.66 points vs. M0, p = 0.53). A simpler selection or credit rule therefore does not reproduce the gain.

We also corrected and reran COMA. Our earlier baseline was implemented on the wrong variable: it held the agent's own action fixed and sampled peers. The definition in Foerster et al. (2018) is the opposite — marginalize the agent's own action and hold peers at their executed actions. The corrected implementation uses a centralized Q(s, a_1 … a_K, i), M = 4 samples of the agent's own action, and the same Q − b z-scoring and blending as CGRA, with G = 1 so that the advantage source is the only difference. It was rerun for five seeds under the S2 physics.

ArmSR (%)COV (%)Local share
COMA (corrected, M = 4)53.30 ± 0.020.00100% from episode 30
CGRA (= M3, S2)53.65 ± 0.822.01 ± 0.1794.9%
All-local control53.31 ± 0.210.00100%

The corrected COMA does not fail, but it does not learn selective offloading either: coverage violations reach exactly zero by episode 30 and stay there in all seeds, an instrumented replay shows the local share rising from 9.6% to 100.0% at episode 30, and the converged SR matches the never-offload control to 0.01 points. The critic explains it: after correct marginalization, mean Q and mean b agree to seven decimals (143.594685 vs. 143.594688), so the advantage is numerically zero. CGRA reaches a comparable SR while offloading on 5.1% of decisions and without training a critic. We withdraw both the old numbers and the old claim that they showed COMA's weakness; COMA is reported as an alternative that avoids harmful offloads.

Q3 — execution distribution

VERA acts locally on 94.9% of active decisions and offloads mostly to LEO at ρ ≈ 0.59; without CGRA the local share falls to 37.4%.

PolicyLocal (%)LEO (%)UAV (%)
VERA94.94.50.6
w/o CGRA37.453.69.0
Backbone46.045.09.0
All-local100.00.00.0

The all-local control puts these numbers in perspective: never offloading scores 53.31 ± 0.21% with 0.00% COV, so low COV is partly conservatism. The incremental value of the 5.1% offloading decisions is +3.33 points over never offloading (p = 0.115 over five seeds). The revision states this bound in the execution subsection and Appendix A.15.

W2 — CGRA is not decoupled from privileged candidate evaluation

This is the same question as R1-W1, and the fixed-information ladder answers it. The CGRA-free arm already receives the privileged scores for VFR supervision and still fails; only using them as a group baseline changes behavior. M1 is the exact control: same queries, same certificates, same supervision, no group baseline, and no gain over M0. The benefit is in the within-group comparison, not in the information itself; a baseline can obtain the same information by paying the same 8× simulator cost, but cannot obtain the comparison for free.

Most of my concerns has been resolved. I have one follow-up regarding Q3, since VERA chooses local execution in 94.9% of cases and its improvement over the all-local baseline is not statistically significant (p = 0.115), how much practical benefit does CGRA provide beyond learning a conservative policy, especially when offloading is necessary?

AuthorsOpenReview ↗

Response to Reviewer 1N9E — follow-up on Q3

Thank you for the follow-up; the premise is correct, and we will give the bound rather than defend a larger claim.

How large is the gain over never offloading? It is small and not statistically resolved, and it depends on the training budget. On the released code (S3, five seeds × 1,000 episodes), the complete method reaches 52.82±0.77%52.82\pm0.77\% SR and the all-local controller 53.31±0.21%53.31\pm0.21\%: the difference is −0.49-0.49 points (paired p=0.315p=0.315, 1/5 seeds positive). In the matched 3,000-episode retrain under the corrected physics, the complete method reaches 56.64±3.59%56.64\pm3.59\% and the difference is +3.33+3.33 points (p=0.115p=0.115, 5/5 positive). Across these runs the margin over never offloading ranges from −0.5-0.5 to +3.3+3.3 points and is never significant. We therefore do not claim a success-rate advantage over a conservative local policy.

What CGRA provides beyond conservatism. The relevant alternative for a learned policy is not all-local execution but an offloading policy. With the same G=8G=8 queries and the same VFR supervision but no group baseline (M1), the policy offloads heavily and reaches 38.64±1.95%38.64\pm1.95\% SR with 46.15±6.24%46.15\pm6.24\% COV, against 52.82±0.77%52.82\pm0.77\% and 2.75±1.42%2.75\pm1.42\% for the complete method in the same released-code run. At the matched 3,000-episode budget, VERA has the highest SR of the compared methods (56.64±3.59%56.64\pm3.59\%) and the lowest COV among those that offload (0.45±0.24%0.45\pm0.24\%, against 45.32% for AB-MAPPO and 69.54% for MAPPO). The practical role of CGRA is therefore to keep a learned offloading policy near the local-execution ceiling while removing the 43–67 points of violation that the alternative learned policies incur. If a deployment can always fall back to local execution, the benefit is the small margin above; if it must use a learned offloading policy, the benefit is the compliance gap, not the SR gap.

When offloading is necessary. We tested whether the paper's shifts create such a regime, using the same five seeds and 30 episodes per seed. A never-offloading controller reaches 25.18±0.20%25.18\pm0.20\% SR under the 0.10-s deadline and 24.28±0.20%24.28\pm0.20\% under heavy tasks. A fixed controller that offloads every task to LEO, at the best of five fixed (node,ρ,b)(node,\rho,b) settings, reaches only 15.53±4.18%15.53\pm4.18\% and 13.48±3.64%13.48\pm3.64\%. These shifts mainly contract the feasible task set; offloading does not pay on average even there. The retrained checkpoint also stays above 90% local on average under both shifts, so the policy does not switch to offloading when the deadline tightens. We therefore cannot claim a large offloading benefit in forced-offload regimes. Our claims are scoped to settings in which local execution is feasible and the objective is compliant offloading; a compute-limited regime would require a different training protocol and is outside the scope of this paper.

Appendix A.15 now reports both controls and states this limitation explicitly; the abstract and the text accompanying Table 2 already report the +3.33+3.33/p=0.115p=0.115 bound. We prefer a stated bound to an overclaim, and we thank the reviewer for pressing the point.

Reviewer_DwZtOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VERA for constrained multi-agent control. Each actor predicts a five-dimensional feasibility representation (VFR) supervised by simulator-computed margins, and the policy is trained with a counterfactual group-relative advantage (CGRA) that scores GG candidate actions per agent. Experiments on a SAGIN offloading simulator, with reward-matched baselines, factorial ablations and cross-domain tests, show higher success and fewer coverage violations; the authors attribute most of the gain to CGRA.

Critical strengths
  1. Careful mechanism attribution. The reward-matched comparison and the 4×24 \times 2 factorial separate the representation from credit assignment, and the paper openly reports that the semantic coordinates are not the main driver.

  2. Informative negative results. The resource-allocation and force/contact failures (Tables 14–15) show concretely when local candidate scoring breaks shared constraints.

Critical weaknesses
  1. The main comparison is not information-matched. At every step, CGRA obtains for each of G=8G = 8 candidates per agent the reward and feasibility certificate computed by the simulator's what-if scorer, which the reward-matched baselines do not receive. Removing CGRA alone raises COV from 2.33% to 62.23% (p. 6), so part of the gain may come from this extra labeled information rather than from the proposed interface.

  2. The implementation differs from the described method. Appendix A.1 states that concurrent bandwidth requests are allocated proportionally, but in the supplementary code bandwidth is divided by a fixed max⁡(0.3K,1)\max(0.3K, 1), independent of how many agents actually share a node, so bandwidth contention is not modeled. The "fixed same-slot peers" of Sec. 3.4 are approximated by an EMA of past loads, and the VFR targets are computed by this what-if scorer rather than from the realized joint step.

Questions for authors
  1. Could the authors compare against a baseline that receives the same candidate queries (e.g., MAPPO with the same one-step candidate baseline) and report the total number of simulator calls?
  2. Which code version produced Tables 2 and 9? Please reconcile the bandwidth model, peer handling and certificate timing with Sec. 3 and A.1.
  3. COV appears to count violations of idle users in the numerator but not in the denominator (one archived trace exceeds 100%). Could the authors recompute COV over active users only?
  4. Could the constrained baseline in Table 2 be rerun under VERA's protocol? It was trained for 1000 episodes (VERA: 3000) and its configuration was selected by success rate rather than constraint satisfaction. If VERA still clearly outperforms a query-matched baseline and the implementation matches the described physical model, I would be willing to raise my score.
4 comments
AuthorsOpenReview ↗

Response to Reviewer DwZt

We thank the reviewer for the careful reading and for four specific, actionable points. Two of them identified genuine implementation defects, one identified a definitional error in COV, and one identified an under-powered baseline. All four are fixed in the revision, and the updated artifact documents each fix. We are grateful that the review states the condition under which the score could be raised; we address exactly that condition below.

Before the point-by-point reply, we summarize the question that all three reviews converge on: whether the gain comes from the extra simulator access rather than from the mechanism. We ran the comparison at fixed information. M1 and M3 draw and score the same G = 8 candidates per agent per step (160 evaluations per step) and receive the same VFR supervision and consistency objective; M3 additionally uses those scores as a within-group baseline. In the released code, with all four fixes active, over five seeds × 1,000 episodes (last 500), M1 reaches 38.64 ± 1.95% SR and M3 reaches 52.82 ± 0.77%, a paired difference of +14.18 points (p = 7 × 10⁻⁵, 5/5 seeds positive). M1 is only +1.72 points above M0, which uses no candidate scores at all (p = 0.44): paying for the queries is not what produces the gain.

Three code states are kept separate throughout this reply, because they produced different experiments. S1 is the submitted artifact: fixed bandwidth denominator, EMA peer loads, what-if certificate target, COV numerator over all users. It produced the original M0–M4 ladder, Tables 7 and 9, and the frozen-policy audits. S2 is the bandwidth-only retrain: proportional bandwidth, exact bandwidth-demand peers, EMA CPU, what-if target, COV over all users. It produced the M1/M3 retrain, VERA 3,000, the constrained baseline, and the corrected COMA. S3 is the released code used for every new result: proportional bandwidth, exact CPU and bandwidth peer context from the registered candidate 0, VFR target from the realized step(), and active-user COV. Only the bandwidth fix changes the executed physics, so only it triggered a full retrain of the method's pair; the S3 headline pair above is the check that the two training-signal fixes do not erase the result.

W1/Q1 — the main comparison is not information-matched

The premise needs one correction, and then one experiment answers the question.

In the released code, use_cgra=False does not remove the candidate queries. train_episode always scores G = 8 candidates per agent and uses the returned certificates to supervise VFR; the flag only sets w_group = 0 (grpo_trainer.py:157,482). Tables 7 and 9 therefore already compare two advantage rules under identical simulator access and identical query cost.

What those tables did not isolate is whether paying for the queries helps by itself. The M0–M4 ladder does. M1 pays the same 160 evaluations per step as M3 but receives no group-relative baseline, and it gains only +1.72 points over M0, which scores no candidates at all (p = 0.44). M3 gains 14–15 points over M1 across the three code states. Permuting the candidate scores within each group returns the policy to the M0 level (M4 − M0 = −2.66 points, p = 0.53), and replacing the per-group scale with a global one changes nothing measurable (M3 − M2 = −0.10 points, p = 0.93). The active ingredient is the scores used as a counterfactual group baseline, not the number of queries and not the z-score form.

M1 is therefore precisely the baseline requested in this question: same eight candidates, same what-if rewards and certificates, same VFR supervision, no group baseline. The call count is exactly 20 vs. 160 per step, verified per run. Wall-clock is not a controlled measurement because the GPUs were shared, so we do not use it as evidence.

AuthorsOpenReview ↗

Response to Reviewer DwZt — Part 2

Continued from Part 1. This part covers W2/Q2 (implementation vs. description), Q3 (COV accounting), Q4 (constrained baseline), and the closing.

W2/Q2 — implementation does not match the description

All three discrepancies are real. The revision fixes them, and the paper now states which code state produced which result.

Bandwidth. The submitted code divides bandwidth by max(0.3K, 1) = 6 regardless of concurrent demand. Replaying frozen policies under the proportional rule leaves actions bit-identical, because bandwidth is not an observation; success moves by at most 0.33 points and the CGRA gap moves from 13.6 to 13.4 points. Retraining the pair under the corrected rule (S2) reproduces the gap: +15.48 points, p = 0.0004, 5/5 seeds. The defect is therefore not the source of the gain.

Peers and certificate timing. For the complete method, the what-if certificate differs from the realized-step certificate by 0.004 on average, with identical coverage verdicts in 100.0% of decisions and delay-buffer sign agreement in 99.5%. The discrepancy is concentrated in the high-offload regime: 0.064 mean absolute difference and 90.7% delay agreement for the CGRA-free arm. The released code registers the realized slot context before scoring and takes the VFR target from step(). This fix was not retrained in S2; it is included in the S3 verification reported above, which is the check that it does not erase the result.

Which code produced which table. Table 7 reproduces to 0.005 pp from the archived logs. Table 9's COV column cannot be recomputed value-for-value, because the archive stores only the aggregated rate under the submitted accounting, not the per-user violations; we therefore quantify the definition on identical rollouts instead (Q3).

Q3 — COV numerator counts all users, denominator only active tasks

Correct. We fix metrics.py and report the two definitions on the same rollout (five seeds × 30 episodes, best-SR checkpoint). Only the within-rollout difference is used; the replay is not compared with the converged-window table values.

ArmSubmitted (%)Active-user (%)Δ (pp)
Full (VERA)5.48 ± 5.414.08 ± 4.75+1.40
w/o CGRA61.16 ± 8.0547.85 ± 5.20+13.31
Backbone56.21 ± 18.8041.94 ± 18.34+14.27
AB-MAPPO-R56.11 ± 13.6042.65 ± 11.68+13.46

The correction lowers high-violation arms by roughly 13–14 points and leaves the CGRA-armed arms essentially unchanged. Two adjacent pairs swap order within the replay; we do not use the replay to re-rank the ladder. Appendix A.10 states the accounting explicitly.

Q4 — the constrained baseline in Table 2 is under-trained and selected by SR

Both points are correct. A pre-registered constraint-first rule (minimize converged COV, break ties by SR) selects the same configuration as the SR-first rule, so selection is not the issue. At the matched 3,000-episode budget the baseline is worse, not better: 33.35 ± 3.69% SR and 54.56 ± 9.22% COV, against 38.07 ± 3.94% SR and 38.43 ± 9.52% COV for the same seeds at 1,000 episodes (paired ΔSR = −4.7, p = 0.015; ΔCOV = +16.1, p = 0.005). The dual variable sits at its ceiling (λ_final = 4.98–5.00) on all five seeds. The revision reports the matched-budget row and keeps the submitted row with its original protocol.

The condition stated in the review was a query-matched comparison and an implementation that matches the described physical model. The query-matched baseline is M1, and the complete method exceeds it by 14.18 points with identical queries; the implementation now matches Section 3 and Appendix A.1, and the bandwidth replay bounds the effect of the former defect at 0.33 points. We hope this addresses the condition, and we would be grateful if the reviewer would reconsider the score in light of the S3 verification.

Thank you for the detailed responses and additional experiments. The information-matched comparison on the corrected code and the matched-budget constrained baseline substantially address my main concerns.

One issue remains: the abstract still reports a 52.19-percentage-point reduction in COV, while Appendix A.10 states that this figure uses the original, incorrect accounting. These inconsistency remain despite the improvements in the revision, so I maintain my original rating.

AuthorsOpenReview ↗

Reply to the remaining inconsistency

Thank you for the careful reading and for stating the remaining issue so precisely. You are right: the revised abstract still reported 52.19 points while Appendix A.10 said that this figure uses the original (incorrect) all-user accounting. Annotating the number was not enough, and we have now removed it from the abstract rather than qualifying it.

What changed in the revised manuscript

  • Abstract. The abstract now reports only the corrected measurement: "With rewards matched over ten paired seeds, VERA improves success by 8.74 points (p = 0.023); a five-seed same-rollout replay under the active-user definition gives a 38.57-point violation gap over AB-MAPPO-R (p = 0.0035, Appendix A.10)." The 52.19 figure no longer appears in the abstract. The revised PDF carries this text, and we are updating the OpenReview abstract field to match.
  • Section 4.1. The ten-seed value is retained only with an explicit label — "reduces COV by 52.19 points under the all-user definition" — and is followed immediately by the corrected replay: AB-MAPPO-R goes from 56.11% to 42.65%, the complete method from 5.48% to 4.08%, a corrected difference of 38.57 points (paired p = 0.0035 over five seeds, all five positive; the correction itself is 12.06 points, p = 0.0083).
  • Conclusion. The all-user rates are labeled as such (0.51% vs 52.70%), and the corrected replay is given in the same sentence (4.08% vs 42.65%, Appendix A.10).
  • Appendix A.10. The appendix now states explicitly why the ten-seed 52.19 cannot be corrected after the fact: the archived logs store only the aggregated all-user rate, not the per-user violation records needed to recompute the active-user denominator. The 38.57 figure is a separate five-seed, same-rollout replay at the best-SR checkpoints, not a post-hoc correction of the ten-seed value. We say this in the appendix so that the provenance of each number is unambiguous.

Why 52.19 remains in Section 4.1 and A.10

We kept the ten-seed value there, explicitly labeled, because the ten-seed protocol was the planned matched comparison and the corrected replay covers five seeds. Removing it entirely would hide a real measurement; presenting it without the label was the actual error. The abstract, which is what a reader sees first, now carries only the corrected replay and no unqualified 52.19.

The revised PDF, abstract, and supplementary artifact are updated in the submission. Thank you again for holding us to this — the inconsistency should not have survived the first revision, and the current text no longer asks the reader to reconcile two accounts of the same quantity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.