acceptodds
Under review as a conference paper at ICLR 2027

Scaling Graph Topology Augmentation with Structured Variations and Spectral Trust in Non-stationary Reinforcement Learning

Abstract

Graph Topology Augmentation (GTA) improves prioritized planning under reward change. Yet the current compute model hinder scalability, with critical bottlenecks in CPU residence, exact matrix operator representation and obnoxious eigensolve. Therefore, we introduce GTA-plus-PS, which stages changed transition rows as rank- shocks and publishes consistent belief state through coherent commits. Moreover, a trace-derived spectral trust certificate replaces periodic second-largest-eigenvalue-modulus estimation. We derive exact Woodbury updates for resolvent actions, connect the trace to discounted signal variance, and prove convergence under deterministic or variance-scaled stochastic selection. Concurrently, we map all parallelizable operations onto GPU-resident computation model, coupling with an efficient implementation that enables streamlined inference-learning. We evaluate the new design on an extensive suite of varying environments, representation, host model, and learn update. Our augmented methods gain performance over their standard baseline, in both tabular and visual-control setting.

Primary area:
reinforcement learning
Keywords:
nonstationary, reinforcement, planning, prioritized, sweeping, topology, resolvent, spectral, variance, trust, Woodbury, successor, drift, acceleration, GPU

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_33uuOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GTA+-PS to improve the scalability of graph-topology-augmented prioritized planning in non-stationary reinforcement learning. It uses batched Woodbury updates to maintain resolvent operators under localized transition changes, introduces a trace-derived Spectral Trust statistic for priority weighting and Gumbel-based selection, and employs coherent commits to keep numerical and queue states consistent.

Critical strengths
  1. Batched low-rank updates and separation of operator changes from reward-only refreshes directly address a relevant computational bottleneck in topology-aware planning.

  2. Coherent commits, sparse/dense backends, and numerical checks form a useful framework for maintaining consistent planning state under asynchronous updates.

Critical weaknesses
  1. In Table 2, all 12 recovery-time entries for GTA-PS and GTA+-PS are explicitly marked as projected, yet they are presented alongside measured baseline results to support substantial wall-clock improvements.

  2. Stage I mainly considers reward shifts, whereas the Woodbury maintenance scheme is intended for changes in transition dynamics. There is a lack of more direct assessments conducted in contexts where transfer characteristics change.

  3. The wall-clock experiments compare CPU-based baselines with GPU-accelerated variants, and Stage II reports only one independent training run per method. These limitations make the improvements less conclusive.

Questions for authors
  1. Table 2 marks all 12 recovery-time results for GTA-PS and GTA+-PS as projected rather than measured, despite using them to support the paper's central efficiency claims. Why are projected values presented as benchmark results, and what empirical evidence supports these numbers?

  2. Can the authors provide actual per-seed wall-clock measurements? Without such evidence, how can the claimed speedups be considered experimentally validated?

  3. Can the authors evaluate GTA+-PS under explicit transition-model changes, rather than primarily reward shifts, to show when incremental operator maintenance is beneficial?

  4. The no-jitter variant is useful, but can the authors further isolate the effect of Spectral Trust?

3 comments
AuthorsOpenReview ↗

Official Rebuttal of 33uu's Review by Authors

We thank Reviewer 33uu for the direct and precise review. We apologize for the projected entries in Table 2. Please disregard them.

This reply first gives the measured Stage-I results. We then answers each weakness and question in order.


Requested Data

We demonstrate the stage-I canonical result, under more developed engineering.

We adjusted three protocols: a 90% gap target (was 80%), 30 seeds at 1k (was 10), and added transition changes. All improve the result's fidelity.

Table C.1. Wall time per seed (s), mean ± 95% interval.

MethodBackendSysAdmin-1024 (dense)CargoTaxi-1280 (sparse)SysAdmin-4096 (dense)CargoTaxi-4500 (sparse)
Standard-PSNumPy, CPU66.1 ± 2.17.1 ± 0.8236.3 ± 4.617.2 ± 3.2
GPS-H2NumPy, CPU60.3 ± 1.710.5 ± 0.9220.0 ± 5.633.8 ± 3.3
GTA-PSJAX, A10082.7 ± 2.7181.1 ± 20.6759.8 ± 2.63,588.9 ± 623.0
GTA-TrustJAX, A10052.5 ± 1.9100.9 ± 11.5584.1 ± 1.61,567.2 ± 272.7
GTA-FullJAX, A10030.5 ± 1.125.9 ± 5.6177.3 ± 1.7311.5 ± 50.7

CargoTaxi uses the sparse option for low branching factor (~5) and inherent 2D model (low-dimensional).

Table C.2. Paired backups against Standard-PS. (x/y): the method wins on x of the y seeds where both reach the target.

MethodSysAdmin-1024, backups to targetCargoTaxi-1280, backups to targetSysAdmin-4096, final L1 gapCargoTaxi-4500, backups to targetTime per Agent Version, SysAdmin-4096
Standard-PS26.3k17.2k58.9942.1kn/a
GPS-H2239 fewer (21/29)4,715 more (3/25)3.30 higher (0/10)22,993 more (0/3)n/a
GTA-PS260 fewer (22/29)160 fewer (19/29)1.27 lower (7/10)422 fewer (5/10)n/a
GTA-Trust1,248 fewer (28/29)174 more (12/29)6.08 lower (10/10)294 more (4/10)66.8 ms
GTA-Full3,272 fewer (28/29)1,804 more (1/28)10.07 lower (10/10)7,931 more (0/9)81.7 ms

We provide one more table of sparse-dense backend capacity on 1k's:

Table C.3. 1k time against topology memory, GTA-Full (sparse / dense).

EnvironmentWall per seed (s)Factor fillFactor memoryTopology memory per Operator Version
SysAdmin-102469.0 ± 2.8 / 30.5 ± 1.176% to 80% of n²50.4 MB / none102.4 MB / 16.8 MB
CargoTaxi-128025.9 ± 5.6 / 26.3 ± 1.01.0% of n²1.7 MB / none145.8 MB / 26.2 MB

This is crucial to address CargoTaxi-4500's dire clock performance of GTA(+) later.

We will provide the CargoTaxi-4k dense-sparse comparison if requested. We have not ablated such setting.


Elaboration

We address the weaknesses first:

  1. Canonical data. As above. Clear win for the dense backend of GTA-Full. GTA-Trust holds its ground, implying the Spectral Trace does raise transparent efficiency and robustness.

  2. Topological non-stationarity gain. As of our localized mass shift -- identical for all methods -- the GTA family retains proficiency. If GTA-PS can beat the two vanillas on such (of both the predecessor's and ours), GTA+ methods should, and have beated, all the three controls.

  3. CPU against GPU. We agree, and we elaborate why in our metareview. We will provide GPU implementation and ablation on further demand.

  4. Stage-II. As per metareview.

We then address the remaining question 4 of the no-jitter variant.

  • GTA-PS against GTA-Trust is our closest isolation. GTA-Trust is faster on 80/80 seeds and saves 988 ± 137 backups on SysAdmin-1024. On CargoTaxi-1280 it needs 334 ± 164 more backups (2%). The likely causes is of the absorbing structure and a larger tuning-derived η\eta (49 for GTA-PS, 12 for GTA-Trust). The CI provide plausible deniability, but we will invest on understanding this behavior in the future.

The theoretical isolations include the LCB-formalism and the numerical guarantee, but have not treated the terminal/absorbing case.


Post-mortem analysis

This is not of your scope but of the data and method understanding.

  1. Sparse substrate loses in this setting, both on time and peak memory. The reason are threefold: because the correction width threshold ℓ\ell is set too large (we set ℓ=3n/16\ell = 3n/16) -- leading to arithmetic loss; that nn is not large enough (so memory loss); and that branching factor b=5b=5 is still substantial (so frequency loss). Arithmetic evidence is not stated, but from the paper one can compute it. Addressing all three loss can pose to a theoretical break-even region for sparse backend.

We ask the Reviewer to assess the data, its integrity, and our subsequent analysis, with openness. The data reflects the theory neatly, and we hope that its fidelity can merit your score adjustment.

Sincerely,

Authors

Reviewer_33uuOpenReview ↗

The response did not address my concerns; I am keeping my original rating.

Regardless of whether this is actually the case, the paper reads as though it were generated primarily—if not entirely—by a large language model (LLM), including the data and figures; as such, it is fundamentally unreviewable. There is virtually no evidence of human input. While the work may contain some interesting ideas, the entire paper requires a complete rewrite; it is certainly not suitable for publication at this conference. Furthermore, it is impossible to provide specific suggestions for improvement; I recommend studying high-quality papers previously published at ICLR to better understand the expected standards. I apologize if this sounds harsh, but asking reviewers to spend their time evaluating such rough, unpolished material is, in itself, quite inconsiderate.

AuthorsOpenReview ↗

You should at least get a number for such bold claim.

https://opentextdetector.com/?collection=iclr%2F2027&q=52511


This is neatly in the lower quartile (if not the lower 10%) of AI-smelliness for this year.


Not everyone is born white.


With the variation of our wording and latex/markdown formatting, we have put significant efforts to maintain readability and smoothness of our exposition. Our metadata and paper statement reflect all the assists, with due uniformity till now. Of course you dont have anything to back yours "virtually no evidence of human input".

You put 2 for clarity. Have you read back what is the description for such? The other two agree on presentation being understandable.

"Unreviewable"?, "impossible to provide specific suggestions for improvement"? DeuG shows substantial extensive angles. We addressed both yours and theirs already.

"The response did not address my concerns". And without any elaboration for such, you turn counterclockwise for a big d*mn red-herring: committing in AI witch-hunting.


What concern?


Your concern about the tex/seaborn figure? The writer of such code is Claude. The data, color palette, margin, sketch, layout,... is of us. I bet you find such figures in ICLR, at least of your batch, that is not at least scripted by AI. We are approaching one-year anniversary of this new normal.

Your concern about our standard? Have you read the predecessor paper? Make a comparison of such? Writing? Pedagogy? Our paper definitely aces, within your criteria.


GTA-PS is accepted in NeurIPS, by a convenient margin.


And dont let us to use Pangram. On your comments. The result is...

We think you should have accepted that you can not poke more professional criticism. Confidence 4 with such shallow angles.


You are not an expert of LLM writing as you thought.

Reviewer_GKCcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GTA+-PS, an enhanced framework for Graph Topology Augmentation that improves scalability and computational efficiency in prioritized planning under reward changes. It introduces rank-r transition updates, coherent belief-state commits, and a trace-derived spectral trust certificate to reduce computational overhead. The method further provides exact Woodbury updates, convergence guarantees, and a GPU-resident implementation for efficient inference and learning. Experiments across tabular and visual-control tasks demonstrate improved learning persistence compared with standard baselines.

Critical strengths
  1. Strong theoretical foundation: The paper provides a solid theoretical foundation, including exact Woodbury-based operator updates, a rigorous connection between the trace-derived spectral trust measure and discounted signal variance, and convergence guarantees for stochastic priority selection. The formal propositions and theorems, supported by mathematical derivations, strengthen the method’s rigor and provide theoretical justification for its correctness and convergence under stated assumptions.

  2. Clarity: The proposed method is presented clearly, intuitively, and in a well-structured manner, making the overall framework, its key components, and their interactions easy to follow and understand.

Critical weaknesses
  1. Limited evaluation and scalability evidence: Although the paper evaluates the proposed method on SysAdmin, CargoTaxi, Pong, and VizDoom, the diversity and scale of the environments may be insufficient to establish its generalizability and scalability across more complex settings. In particular, the evaluation could provide stronger evidence of computational efficiency and scalability as the state space, transition complexity, and frequency of model updates increase. More extensive experiments in larger-scale environments, along with runtime, memory, and computational-cost comparisons against the baseline, would better substantiate the claimed scalability improvements.

  2. The paper would benefit from a more comprehensive analysis of the computational costs of GTA+-PS compared with the baseline methods. Although the proposed framework shows promising improvements in learning persistence, the method's overhead may limit its practical applicability. This is particularly important given the limited range of evaluated environments and the need to substantiate the claimed scalability benefits. The authors should report and compare metrics such as training time, GPU hours and memory consumption across GTA+-PS and its baselines, particularly as state-space size increases. Such an analysis would clarify whether the observed performance gains justify the additional computational resources and establish the practical benefits of the proposed approach.

Questions for authors

Please refer to the Weaknesses section, as they are also my concerns and questions for the authors. Any additional clarifications would be welcome.

1 comment
AuthorsOpenReview ↗

Official Rebuttal of GKCs's Review by Authors

We thank Reviewer GKCc for the detailed and constructive review. We appreaciate the praise for formalism and exposition quality.

This reply gives the scale and cost data first. It then answers the two weaknesses.

We was intending to scale stage-I to 32k-state environments, but was cut by our dire free time.


Requested Data

Table G.1. Wall time per seed (s) from 1k to 4k states, with growth.

MethodSysAdmin, 1,024 to 4,096GrowthCargoTaxi, 1,280 to 4,500Growth
Standard-PS66.1 to 236.33.58x7.1 to 17.22.41x
GPS-H260.3 to 220.03.65x10.5 to 33.83.22x
GTA-PS82.7 to 759.89.18x181.1 to 3,588.919.82x
GTA-Trust52.5 to 584.111.12x100.9 to 1,567.215.54x
GTA-Full30.5 to 177.35.81x25.9 to 311.512.01x

The state count grows 4.0x on SysAdmin and 3.52x on CargoTaxi.

Table G.2. Compute for 10 seeds at 4k states.

MethodDeviceSysAdmin-4096CargoTaxi-4500
Standard-PSCPU39.4 min2.9 min
GPS-H2CPU36.7 min5.6 min
GTA-PSA1002.11 h9.97 h
GTA-TrustA1001.62 h4.35 h
GTA-FullA10029.5 min51.9 min

Each GTA run uses one A100 (40GB VRAM) plus 4 CPU threads. The topology module of GTA-Full holds 16.8 MB on SysAdmin-1024 (dense) and 145.8 MB on CargoTaxi-1280 (sparse). A dense module needs 16n² bytes: 268 MB at 4,096 states.

Table G.3. Backups to target from 1k to 4k states, CargoTaxi.

MethodCargoTaxi, 1,280 to 4,500Growth
Standard-PS17.2k to 42.1k2.45x
GPS-H220.8k to 51.8k (3 of 10 seeds)2.49x
GTA-PS17.1k to 41.7k2.44x
GTA-Trust17.4k to 42.4k2.44x
GTA-Full18.6k to 48.4k (9 of 10 seeds)2.60x

No SysAdmin-4096 run reaches the target within 70,000 backups. Their remaining error is around 0.15x, so we regard the error scale for planning precision (see 33uu's).

Table G.4. Peak memory of GTA-Full, with Standard-PS for reference.

EnvironmentTopology memory per Operator VersionCoherent Commit volumePeak host memoryPeak host memory, Standard-PS
SysAdmin-1024 (dense)16.8 MB50.4 MB2.05 GB0.78 GB
CargoTaxi-1280 (sparse)145.8 MB303.1 MB4.08 GB0.77 GB
SysAdmin-4096 (dense)268.5 MB805.5 MB3.38 GB0.91 GB
CargoTaxi-4500 (sparse)1,763.6 MB3,707.8 MB6.31 GB0.89 GB

Elaboration

  1. Scale and diversity. Read "scaling" against the predecessor GTA-PS.
  • The prior work evaluation lays on simple PoC, CPU-resident substrate, scale at most 200 states.
  • This paper runs up to 4,500 states. We also propose visual tasks with one-seed result; and tasks with irregular topology: a recurrent, hyper-connected bayesian (b≈22b \approx 22) network (SysAdmin) and an absorbing transport graph (CargoTaxi).
  • We incorporate exogenous transition drift: one transition event per 100 backups, approximately 718 events per 4k run.
  • GTA-Full grows more slowly than GTA-PS on the two tasks. It grows faster than Standard-PS on the two tasks.
  • See the backups to target in Table G.3: all methods grow 2.4x to 2.6x on CargoTaxi for 3.52x more states.
  1. Cost against gain.
  • On SysAdmin-4096 the gain justifies the cost: GTA-Full uses the least wall time and reaches the lowest final gap, in 0.49 GPU-hours against 2.11 for GTA-PS.
  • On CargoTaxi-4500 it does not: Standard-PS is 18 times faster and needs fewer backups.
  • See the memory growth in Table G.4: the topology memory grows 16x on SysAdmin and 12x on CargoTaxi, close to the square of the state growth.

Our revision will states these limits and adds the four tables.

  1. Stage II. As per metareview. Though given that one seed, the GTA-augmented sibling beats vanilla one -- with side context of arduous engineering effort.

We hope that our improvised data will assist you on your assessment about scalability. We thus yearn for a review of your score verdict.

Sincerely,

Authors of Submission52511

Reviewer_DeuGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper extends Graph Topology Augmentation for Prioritized Sweeping (GTA-PS) to nonstationary reinforcement learning. It combines batched low-rank operator maintenance, a trace-derived spectral trust statistic, randomized priority selection, consistent publication of operator and queue updates, and GPU/sparse implementations. The intended contribution is to reduce the cost of maintaining topology-aware planning priorities while improving adaptation after environmental changes. Evaluation covers tabular tasks and visual control tasks with pretrained representations. Central questions are whether the proposed maintenance remains inexpensive under realistic changes and whether the additional scheduling doubling execution mechanisms improve recovery beyond existing GTA-PS and simpler alternatives.

Critical strengths
  1. The paper addresses a consequential computational bottleneck. Maintaining topology-aware priorities as the transition model and policy change can be expensive. Combining batched updates with consistent publication of operators, statistics, and priorities is a potentially useful systems contribution. Its value would be substantial if the full maintenance cost and resulting planning benefits were demonstrated under controlled transition changes.

  2. The evaluation protocol includes useful elements. The tabular experiments specify ten paired seeds, and the stated timing boundary includes initialization, queue management, topology maintenance, and backups. The inclusion of visual tasks also explores the method beyond tabular settings. These are useful foundations, although protocol descriptions do not replace completed measurements and visual evaluation requires independent training repetitions.

Critical weaknesses
  1. The empirical evidence does not yet establish the claimed computational and adaptation benefits. Table 2 marks the GTA-PS variants’ recovery times as projected results pending subsequent experiments, while Appendix G remains a placeholder despite being referenced for throughput, timing breakdowns, and numerical behavior. These materials cannot substantiate achieved runtime or throughput gains. Figure 3 is not explicitly marked as projected, so its status should be clarified rather than presumed. Main controlled changes are reward shifts (Appendix F.4), which do not directly isolate the transition-maintenance optimization. The comparisons simultaneously change maintenance, spectral statistics, consistency mechanisms, and execution backends; distinguishing versions with and without randomization does not isolate the remaining contributions. CPU NumPy baselines versus JAX/GPU implementations further confound algorithmic and implementation gains. Stage II uses only one training run per method (Appendix F.3): multiple evaluation episodes characterize that trained policy, not variation across training runs. Together, these defects prevent a reliable attribution of the reported benefits to the proposed mechanisms.

  2. The theoretical guarantees do not yet cover the full algorithm as claimed. A small number of changed transition rows need not imply a small update width for both maintained directions. Appendix B.2 allows the incoming update to depend on the number of affected destination columns; even one changed outgoing row can affect many such columns. The complete complexity therefore needs separate bounds for the two directions. Theorem 1 also establishes selection conditional on recurring candidate inclusion, whereas the thresholded candidate construction does not establish that every state requiring an update enters the candidate set sufficiently often. Its positive-temperature assumption does not establish convergence of the deterministic variant. In addition, the variance result concerns centered isotropic signals, whereas the scheduler uses structured absolute Bellman residuals and mixes propagation directions. This supports a design motivation but does not by itself establish calibrated trust or a statistical lower confidence bound. These gaps require explicit assumptions, additional proofs, or narrower claims.

Questions for authors
  1. Completed evidence: Which entries and curves are based on completed experiments, and which are projections? Please provide the measured replacements for Table 2, the results intended for Appendix G, and per-seed data and plotting scripts for Figures 3–4. Does Figure 3 summarize ten completed paired runs? For threshold-based comparisons, please report runs that fail to reach the target within budget as censored rather than infer an exact speedup from an unreached target.

  2. Attribution and fairness: Can you provide comparisons on the same backend and resources that isolate batched maintenance, the trace statistic, consistent publication, and randomization? The most informative maintenance comparison would directly control changed transition rows and affected destinations and compare sequential updates, batched updates, and full recomputation. Please also specify the predecessor GTA-PS implementation and explain the GPS-H2 modifications.

  3. Complexity and numerical scope: What are the separate incoming and outgoing update widths, measured costs, rebuild frequencies, factorization fill, and peak memory? How do these change when a single modified row affects many destinations or a shared policy update changes many states? Which numerical guarantees apply to every published version, and which apply only at scheduled checks? Sparse input and sublinear correction width alone do not establish unconditional linear storage.

  4. Convergence: What guarantees recurrent candidate inclusion for every state with a nonzero residual? With a positive threshold, what residual tolerance is guaranteed? What additional argument covers zero-temperature deterministic selection? Please distinguish fixed tabular convergence from claims concerning changing environments.

  5. Spectral trust and sampling: Can the trace statistic predict propagation variance or planning benefit for the actual absolute residual signals and both propagation directions? What supports the lower-confidence-bound interpretation? For Appendix E.4, please specify the target distribution and batch estimator: first-draw probabilities generally differ from inclusion probabilities under sampling without replacement. If the weights are heuristic, please identify them as such. Fixed-temperature and residual-normalized controls, including reward-rescaling tests, would help determine whether the spectral coupling itself is useful.

  6. Visual experiments and reproducibility: How are full Pong observations mapped to at most 512 planning states, and how is temporal information represented? A token vocabulary of size 512 does not itself define 512 full-frame states. Please clarify pretrained-representation costs within the stated budget, provide complete configurations and run manifests. Existing episode-level intervals should be explicitly conditional on single trained policy.

3 comments
AuthorsOpenReview ↗

Official Rebuttal for DeuG's Review by Authors

We thank Reviewer DeuG for a thorough, expansive review and appreciate your approval of our system design.

We sharply value your remark on both update widths. It is correct; we discovered it in October. GTA-PS softmax policy deeply worsens this symptom, and Table D.1 measures it.

Answered in metareview and earlier replies (Tables C.1–C.3, G.1–G.4): reconciled Table 2 and Appendix G, transition shift, CPU against GPU, single Stage-II run, backend limit, and landmark scope.


Requested Data

Table D.1. Changes per version (m: measured in mean; s: structural count in one increment).

Environmentbπ rowsp entriesOut / in columns
FourRooms-1043.215.6016.3 / 28.7 m
SysAdmin-102423.31640.16172 / 906 s
CargoTaxi-12804.5330.1636 / 83 s

Elaboration

  1. Completed evidence (Q1). All Stage-I entries and curves come from completed runs: 30 paired seeds at 1k, 10 at 4k. (x/y) in Table C.2 is the censoring record: runs below target enter no speedup. GPS-H2 reaches target on 3 of 10 CargoTaxi-4500 seeds; no SysAdmin-4096 run does, so that column reports final gap.

  2. Attribution (Q2). Compare GTA-PS, GTA-Trust and GTA-Full on one backend.

    • -Trust changes trust statistic; -Full adds jitter and a wider version (4 lanes × 2 bursts gives 64 state backups per version). Both steps also retune parameters.
    • Batch consumption and consistent rescore are common (qcf=8q_{cf}=8) to all five method. This follows GTA-PS and GPS-H2's (extra-)formalism. GTA-PS runs on the same JAX backend with exact SLEM every 50 versions. GPS-H2 ranks by ξ + ωV, ω = 0.0157, partition 8.

A per-component ablation (sequential vs. batched vs. full recomputation) exceeds our rebuttal budget and probe data; we defer it to revision.

  1. Update widths and storage (Q3). We agree and welcome your instinct.

    • Outgoing width is changed-row count r; incoming width is r + |supp(δ)|, as each changed row moves the incoming diagonal at its successors. A Coherent Commit fires once changed incoming rows exceed ℓ = 3n/16; it counts incoming rows only, as incoming width bounds outgoing width (Table D.1).
    • Incoming width first grows as r·b, then saturates near O(n), as buffering aggregates deltas sharing a destination column. A FourRooms-104 probe (our tuning fixture, b ∈ {2, 3, 4}) measures both widths (Table D.1).
    • On sparse backend, stored correction width k reaches n in run records, so storage φ + nk + k² is quadratic; dense backend stores no correction and holds four n × n matrices. Factor fill is 76–84% of n² on SysAdmin, 1.0% and 0.47% on CargoTaxi. Woodbury updates are exact per version in exact arithmetic; Inverse Certificate (5 × 10⁻⁴) holds at scheduled checks. Inbound above several multiples of ℓ within one phase's shock is processed as even, fixed-size chunked commits.
  2. Convergence (Q4). Our proof concerns tabular, stationary value iteration after the last model change, with no quantitative recovery guarantee. Theorem 1 covers stochastic selection of GTA-Full. For deterministic GTA-Trust, the predecessor's GTA-PS proof applies directly, as it needs only a topology-weight bound, β1(1 + β2)/η < 1. Tuned ratios c are 0.70 (GTA-PS) and 0.93 (GTA-Trust), so each pick holds at least 1 − c of the largest residual. Every state with ξ > ε_ξ is a candidate at each refresh (Appendix E.3). With ε_ξ > 0, value lies within ε_ξ/(1 − γ) of V*; all our runs set ε_ξ = 0.

  3. On trust (Q5). Proposition 2 averages over an isotropic ensemble; trust weight keeps its LCB form (base minus scaled trust) within that scope.

  4. Replay weights (Q6). E.4 is a design extension; we claim no theoretical validity yet.

    • Its target distribution is joint-logit softmax; its batch, a Gumbel top-k sample without replacement. Weights use first-draw probability π, as PER does.
    • For batch size k, inclusion probability q obeys 1-(1-π)^k ≤ q ≤ kπ/(1-(k-1)π_max), so both agree within a factor 1 + O(kπ_max). Batch normalization removes constant k. PER itself uses partial correction (exponent below 1), so its weights are biased too. For reward scale, we plan to normalize scores as advantages over a positive value scale (max |V|), making them unitless.
  5. Pong states (Question 6). Latent tabular states are not "immovable tokens". Each full observation maps to one centroid (a.k.a "code"), and clustering moves via VaST's statistical learning. A learned 9-bit code for VaST give at most 512 latent states, and some might not be active. 9 is a vibe choice for long runs, we have not ablated it.


We will upload curves and tables later in revision. Our supplement expects ~200 more records, plotting scripts, configurations and run manifests. Each Stage-II interval is conditional on one trained policy, as captions state.

We hope that our direction depict your concerns. We yearn for more discussion from you, as your insights are really impactful.

Sincerely,

Authors

Reviewer_DeuGOpenReview ↗

Response to the Authors

Thank you for the additional clarifications. The following four points still require clarification to assess whether the new evidence adequately supports the paper’s core contributions and conclusions.

  1. Reliability of the reported results. Regarding the repeated experiments mentioned in your response, please provide the random seed lists used for the reported experiments. Please confirm that the seeds were randomly selected and documented to enable reproducibility, and that neither seeds nor runs were selected based on the quality of their results.

  2. Ablation evidence for the core innovations. My concern is whether the newly introduced mechanisms themselves have been validated through ablation studies. Table 2 in the original manuscript compares the previous GTA-PS method, the deterministic version of the new method, and the full version. However, this comparison alone does not isolate the respective contributions of batch maintenance, spectral trust, and stochastic perturbations. Please identify which ablations in your response directly test these core innovations. In particular, do they establish whether spectral trust and its coupling with stochastic scheduling provide additional benefits when all other conditions are held constant?

  3. Comparisons on the same GPU. Please provide measured comparative results for Standard-PS, GPS-H2, and the GTA variants using the same GPU, comparable computational resources, and a consistent timing protocol. This comparison is important to the evaluation.

  4. The evidential basis for the conclusions. I appreciate the more accurate and updated data, but they raise a question about the basis for the original conclusions. Table 2 in the original manuscript labels the runtimes for GTA-PS and both new variants as projected values, yet Section 7.1 already concludes that the method delivers consistent recovery and throughput gains across tabular and visual tasks. Which completed experiments supported these empirical performance claims when the key runtime results were still projections? My concern is how conclusions of this scope could have been reached before the corresponding empirical evidence was available.

These clarifications will help me assess whether the core innovations have been directly validated and whether the conclusions are commensurate with the empirical evidence. I look forward to your further clarification and hope it will provide a more convincing basis for assessing the paper’s contributions.

AuthorsOpenReview ↗

Clarification and Sincerity to the Reviewer

We thank Reviewer DeuG for a prompt, focused follow-up.

Your comment arrived three hours after our revision went live (17/11, 11:18 UTC). That revision answers points 1–3, and we cite its sections below. Point 4 we concede.


1. Seeds. We are keenly aware about the "Seed Tuning" misconduct trope. Thus, we craft the seed "Separator" prior to any method implementation.

  • Train, Validation, Test and Optuna-assigned partitions fix every seed in code before any run (Appendix F.3). Stage-I uses Test seeds 200–229 at 1k and 200–209 at 4k; tuning uses its own partition.
  • Moreover, each Actor and action query possess their own sub-seed. Such follow index order: the main seed poses a pseudorandom stream and a sub-seed Generator. Thus, the profile retains plausible randomness and grants exact replay. Every method–environment cell reports all its seeds, 400 records in total (300 at 1k, 100 at 4k). All five methods of one environment share one transition-event log, with identical SHA-256 digests per seed. Stage-II runs one Test seed (index 0) per method.

Trace for all the containers can/might be found on our supplement's manifest.

30 seeds might still not acquire distinguished significance. Sone CIs give plausible deniability on qualitative testing.

2. Ablation. Table C.2 forms a nested ladder on one JAX backend, one A100, one seed list and one event log. Each arm runs its own 30-trial search under one shared protocol (Appendix F.2).

  • GTA-PS → GTA-Trust replaces SLEM tracking with trace trust -- that is the only swap. Tuned values differ, of course. GTA-Trust runs faster on all 80 paired seeds across four tasks. It saves 988 ± 137 backups on SysAdmin-1024 and costs 334 ± 164 (2%) on CargoTaxi-1280. An O(n) diagonal-sum operation should be faster than a super-quadratic routine, right.
  • GTA-Trust → GTA-Full adds variance-scaled jitter together with a wider version. (Actors count) 4 x (Burst persistence) 2 x 8 (states/dequeue) = 64 states (backed per Agent Version). On consumption quality, SysAdmin also gains: 3,272 against 1,248 fewer backups at 1k, and a final L1 gap 10.07 against 6.08 lower at 4k. CargoTaxi loses: 1,804 more backups at 1k.

In short, Trust stands isolated on its own. Jitter coupling stands isolated jointly with version width, since no trust-free jitter arm ran. Batch maintenance is common to all three GTA arms, with dequeue width of 8.

3. Same GPU. Section 6.1 and Table G.2 hold it. All 400 runs executed on one Colab host class. A total of 12 logical cores, out of that, 6 on main computation service. Accelerator is always A100-SXM4-40GB, one Learner Job per container, four threads for agent instance, and one timing boundary. Standard-PS and GPS-H2 leave that A100 idle by design: each backup pops one state and needs its predecessor's result, so a GPU port pays obnoxious host–device round trip per backup. Table G.2 charges GTA its GPU hours. Besides, Backups to target (Table C.2) count no hardware at all, and they carry our planning-efficiency claim.

4. Original conclusions. We concede this point in full.

  • On 15/9, our predecessor infrastruture (being Kaggle with P100) cuts their service. We thus have to re-design, re-tune and adjust our enumeration for all GTA methods. We have run mock 1k/4k qualification run (one seed of 199), thus the Claude handoff's projection. The conclusion (Section 7.1) is conversed in the same rushed fashion, and the abstract, written up from August, was not revised. No full-campaign (of our satisfaction) 1k or 4k GTA run had finished. They could have required 2 more days of Colab GPU service -- derived from our revision. Between 15 and 27 of September we also run Stage-II, with only one seed available, and a full rebound padding cover all lifecycle.
  • We continue evaluation post draft deadline. The tuning selections landed 11/10, and our serious campaign ran 17/10 to 11/11. Most of that time is again extensive runtime engineering and obnoxious producer/consumer synergy. For such, a complete re-run for both stage-I and stage-II is a must, to reflect all the apparatus.
  • Our September conclusion thus was stray of quantitation, and we firmly retract it. Our revision rebuilds every Stage-I claim on completed runs and reports its losses openly: Standard-PS beats GTA-Full on CargoTaxi at both sizes. Stage-II follows an "experimental" label at one seed per method. On SysAdmin, GTA-Full plans best in least wall time at both sizes.

This is such long story for our remaining time budget. If allowed, we can and will welcome to further investigation past 18/11.

We yearn for your re-assessment of our score, with our attentiveness to details, and against this 17/11 revision.

Sincerely,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Metareview of Submission52511's status quo and revision by Authors

We thank the three Reviewers for their careful and substantial assessment. We apologize for the late response, due to various side conflicts.

We appreciate that all three reviewers raise unaminity for the theoretical side, and of our settings.

Our prior premise for scaling GTA-PS (inferred), is to: (1) provide a plug-in universal application for algorithms, which some protocols can be hugely accelerated given adept prioritization, and (2) eventually establish the prioritization scaling axis, where up to a region, paying more compute for prioritization gains more efficiency for most to all objectives -- including wall-clock competency.

We have completed a broad incremental revision (17/11). We detail more know-how, smoothen terminology load, as well as incorporate the corrected data. The stage-II data is marked as experimental, and with narrower decision count -- again by our limited time. The fresh supplement also reflects such changes. Full-range run will be available for at most tomorrow.


We state six points about the submission before the individual replies.

  1. We prepared the submission in haste and with limited resources. The "projected' mark, upon our discover (14/11), is due to rash conversion one hour before deadline, leading the LLM (being DeepSeek) load the other handoff (of Claude) containing runtime projection. We affirm that this is an accident of ours. Broken data is unacceptable for (high-tier) academics. The only recoup we have is to confess: this is an LLM-involved accident. We hope that the newer data can compensate for such.

We do not demand acknowledgement, but we will stand against any escalation, especially personal one (15/11). We fidelitously stated the assistance of LLMs on draft-making, and have always been consistent with such.

  1. As the Reviewers pointed out, the submission omitted four crucial notes:
  • Appendix G was totally empty. We had not, at least, cleared stray references prior to submission.
  • There are the localized transition shift -- 100 backups -- on two adjacent states. The detail unfortunately slipped in the deadline draft, and are now present in the revision (17/11).
  • Stage-II learning indeed ran on one seed only (and also with limited engineering). The straight lines are just linear connectors between our snapshot marks -- does not reflect inner reward landscape. Details of checkpoint marks is in Appendix I (17/11).
  • The moat of GTA-Full on stage-II lies on batched inference. Our GTA-Full setup consumes 8x the data and ~4x the speedup, per Agent Version. The vector samples -- thanks to stochasticity -- encase diverse variations. As inference can horizontally scale on comfort from GPU residence, there is region when it beats all methods convincingly. We provide the new data (above).
  1. The predecessor GTA-PS paper contains extensive, recently revised (27/10) materials. The page/character limit adversed restatement.

  2. The reason Standard-PS and GPS-H2 keeps their CPU usage is: to be faithful to their sources, and bringing them to GPU actually harms speed -- because of obnoxious inter-device round-trip --- here assume environment simulation logic reside in CPU. They are not horizontally scalable nonetheless.

  3. The GTA method, though described as a plug-and-play, meets considerable friction in practical deployment. All of this lies not on the math, but on the training dynamics of the host (as in VaST per 6.) The submission did not report this friction adequately. Reviewer GKCc correctly raised this.

  4. We recently found a numerical ceiling for sparse storage, lying not on fill, but on connectivity of the MDP. Sub-quadratic LU storage hold requires supressed connectivity as a necessity (~1/11). SysAdmin, for example, does not satisfy due to its Bayesian, high-branching nature. Reviewer DeuG correctly raised this.

  5. The combination of VaST with our method is exceptionally difficult to engineer. The VaST formulation lacks sharp state separation, and its centroids reconfigure during learning. This reveals clearly in Atari Pong. A sharper plugged planning might and is surely harm the refiguring. Details on our Appendix G.

  6. Targeting for the desired break-even for prioritization spend and gain, especially for Deep RL, is arduous. Reviewer GKCc anticipated this. Details also on Appendix G. Hope that future work can help.

As these issues are posed, our subsequent rebuttal defends will focus primarily the tabular results, which poses clear success. The deep-learning study needs broad rework of ours. That work exceeds the remaining rebuttal budget. We thus hope to narrow our practicality, nearer to the simpler region when our math do translate.


We ask the reviewers to read the individual replies with these, plus the new 17/11 version, in mind. We again appreciate our Reviewers for their evocations.

xx/yy is date, in UTC

Sincerely,

Authors of Submission52511

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.