acceptodds
Under review as a conference paper at ICLR 2027

cuBCR: Batched GPU Evaluation for Counterfactual Responsibility

Abstract

Multi-agent systems coordinate multiple AI agents to complete tasks, but attributing responsibility becomes challenging when collaborative execution fails. In time-sensitive services, slow root-cause analysis delays mitigation and degrades service quality. Explaining team failure requires analyzing how individual actions constrain the overall system's ability to recover. In this paper, we formalize responsibility attribution in finite stochastic models: holding a subset of agent policies fixed, where a policy specifies how an agent chooses its actions, we evaluate whether the remaining agents can still coordinate an intervention to prevent task failure before a deadline. If the failure is avoided through intervention, we use Shapley values to allocate responsibility among agents who contributed to the failure. Nevertheless, exhaustively evaluating every coalition of agent policies requires an exponentially large number of counterfactual simulations. We tackle this challenge with CUDA-accelerated Batched Counterfactual Responsibility (cuBCR), which accelerates the process through Monte Carlo sampling and GPU parallelism. Sampling curbs the combinatorial explosion in the number of required counterfactual checks, while batched GPU kernels evaluate the recovery simulations concurrently. On an event-driven ticketing benchmark, cuBCR achieves a median 233× speedup over the CPU-based exhaustive failure attribution across 36 workflow tasks. By significantly reducing attribution latency, cuBCR makes iterative counterfactual analysis practical for production debugging, enabling system designers to evaluate how individual policy revisions alter multi-agent failure dynamics quickly.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
GPU acceleration, multi-agent systems, causal inference, counterfactual responsibility, failure attribution, Shapley values

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BvDQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

cuBCR allocates avoidable failure risk using Shapley values in finite stochastic multi-agent models. It combines paired-permutation sampling with batched GPU backward induction for coalition recovery queries. The evaluation separates sampling savings from hardware acceleration and checks numerical results against exhaustive calculations and independent verification tools.

Critical strengths

The computational decomposition is well explained. Sharing a transition model across many policy-restriction queries is a sensible way to exploit GPU parallelism.

The validation is stronger than a CPU/GPU agreement check alone. Identical-query comparisons, batching ablations, reachability-pruning controls, and checks against PRISM-games and Storm provide useful support for correctness and performance.

Critical weaknesses

The headline 233× speedup combines GPU execution with savings from established permutation sampling. The paper appropriately separates these effects, but does not compare against stronger Shapley estimators mentioned in the related work. Equal-error comparisons would clarify how competitive the complete attribution pipeline is.

All reported attribution timings exclude model construction and engine initialization, which the paper notes can dominate small jobs. This matters for the proposed interactive debugging use case, where a policy or system revision may require rebuilding parts of the computation. Cold-start and incremental-update measurements would establish when the acceleration translates into practical savings.

The demonstrated scope is restricted. The implementation permits at most three simultaneously active agents and 64 joint-action rows per state, and assumes an explicit transition model. These limits make the tested workloads manageable but leave denser concurrent systems unexplored. The deployment discussion should remain tied to these assumptions.

Questions for authors
  1. How does paired-permutation sampling compare with alternative estimators under the same attribution-error target?
  2. What are cold-start latency and update costs after changing policies or transition probabilities?
  3. How do memory use and runtime grow as the number of simultaneously active agents increases?
2 comments
AuthorsOpenReview ↗

Response to BvDQ: estimator comparisons, complete costs, and scaling

Thank you for recognizing the computational design and validation, and for suggesting comparisons with stronger estimators and measurements of complete analysis costs and scaling. Below, we respond to your comments and explain the changes made in the revised paper.

Weakness 1 / Question 1: Alternative estimators at the same attribution-error target.

The new comparisons show that better sampling methods can directly benefit from cuBCR's GPU engine. Leverage SHAP, 2-ADD-SHAP, and Stratified SVARM run on the unchanged engine across seven whole-agent TicketBench tasks with 16–28 agents. All 162 runs with exhaustive references reproduce their table-based estimates bit for bit. Thus, the engine's value extends beyond the paired-permutation estimator used in the original experiments.

Appendix C.3 compares the number of distinct coalition evaluations needed at the same error target. Each query evaluates recovery after fixing a set of agents' policies. On 12 TicketBench games with 20 agents, the median budgets are:

Error targetPaired permutationsLeverage SHAP2-ADD-SHAPStratified SVARM
5% of total attributable risk848519533654
1% of total attributable risk15,3335,1493,7938,760

Total attributable risk is the failure risk that coordinated recovery can avoid. These empirical budgets use exact references and 30-seed error curves, interpolated where the observed 95th percentile of the largest error across agents reaches the target. They measure empirical error, without online coverage guarantees.

Table 5 measures warm attribution time using budgets calibrated against exhaustive references on separate seeds, then frozen for 30 held-out seeds per game. We report the 1% target because all four methods meet the empirical 95th-percentile maximum-error criterion on all twelve games (at 5%, paired permutations pass 6/12). Leverage SHAP and Stratified SVARM are faster on every game, taking 40% and 42% of paired time (median per-game ratios). For 2-ADD-SHAP, the group median is slower at 961 states and faster at 9,961 states.

Online stopping does not require exhaustive Shapley references to decide when to stop. Across 12 games, 30 seeds, and 2 tolerances, all 720 runs reach tolerance; every agent's error against the FP64 reference lies within its half-width. Median warm times are 0.224 s at 5% and 1.193 s at 1% (medians of game medians). Containment is empirical; coverage remains conditional on Theorem 1 (Appendix C.3).

Weakness 2 / Question 2: Cold start and revision costs.

Complete first attribution takes 0.438–1.42 s across 20 primary configurations. These measurements include process startup, model construction, initialization, upload, and attribution. Complete policy re-attribution takes 0.074–1.06 s with the process and model structure retained.

For the 28-agent example, first attribution takes 0.438 s, policy re-attribution 0.112 s, and a transition-probability revision 0.106 s when the possible transitions remain unchanged. Policy changes reuse the transition structure and refresh reduction weights. Probability changes upload new values; changes to the possible transitions require rebuilding.

Overcooked shows why retaining the model matters. First attribution takes 13.2 s, and rebuilding in a warm process still takes 12.7 s. Policy re-attribution with the model retained takes 1.69 s. The update interface was added for the rebuttal around unchanged solve kernels. All 171 complete sampler comparisons match fresh builds bit for bit. Appendix C.2 gives the timing breakdown and reuse requirements.

Weakness 3 / Question 3: Memory and runtime with more agents acting at once.

Appendix C.4 separates state count from concurrent actions. At a batch of 65,536 coalitions and horizon 50, tenfold state growth raises runtime about tenfold. Across 8–64 action combinations per state, memory is 0.975–0.984 GiB at about 1,000 states and 9.79–9.88 GiB at about 10,000, dominated by value buffers. Increasing action combinations from 8 to 64 raises runtime 13.4–16.7×. For a fixed number of actions per agent, combinations grow exponentially with the number of agents acting at the same step.

The GPU code used for the main experiments supports up to three agents acting at the same step and up to 64 action combinations per state. For additional scaling tests, we used a separate extended CUDA kernel. With 993 states and a batch of 65,536 coalitions, it takes 13.2 s for 256 action combinations per state and 26 s for 512. These tests use synthetic models with explicitly listed transitions.

Dear Reviewer BvDQ,

Thank you for suggesting a comparison at the same attribution-error target. All four estimators met the 1% target on held-out runs across twelve TicketBench games. Leverage SHAP took 40% of the paired sampler’s warm runtime on a typical game, and Stratified SVARM took 42% (median per-game ratios, Table 5). Appendices C.2 and C.4 also report cold-start and update costs and show how runtime grows as more agents act at once. Could you let us know whether these results address the concerns behind your initial rating? If they do, we hope you will consider raising it.

Best regards, The Authors

Reviewer_mvDwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies counterfactual responsibility attribution in finite-horizon stochastic multi-agent systems. It fixes the original policies of a subset of agents, allows the remaining agents to perform recovery, and uses Shapley values to allocate the difference between the original failure risk and the minimum failure risk achievable through recovery. The authors propose cuBCR, which combines permutation sampling with reverse pairing, confidence-sequence stopping, and batched GPU Bellman updates to reduce the cost of coalition-value computation. Experiments on TicketBench and a restricted Overcooked setting evaluate the runtime and scaling of this system. The contribution is primarily systems-oriented.

Critical strengths
  1. The implementation reuses shared computation across coalition queries. Different coalitions share transition probabilities, action spaces, and original policies. The authors organize multiple recovery calculations into GPU batches and support both whole-agent and decision-time attribution, addressing the repeated recovery computations caused by the growing number of coalitions.
  2. The experiments separate the main sources of runtime improvement. In addition to reporting the overall speedup of sampled GPU over exhaustive CPU, the authors compare CPU and GPU on identical query batches and evaluate sampling, GPU execution, and batching separately. These comparisons distinguish the reduction in computational work from the execution gains obtained on the GPU.
Critical weaknesses

The technical contribution is primarily a systems integration, with limited algorithmic novelty. The paper combines existing Shapley-based responsibility attribution and permutation sampling with batched GPU computation and adaptive stopping. The experiments show runtime gains and larger tested problem sizes, but do not introduce a new attribution definition or sampling estimator. The incremental contribution over prior work is therefore mainly engineering, and the paper provides limited evidence about the value of this integration beyond the tested benchmarks.

Questions for authors
  1. Accuracy parameters: What value of the confidence parameter α is used in the main experiments, and what motivates choosing ε = Υ̃/20 as the stopping tolerance? A sensitivity analysis on a few representative settings would help clarify the effect of this choice on runtime and estimation error.
  2. Code release: Do the authors plan to release the cuBCR implementation and the materials needed to reproduce the reported experiments?
2 comments
AuthorsOpenReview ↗

Response to mvDw: reusable GPU execution, useful resolution, and reproducibility

Thank you for your careful review and for asking us to clarify the technical contribution, tolerance choice, and reproducibility. Below, we respond to your comments and explain the changes made in the revised paper.

Weakness 1: Technical contribution and value of the integration.

cuBCR's contribution is a reusable GPU engine for repeated counterfactual recovery calculations. Coalition queries share the same sparse transition model and preprocessing while fixing different agents' policies. The engine evaluates these queries together and supports both whole-agent and decision-time attribution. The same execution design also serves alternative Shapley estimators, as the new comparisons demonstrate.

With queries and batch schedules fixed, GPU speedup is 1.4–5.1×. The same-GPU batching ablation gives 136–1836× over one coalition per launch. These controls separate execution gains from the 233× headline, which also includes sampling savings.

Section 5.1 adds a generic PyTorch FP64 GPU baseline. For warm solve batches of 4,096–65,536 coalitions, cuBCR is 2.49–5.37× faster than the best eager-mode implementation we tested. PyTorch used an 8 GiB memory cap; cuBCR could use the full device. On one 9,961-state model, cuBCR is 4.70× faster at batch 16,384 with PyTorch's cap raised to 64 GiB, and 5.54× faster at batch 65,536 with a 32 GiB cap.

The new experiments show that the engine remains useful when the estimator changes. We test Leverage SHAP, 2-ADD-SHAP, and Stratified SVARM on the unchanged GPU binary across seven whole-agent TicketBench tasks with 16–28 agents. On the reference-available subset with at most 24 agents, all 162 alternative-estimator runs match their table-based estimates bit for bit. Across the three alternatives and two paired controls, median GPU time per coalition differs by at most 6.6% (median difference 1.1%) when approximately the same number of distinct coalitions is evaluated. These comparisons use three-seed medians after warm-up.

The engine also supports faster attribution at a common empirical error target. Table 5 reports warm attribution totals on twelve 20-agent TicketBench games. Budgets were calibrated on separate seeds and frozen before evaluation on 30 held-out seeds per game. At a target of 1% of total attributable risk, all four methods meet the empirical 95th-percentile maximum-error criterion on every game. Leverage SHAP and Stratified SVARM take 40% and 42% of the paired sampler's time, respectively, measured by median per-game ratios. These totals exclude model construction, initialization, and warm-up; fixed budgets provide no online stopping guarantee (Appendix C.3).

The full latency measurements show how the engine supports repeated policy analysis. Across 20 primary configurations, complete first attribution takes 0.438–1.42 s, and warm policy re-attribution takes 0.074–1.06 s. Overcooked makes model reuse particularly clear: rebuilding the model in an already warm process still takes 12.7 s, whereas retaining its structure for policy re-attribution takes 1.69 s. The API was added for the rebuttal with unchanged solve kernels. Appendix C.2 specifies which data must be refreshed and which can be retained.

Question 1: Confidence parameter, tolerance choice, and sensitivity.

All reported confidence-sequence runs use α=0.05\alpha=0.05, allocated across players and both tails. The default ϵ=Υ~/20\epsilon=\widetilde{\Upsilon}/20 sets a common stopping target of 5% of computed total attributable risk Υ~\widetilde{\Upsilon} for every player's interval radius. Here Υ\Upsilon is the exact total. The results below support this default for identifying a leading group at low cost; tighter targets improve agreement on the highest-responsibility agent.

Appendix C.1 quantifies the trade-off:

ToleranceMedian GPU timeMedian / largest maximum error
Υ~/10\widetilde{\Upsilon}/100.125 s0.83% / 2.49% of Υ\Upsilon
Υ~/20\widetilde{\Upsilon}/200.208 s0.73% / 1.40% of Υ\Upsilon
Υ~/50\widetilde{\Upsilon}/500.415 s0.31% / 0.93% of Υ\Upsilon
Υ~/100\widetilde{\Upsilon}/1001.59 s0.17% / 0.43% of Υ\Upsilon

Times cover 180 task–seed runs per tolerance; errors cover the 126 runs with exhaustive references. On 840 additional-seed runs, the default recovers the exact top-three set in every run. Top-one agreement rises from 74% to 97% at Υ~/100\widetilde{\Upsilon}/100, although the leaders' intervals still overlap. For the leading-group question, the existing top-three stopping rule reaches interval separation in 126/126 reference runs at median 0.358 s, with every returned set exact. Its coverage interpretation remains conditional on Theorem 1; Appendix B.5 reports the numerical implementation audit.

Question 2: Code release.

We will officially publish our code after the review process. For review and rebuttal, the code is available at: https://github.com/moswren7935/satorbit7062

Dear Reviewer mvDw,

Thank you for asking what cuBCR contributes beyond its choice of sampler. We tested three alternative Shapley estimators on the same GPU engine without changing its code. This lets researchers choose how to estimate responsibility while reusing the same GPU implementation for the costly recovery calculations. Appendix C.1 also shows how the stopping tolerance affects accuracy and runtime, and we have shared the code for review. Does this evidence address your concern about the systems contribution? If so, we would appreciate it if you would reconsider your rating.

Best regards, The Authors

Reviewer_QQo1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies failure responsibility attribution in multi-agent systems modeled as finite stochastic games with explicit transitions and reference policies. For a coalition of players whose original policies remain fixed, the coalition value is the minimum probability of failure within a finite horizon when the remaining agents coordinate recovery. Shapley values allocate the gap between the original failure risk and the minimum risk achievable when all agents may coordinate recovery, either at the whole-agent or agent–time level. To reduce the cost of evaluating all 2^d coalitions, CUBCR combines paired-permutation sampling, batched GPU backward induction over a shared sparse model, and adaptive stopping based on confidence sequences.

On TicketBench, the paper reports a median 233× speedup over exhaustive CPU attribution among tasks for which the exhaustive CPU runs completed. This combines sampling savings with GPU acceleration. With identical sampled queries, per-configuration median CPU/GPU runtime ratios are 1.4–5.1×; a separate identical-batch study reports a median GPU speedup of 5.99×. The timings exclude model construction and engine initialization. The reported interval checks contain the corresponding exhaustive FP64 reference values, and coalition-risk calculations are additionally checked against PRISM-games and Storm. Further experiments examine differences between leave-one-out and Shapley rankings, demonstrate decision-time attribution with up to 400 players, and include a restricted Overcooked model with policies fitted to human play.

Critical strengths

S1. Clear formulation and useful independent numerical validation. The paper clearly defines the attribution target and the recovery-controller assumptions, and the Bellman updates match these assumptions. The monotonicity argument provides the bounded marginals needed for the confidence-sequence construction, and the paper is careful to distinguish allocating modeled avoidable failure risk from explaining a particular observed failure. In addition, the coalition-risk calculations are checked against PRISM-games and Storm with very small discrepancies, providing useful evidence beyond agreement between the authors’ own CPU and GPU implementations and strengthening confidence in the core recovery calculations under the stated model assumptions.

S2. A practical systems design supported by informative performance controls. Coalition queries share the same sparse transition model while differing only in their policy-pinning masks, making shared-model batching a well-motivated design. The evaluation also does a good job separating sampling savings from GPU execution gains instead of relying only on the headline speedup, and the same-GPU batching ablation is particularly convincing in showing large gains over evaluating one coalition per launch. Together with the decision-time experiments, these results support the paper’s main practical contribution: making this style of fine-grained Shapley-based attribution computationally feasible at the tested scales.

Critical weaknesses

W1. The reported accuracy–runtime trade-off is not sufficiently specified for reproduction. The provided manuscript gives a general confidence-sequence construction but does not specify the experimental values of α, U, and β, the update rules for the predictable weights and predictions, or the batch size. These choices determine interval widths and stopping behavior, and therefore directly affect the reported runtime results. Without them, it is difficult to reconstruct the accuracy conditions under which the headline speedups were obtained. Appendix B.3 also states that several numerical conditions required by the coverage theorem remain unverified in the implementation. The authors appropriately acknowledge this limitation, and the checks against exhaustive FP64 references are valuable empirical evidence. However, those checks do not establish the implementation’s nominal simultaneous coverage guarantee. For a paper centered on efficient attribution with adaptive stopping, a fully specified experimental stopping procedure and a clear account of which guarantees apply to that procedure are important to assessing the central contribution.

W2. The connection between low runtime and practically useful attribution resolution remains unclear. The primary experiments use ε = Υ̃/20, approximately 5% of the total attributable risk. As a rough reference point, the mean responsibility in a 32-agent game is Υ/32, and in the 400-player experiments the mean responsibility cannot exceed 0.0025. These comparisons do not imply that the actual estimates are inaccurate or that their rankings are unstable. They do show, however, that satisfying the reported stopping target alone does not establish whether individual responsibilities can be meaningfully distinguished. The paper does not report actual estimation errors, interval widths relative to individual responsibilities, or the stability of the highest-responsibility players under sampling. Consequently, the practical meaning of the reported low latencies remains difficult to assess. In addition, the timings exclude model construction and engine initialization, which the paper acknowledges can dominate small jobs. These limitations do not undermine the demonstrated acceleration on preconstructed models, but they leave the broader claim of a deployable, interactive debugging engine insufficiently supported.

Questions for authors

Q1. Experimental stopping procedure and applicable guarantees. What values and update rules were used for α, U, β, the predictable weights, the predictions, and the batch size in the reported experiments? Which numerical conditions in Appendix B are satisfied by the implementation, and which remain unverified? Please provide sufficient configuration details to reproduce the reported stopping behavior.

Q2. Attribution resolution and stricter accuracy targets. On tasks with exhaustive references, what are the actual estimation errors and interval widths relative to individual responsibilities? Are the highest-responsibility players reliably distinguished across sampling runs? How do runtime and speedup change under a substantially tighter tolerance, such as Υ̃/100, that provides greater resolution of individual contributions?

Q3. Initial and repeated attribution costs. What are the total latencies for an initial attribution and for repeated attribution after a policy revision, including the required model construction, engine initialization, and preprocessing? Which costs can be reused across revisions, and how does this affect the interactive policy-analysis use case emphasized in the paper?

3 comments
AuthorsOpenReview ↗

Response to QQo1: stopping rules, accuracy, and complete costs

Thank you for recognizing the shared-model design and independent validation, and for your detailed questions about stopping guarantees, attribution accuracy, and analysis costs. Below, we respond to your comments and explain the changes made in the revised paper.

Weakness 1 / Question 1: Stopping configuration and guarantees.

The revision specifies the full stopping procedure in Appendix B.4 and adds a numerical audit in B.5. In all 441 checked runs, the recomputed intervals contain their FP64 references, and all 183 checked stopping decisions remain unchanged.

All confidence-sequence runs use α=0.05\alpha=0.05, allocated across players and both tails. Each observation averages the contributions from a permutation and its reverse. In primary runs, UU equals the computed total attributable risk plus an allowance for numerical error. Total attributable risk is the difference between the original failure probability and the lowest failure probability achievable through coordinated recovery. The parameter β\beta allows for numerical errors in each paired contribution. Appendix B.4 gives these allowances, the variance-based weight updates, intended predictions from previous observations, initialization, and settings for each experiment family.

Primary batches start at 32 permutation pairs and grow to at most 512. Weights stay fixed within each batch; stopping is checked after it. Decision-time and Overcooked batches use 16–256 pairs. The fixed-budget runs have no tolerance-based stopping. Appendix B.4 also lists the targets and caps.

We compared the original FP64 implementation with a corrected version whose predictions use earlier observations. Across 237 live comparisons under matched settings, stopping batches, pair counts, coalition evaluations, and estimates are unchanged; the median corrected-to-original runtime ratio is 1.0003. Appendix B.5 also replays all 54 decision-time runs with unchanged stopping batches. All nine runs with exhaustive references retain containment. The outward-rounded interval calculations were replayed on recorded observations. Appendix B.5 identifies the remaining live-implementation prediction issue at rounding precision and the unverified GPU operation counts, order, and full rounding bounds.

Weakness 2 / Question 2: Attribution resolution and tighter targets.

The default tolerance identifies the exact top-three set in all 126 original-seed and all 840 additional-seed TicketBench reference runs. These three leaders account for a median 62% of total attributable risk, making this a useful group to identify.

For the original runs, median interval half-widths are 13%, 14%, and 42% of the exact first-, second-, and third-ranked responsibilities. Across the 840 additional-seed runs at the default tolerance, every top-one mismatch selects the exact runner-up, with responsibility shortfall at most 0.91% of computed total attributable risk. These top-one mismatches therefore concern the ordering of two close leaders.

Tightening the target from 5% to 1% of computed total attributable risk raises top-one agreement from 619/840 (74%) to 815/840 (97%). For each original run, we measure the largest error across agents. Its median and worst value fall from 0.73%/1.40% to 0.17%/0.43% of the exact total attributable risk. Across the full 180-run grid, median GPU time rises from 0.208 to 1.59 s, while median matched speedup is 41.9× over logged exhaustive CPU times on 36 tasks.

For users seeking a leading group, stopping when intervals separate the top three from all other players returns the exact set in 126/126 reference runs at median 0.358 s. Its coverage interpretation remains conditional on Theorem 1. Higher top-one agreement alone does not establish interval separation between the leaders. Appendix C.1 also reports larger decision-time cases, whose rankings remain unresolved at the default target.

Question 3: Full initial and revision costs.

Across 20 primary configurations, complete first attribution takes 0.438–1.42 s, including startup, model construction, initialization, upload, and attribution. Complete policy re-attribution takes 0.074–1.06 s with the process and model structure retained. For the 28-agent example, first attribution takes 0.438 s, policy re-attribution takes 0.112 s, and a transition-probability revision takes 0.106 s when the possible transitions remain unchanged.

Overcooked shows why retaining the model matters: first attribution takes 13.2 s and warm model reconstruction still takes 12.7 s, whereas policy re-attribution with the model retained takes 1.69 s. Policy revisions refresh reduction weights; changes to possible transitions require rebuilding. The update interface, added for the rebuttal, uses unchanged solve kernels, and all 171 complete sampler comparisons match fresh builds bit for bit. Appendix C.2 provides the full timing breakdown and reuse requirements.

Dear Reviewer QQo1,

Thank you for asking what the stopping tolerance means in practice. Across 840 additional TicketBench runs with exact references, tightening the tolerance raised agreement on the highest-responsibility agent from 74% to 97%. Appendices B.4–B.5 now specify the stopping procedure and identify which conditions for the coverage guarantee still need verification. Appendix C.2 reports the full time for both first and repeated analyses. Before the discussion closes, could you let us know if any of these points need clarification? If the revision addresses your concerns, we hope you will consider a higher rating.

Best regards, The Authors

Thank you for the thoughtful response. The rebuttal has partially addressed my concerns. However, I still believe that my original rating appropriately reflects my assessment of the paper, so I maintain my current rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised paper: attribution accuracy, complete costs, and GPU reuse

Thank you all, for recognizing the shared-model design, the controlled performance comparisons, and the independent checks against PRISM-games and Storm. The revision adds attribution-accuracy checks, complete latency measurements, and comparisons of alternative estimators. We have limited the practical claims to repeated analysis on the tested explicit models.

The new evidence addresses three concerns:

  • Attribution accuracy. The default tolerance recovers the exact top-three set in all 840 additional-seed TicketBench reference runs. Tightening the tolerance raises top-one agreement from 74% to 97%. Appendix C.1 reports the errors, interval widths, and runtime trade-off.
  • Complete analysis costs. Across 20 primary configurations, first attribution takes 0.438–1.42 s, including startup and model construction. Policy re-attribution takes 0.074–1.06 s with the process and model structure retained. Appendix C.2 also measures the higher costs of the Overcooked and decision-time models.
  • Choice of estimator. Three alternative Shapley estimators run on the unchanged GPU engine. All 162 runs with exhaustive references reproduce their table-based estimates bit for bit. Table 5 now adds warm attribution times at a common empirical error target of 1% of total attributable risk. All four methods meet the held-out criterion on twelve TicketBench games; Leverage SHAP and Stratified SVARM take 40% and 42% of the paired sampler's time, respectively, measured by median per-game ratios. Appendix C.3 gives the protocol.

Appendix B.4 specifies the experimental stopping procedure. Appendix B.5 distinguishes the numerical audit from the conditions required for the coverage theorem. The revised paper also keeps sampling savings separate from GPU execution gains.

We will officially publish our code after the review process. For review and rebuttal, the code is available at: https://github.com/moswren7935/satorbit7062

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.