Measuring Counterfactual Credit in a Replayable Wildfire-Suppression Benchmark

Predicting a team's return and assigning credit to individual actions are distinct tasks. FLARE, a stochastic wildfire-suppression benchmark, uses coupled-draw replay to measure immediate placement effects for training and long-horizon action contrasts for grading critics. In an eight-seed MAPPO comparison, simulator credit reduces interquartile mean burned area from 26.6% with shared reward to 20.9%. A fixed non-replay local reward scores 24.7%; simulator credit wins every paired seed against both. Informed learned credit scores 27.0% and meets a prospectively fixed $\pm 1$-point equivalence criterion against shared reward. Four informed critics paired with responsive main-comparison actors ($0/256$ saturated audited decisions) attain factual-return $R^2=0.911\text{–}0.954$, yet raw credit RMSE is $8.81\text{–}15.59$ times zero's. Matching critic credit to the finite replay reference's RMS still leaves squared error at $1.44\text{–}1.82$ times zero's; ten earlier-rule critics show $1.80\text{–}2.24$. Held-out MSE fitting instead attenuates current predictions by over 97%, approaching zero-credit error. Current critics also give heterogeneous agents nearly identical action rankings (median Spearman $0.980\text{–}0.988$), with largest-magnitude credits on below-1%-probability actions in $219/256$ rows. These measured structures complement the finite-reference calibration diagnostic. Coupled replay separates feedback performance, numerical scale and action-level structure while quantifying replay precision.