- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies repeated retraining in a persistent graph environment, where predictions affect subsequent features and, in the synthetic setting, outcomes. It distinguishes evaluation using held out data on a common graph state from evaluation along each deployment policy’s own trajectory.
Three main contributions:
-
An exact decomposition of policy loss differences into contributions from fitting on a common state and from differences in policy induced states.
-
Results establishing that exact convex refitting is well defined and admits an invariant distribution, a sufficient condition over a finite horizon for preserving a common state advantage between two refitted representations, and a recourse margin condition at a fixed state.
-
Synthetic evidence about interactions between representations and update policies, including a reversal in the ranking of GCN and skip GCN, and a product description recourse simulation reporting threshold retention separately from aggregate prediction loss.
The claims are evaluated through proofs, simulations on 20 independently generated graphs, a neural extension trained end to end, and a deterministic application using interactions between Amazon users and products.
Critical strengths
S1. The theoretical results appear sound under their stated assumptions, with invariant distribution existence clearly distinguished from convergence and uniqueness.
S2. The GCN and skip GCN ranking reverses across update policies, with a paired interaction of 0.05817 and a 95% interval of [0.04131, 0.07664]. This supports evaluating architecture and retraining jointly.
S3. The convex study handles dependence by averaging five outcome realizations per graph before bootstrapping 20 graphs. The application reports eligibility retention of approximately 88%, 100%, and 94% for R, G, and C, while clearly distinguishing simulated retention from predictive loss and marketplace benefits.
Critical weaknesses
W1. The paper does not directly demonstrate that better held out performance on a common state leads to worse policy loss. The adverse example in Appendix E.6 supports a training objective interpretation, but its corresponding held out decomposition is missing.
W2. The transfer bound is valid under its assumptions, but its usefulness under nonzero feedback remains un demonstrated. Neither the main bound nor the separate refitting versus fixed classifier bound is evaluated in the reported experiments.
W3. The application reports loss differences around 10⁻⁶ but omits the promised numerical sign checks, complete attributions, and solver accuracy evidence. The reported directions therefore cannot be verified from the paper, although the conclusion that aggregate differences are small remains supported.
Questions for authors
Q1. Does “better fit” mean the regularized training objective or held out loss, and does the +0.060152 example have a negative common state fitting term?
Q2. Is there a nonzero feedback example with C_T < Gamma_T, and what information is needed to evaluate the bound for the relevant policy comparison?
Q3. Can the application’s numerical sign certificates, loss attributions, and solver accuracy checks be provided?
5 comments
W3, Q3
W3, Q3
We have verified the signed bounds, loss attributions, and fitting accuracy for all three application models. Appendix G.4 (Tables 14–15, pp. 38–40) now reports the complete results, summarized in Section 4.3 (p. 9).
(i) Numerical sign checks and loss attribution. We report the average policy gap over rounds 1–20, , and the final-round gap, . Across all 24 comparisons—three representations, four evaluation sets, and two evaluation windows—we found that each interval lies entirely above or below zero, confirming the sign of the corresponding policy gap. For example, R's full-warm average gap is , with bounds . Using the initial classifier as the reference, the fitting and induced-state contributions are and , respectively.
(ii) Solver accuracy checks. We independently recomputed the gradients for all three initial fits and five subsequent refits. Every original fit satisfies . Tightening both the initial fits and subsequent refits yields . Recomputing both policies with these more accurate fits preserves all implemented description sequences, threshold decisions, and 24 loss-gap directions. The largest change in a final or average gap is .
Please refer to the revised manuscript for detailed experimental results
Thank you for the detailed response.
W1, Q1: The held out decomposition answers my question.
W2, Q2: The four node example shows that the condition can hold under nonzero feedback. It covers one round on a very small graph, so the usefulness of the bound in the reported experiments remains open.
Thank you for your comments.
“Better fit” and held-out loss. "Better fit" refers to lower held-out test loss R(f,s) on a common state. And yes, in the +0.060152 example, the mean fitting contribution is negative: −0.046152.
Held-out decomposition. We define the policy gap averaged over rounds t0+1,…,H as
\coloneqq J_{t_0,H}(\mathrm{refit})-J_{t_0,H}(\mathrm{fixed}).$$ For this example, $(t_0,H)=(600,1000)$. Using the initial classifier $f_0$ as the reference, the fitting contribution $\mathcal R(f_t^{\mathrm{refit}},s_t^{\mathrm{refit}})-\mathcal R(f_0,s_t^{\mathrm{refit}})$ averages $-0.046152$ and the induced-state contribution $\mathcal R(f_0,s_t^{\mathrm{refit}})-\mathcal R(f_0,s_t^{\mathrm{fixed}})$ averages $+0.106304$. The regularized training objective $\mathcal L$ is $0.127136$ lower for the refitted classifier than for $f_0$, with both evaluated on the same current training data. **Experiments.** This example uses five independent graphs with $n=400$ nodes, three outcome realizations per graph, the unscaled two-hop representation $(K=2)$, ridge penalty $\lambda=0.01$, and initial outcomes $y_{i,0}=g_i$, where $g_i$ is the latent group indicator. We average rounds 601–1000 and group realizations within each graph. The fitting-contribution and policy-gap pointwise 95% graph-bootstrap intervals are $[-0.068982,-0.024325]$ and $[0.039277,0.076693]$, respectively. All five graph averages have negative fitting contributions and positive policy gaps. These remain retrospective exploratory estimates from five graphs. Using the primary Bernoulli initialization in this same comparison favors refitting. Appendix E.6 (Table 5, pp. 27–28) now reports the decomposition, setting, and uncertainty. Appendix E.4 (p. 26) specifies the interval calculation, and Section 4.1 (p. 7) highlights the example. > W2, Q2 **(i) Nonzero-feedback example.** On a complete graph with 4 nodes, we compare raw features (A) with one-hop propagated features (B) and refit both classifiers each round. With $\lambda=1$, $(a,c)=(0.1,1)$, and $\varepsilon=0.1$, the two deployments produce different feature states after one round. The common-state advantage is $\Gamma_1\approx0.240969$ and the trajectory term is $C_1\approx0.226933$. Numerical validation showed that $\Gamma_1-C_1>0.0140$. Theorem 1 therefore guarantees lower expected policy loss for A, $J_1(A)<J_1(B)$. Appendix D.4 (pp. 24–25) provides the complete construction and calculation. **(ii) Information needed to evaluate the bound.** To apply Theorem 1, we compute the common-state advantage $\Gamma_T$ and trajectory term $C_T$, then check whether $\Gamma_T>C_T$. To compute $\Gamma_T$, we refit both classifiers on B's states and evaluate their losses on the same test nodes. We average B's loss minus A's loss over B's state distribution and across rounds. To compute $C_T$, we first calculate the expected logit discrepancies $\nu_t$ between these fitted classifiers. We compute $(\mathbf B,\mathbf g,\mathbf c_*)$ from the response parameters, ridge penalty, training/test sizes, and representation bounds, as specified in Appendix D.2. Substituting these constants and $\nu_t$ into Equation (11) yields $C_T$. For the refitting-versus-fixed comparison, we obtain an upper bound on the policy gap by adding the fitting contribution to a bound on the absolute induced-state contribution. We compute the fitting contribution by comparing the refitted and initial classifiers on the refitting policy's states and averaging their loss difference. To compute the induced-state bound, we calculate their expected per-node logit discrepancies $\mathbb E\boldsymbol\delta_t$ on those same states. We evaluate the bound in Appendix D.3, Equation (37), using these discrepancies, the initial classifier $f_0$, graph operators $\mathbf S^K$ and $\mathbf N$, and response parameters.