- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes RWCP for detecting authorship transitions in mixed human–LLM text. It replaces WCP’s weighted mean-based contrast with a Huber profile-loss gain, combined with capped reliability weights and reliability-coordinate interval search. A merge-cost analysis establishes theoretical guarantees under certain assumptions. The paper also evaluates RWCP-R, a separate recurrent-state decoder that shares source centers across nonadjacent passages.
Critical strengths
The paper provides a useful way to analyze robust profile gains without explicitly solving for mixed-segment Huber centers. In Theorem 3.11 establishes finite-sample count-recovery and localization guarantees under explicit conditions in Assumption 3.1-3.6. The experimental results cover five benchmark families and show consistent improvements in WindowDiff (WD) metric. Core RWCP reduces family-macro WP by 17.6% compared with WCP, while RWCP-R achieves further improvements.
Critical weaknesses
-
There are inconsistencies in the assumptions throughout that substantially weaken the contributions in the paper: W1: For example, how are the Assumptions 3.1-3.6, consistent with WCP? Why is the recurrent decoder in Tables 4 and 17 different from RWCP? Due to such differences, the actual effect of the proposed method cannot be evaluated.
-
There are several weaknesses in the experimental results particularly in some important regimes. One is the boundary recovery problem. Appendix B.2, Table 6 shows that core RWCP detects no boundaries in 82.4% of CoAuthor documents and 93.1% of multiple-change documents, with Loc. cov. of 0% and 1.0% respectively. This issue persists for RWCP-R. Appendix B.7.2 Table 16 reports an 84.1% no-boundary fraction on CoAuthor, where its WD (0.4869) is only slightly better than the baseline (0.4893). Moreover, Table 17 shows that replacing Huber loss with quadratic loss improves count MAE and boundary F1, and only moderately lower WD (0.3110 to 0.3179). These results raise practical concerns about the porposed work.
Questions for authors
-
Could the authors directly compare RWCP’s assumptions and localization guarantees with those of WCP both theoretically and experimentally?
-
Could the author provide a comparison about (WD)-(boundary F1) trade-offs between RWCP and WCP under various detection thresholds or transition penalties? This would be helpful to clarify whether the method actually improves boundary detection, or primarily achieves lower WD by predicting fewer boundaries.
3 comments
3. Boundary recovery (continued): CoAuthor, localization and Table 17
CoAuthor sessions average 24 sentences and 10.4 changes, 62% of runs are one sentence, the sentence-source AUC is 0.68 and the adjacent-difference boundary AUC is 0.53. Under the frozen setting RWCP-R has the lowest CoAuthor WD (0.4869, null 0.4893). To compare boundary recovery, both methods were tuned on the 289 validation sessions by WD alone or by and tested once on 290 sessions:
| CoAuthor | WD ↓ | MAE ↓ | F1 ↑ | Zero CP |
|---|---|---|---|---|
| WCP, default | 0.488 | 7.92 | 0.302 | 15.9% |
| WCP, composite | 0.524 | 6.35 | 0.440 | 1.7% |
| WCP, WD-only | 0.489 | 9.86 | 0.098 | 70.7% |
| RWCP-R, frozen | 0.487 | 10.19 | 0.058 | 84.1% |
| RWCP-R, composite | 0.532 | 6.01 | 0.472 | 0.3% |
| RWCP-R, WD-only | 0.490 | 10.19 | 0.056 | 84.1% |
| No boundary | 0.489 | 10.38 | 0.031 | 100.0% |
With composite selection RWCP-R has the best F1 and the lowest count MAE among the configurations in this table: RWCP-R minus WCP is ΔF1 = +0.032 [0.005, 0.058], ΔMAE = −0.34 [−0.74, 0.04], ΔWD = +0.008 [−0.010, 0.025] (session bootstrap, 2,000 draws). With WD-only selection the difference in F1 is −0.043 [−0.064, −0.022].
Mean absolute boundary error (sentences) on documents where both methods get the count right:
| Family | Shared docs | WCP ↓ | RWCP-R ↓ |
|---|---|---|---|
| Single change | 70 | 0.686 | 0.743 |
| CoAuthor | 0 | - | - |
| Multiple changes | 34 | 2.840 | 3.081 |
| Text attacks | 136 | 1.816 | 1.654 |
| LLM proportion | 58 | 3.897 | 4.121 |
Once the count is right, both methods place boundaries with similar precision (within 0.3 sentences, RWCP-R ahead on attacks); RWCP-R's advantage is that it gets the count right far more often.
Table 17 ablations:
| Variant | Macro WD ↓ | MAE ↓ | F1 ↑ | Zero CP |
|---|---|---|---|---|
| Full, frozen | 0.3110 | 3.019 | 0.230 | 60.2% |
| Quadratic loss | 0.3179 | 2.945 | 0.239 | 54.9% |
| Uniform weights | 0.3118 | 2.917 | 0.261 | 51.0% |
| Fixed scale | 0.3170 | 3.071 | 0.211 | 64.0% |
| No recurrence | 0.3584 | 3.072 | 0.230 | 49.5% |
| No penalty | 0.7299 | 10.649 | 0.333 | 0.3% |
The full model has the lowest WD. Huber loss is a safeguard against extreme scores: with 5% extreme scores, removing it raises WD by 0.056, versus 0.007 on original scores. With 10% contamination and 20-fold proxy inflation, removing Huber changes WD by −0.022 and removing weights by −0.148; the revision discusses proxy quality accordingly.
4. Threshold and penalty trade-offs
Each method varies one detection control on the same scores: WCP the AIC multiplier and NOT threshold (CoAuthor), with reproducing all 2,684 saved predictions; Core a threshold multiplier; RWCP-R the penalty λ.
| Setting | WD ↓ | MAE ↓ | F1 ↑ | Zero CP |
|---|---|---|---|---|
| WCP r/m 0.25/0.25 | 0.7157 | 6.446 | 0.354 | 1.8% |
| WCP 0.5/0.5 | 0.5980 | 4.017 | 0.349 | 4.0% |
| WCP 1/1 (default) | 0.4070 | 2.808 | 0.287 | 21.7% |
| WCP 2/1.5 | 0.3316 | 3.082 | 0.172 | 65.2% |
| WCP 4/2 | 0.3385 | 3.318 | 0.077 | 89.3% |
| WCP 8/3 | 0.3567 | 3.416 | 0.020 | 98.4% |
| WCP 16/4 | 0.3595 | 3.431 | 0.012 | 99.7% |
| Core ×0.1 | 0.5993 | 3.347 | 0.328 | 3.8% |
| Core ×0.25 | 0.4567 | 2.657 | 0.305 | 13.6% |
| Core ×0.5 | 0.3622 | 2.870 | 0.230 | 40.3% |
| Core ×0.75 | 0.3404 | 3.110 | 0.168 | 64.7% |
| Core ×1 (default) | 0.3355 | 3.229 | 0.129 | 78.9% |
| Core ×1.5 | 0.3407 | 3.321 | 0.087 | 88.8% |
| Core ×2 | 0.3461 | 3.370 | 0.061 | 93.8% |
| RWCP-R λ=0.25 | 0.6983 | 9.062 | 0.344 | 0.3% |
| RWCP-R λ=0.5 | 0.6654 | 7.669 | 0.356 | 0.3% |
| RWCP-R λ=1 | 0.5725 | 4.951 | 0.382 | 0.6% |
| RWCP-R λ=2 | 0.3700 | 2.510 | 0.383 | 12.5% |
| RWCP-R λ=4 (frozen) | 0.3110 | 3.019 | 0.230 | 60.2% |
| RWCP-R λ=8 | 0.3301 | 3.297 | 0.114 | 86.6% |
| RWCP-R λ=16 | 0.3524 | 3.401 | 0.039 | 96.8% |
Every WCP point on this path is dominated in both WD and F1 by an RWCP-R point. At λ = 2 RWCP-R beats default WCP on all three metrics. At matched boundary counts (λ=4 vs WCP 2/1.5: 0.49 vs 0.50 boundaries per document) RWCP-R still has ΔWD = −0.021 [−0.026, −0.015] and ΔF1 = +0.058 [0.044, 0.072], so the gain does not come from predicting fewer boundaries. Minimum WD with F1 at least the family's default-WCP F1:
| Family | F1 floor | WCP min WD | RWCP-R min WD |
|---|---|---|---|
| Single change | 0.435 | 0.396 | 0.190 |
| CoAuthor | 0.302 | 0.488 | 0.507 |
| Multiple changes | 0.205 | 0.436 | 0.428 |
| Text attacks | 0.272 | 0.361 | 0.268 |
| LLM proportion | 0.220 | 0.354 | 0.393 |
| Family macro | 0.287 | 0.407 | 0.370 |
At equal F1, RWCP-R lowers macro WD by 0.037 (selection-aware bootstrap −0.037 [−0.048, −0.024]), with large gains on single changes and attacks.
Thank you for the rebuttal. My main experimental concern is not resolved. It appears that RWCP-R performs worse than WCP in aggregate boundary F1 and count MAE. This reinforces my original concern that improvements in WD do not necessarily translate into improved authorship boundary detection. I will keep my original score.
Thank you very much for your support and valuable comments. We have carefully reviewed them, which are very helpful. We summarize the key concerns you raised, supplemented the corresponding experiments, and provide our responses as follows.
1. Assumptions and guarantees
Assumptions 3.1–3.6 extend WCP's setting rather than restating it. WCP assumes independent heteroscedastic sub-Gaussian scores with source-specific means and inverse squared proxy weights. Our theorem targets segment-wise Huber locations under local curvature and concentration of bounded transforms, which covers heavy-tailed scores and an explicit dependence condition.
Corollary 3.12 transfers the guarantee to estimated weights that are uniformly comparable to oracle weights.
On rates: WCP gives exact-count recovery and weighted localization of order log(N/δ)/κ2 with κ a mean gap. Theorem 3.11 gives, on one event of probability at least 1−δ, K=K and dw(τj,τj)≤CΛN(δ)/κj2, with ΛN(δ)=qN2log{64(1+N2+N3)/δ}, κj a Huber-location jump, qN=O(1) under independence or fixed-order dependence and O(log2(eN)) under geometric mixing. When the two targets coincide, the independent-case orders match, so robustness and dependence come at no cost in rate. Proposition 2.2 shows that the quadratic profile gain equals squared weighted CUSUM, so WCP's contrast is the quadratic limit of ours. Theory-aligned simulations will be added.
2. Relationship between Core and RWCP-R
RWCP-R is a separate decoder built on the same weighted Huber loss. It shares source-state centers across nonadjacent passages, fits a scale, and solves a transition-penalized objective by alternating exact decoding and parameter updates; this is why the decoder in Tables 4 and 17 differs from Core. Since RWCP-R carries the empirical results, the revision is organized around it.
Averages are taken within settings, families, then over five families; F1 uses a one-sentence tolerance. RWCP-R has the lowest WD of all methods. Against null it gives ΔWD = −0.0511 [−0.0588, −0.0432], ΔMAE = −0.414 [−0.444, −0.383], ΔF1 = +0.221 [0.203, 0.240] (95% paired source-component bootstrap), so it is clearly better than predicting nothing on all three metrics.
3. Boundary recovery
Exact-count coverage is the share of documents with changes where K=K. RWCP-R recovers the exact number of changes more often than WCP on single changes (65.8% vs 30.9%), text attacks (40.9% vs 34.9%) and LLM proportion (26.6% vs 22.0%), and raises coverage over Core in every family except CoAuthor. Core's conservative behavior comes from its analytic threshold: the first-step maximum gain is below it on 82.4% of CoAuthor, 93.1% of multiple-change, 94.6% of attack and 92.8% of proportion documents. RWCP-R replaces this threshold with an explicit penalty, which reduces the no-boundary rate by 26–40 points on multiple changes, attacks and proportions.