acceptodds
Under review as a conference paper at ICLR 2027

Stabilizing Personalized AI-Text Detection with Cross-Domain Consistent Features

Abstract

Large language models (LLMs) have rapidly advanced in generating fluent and personalized text, making it crucial to distinguish machine-generated text (MGT) from human-written text (HWT) in personalized scenarios. However, many MGT detectors exhibit significant performance fluctuations after domain transfer, as they rely heavily on features whose discriminative directions flip across domains. This motivates us to investigate whether stable discriminative signals coexist with inverted ones. Our feature-space analysis provides evidence for both an inverted component whose discriminative direction reverses across domains and a consistent component that preserves it. Building on this, we propose \method, an interpretable and detector-agnostic adjustment framework designed to counter the inversion effect. Without modifying detector architectures, \method adjusts logits using sample projections on consistent features to amplify their influence and suppress inverted ones, thereby restoring stable discrimination in personalized detection. Experiments on nine representative MGT detectors show that \method raises the mean personalized AUROC from 0.45 to 0.84 and lifts every detector above 0.79, with the same selected adjustment transferring across scenarios and across generator LLMs at a mean improvement of AUROC. \method also improves the general-domain AUROC from 0.75 to 0.88, offering a stable and interpretable solution for personalized text detection.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Personalization, Machine-generated text detection, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_n3jAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the performance degradation of machine-generated text detectors in personalized domains, where model-generated text imitates an individual author’s writing style. It proposes PERSFIX, a post-processing method that augments an existing detector score with an auxiliary score obtained by projecting the final-token residual-stream representation from GPT-2 layer 10 onto an extracted direction.

Motivated by the hypothesis that discriminative features may include inverted components, whose machine-versus-human directions flip across domains, and consistent components, whose directions remain stable, the authors extract a consistent linear direction from the final-token residual-stream representation at GPT-2 layer 10. This direction is obtained by aligning machine-minus-human class contrasts across general and personalized domains, and its projection is added to the original detector score.

The correction produces large improvements across multiple detectors on the evaluated personalized benchmarks and transfers across authors and generator models within the personalized literary setting.

The use of a final-token representation is not supported by controls for token aggregation, sequence length, suffix conventions, or other surface-level artifacts. The extraction data also vary simultaneously in genre, task, author style, and historical period, making it difficult to isolate a machine-generation signal from these confounds. The method adds an auxiliary linear projection score rather than directly identifying and suppressing inverted components within the original detector.

The paper shows utility as a post-hoc score correction, but its broader claims concerning representational feature duality and the suppression of detector-internal inverted features are not yet sufficiently supported.

Critical strengths
  1. Practical utility and detector-agnostic applicability.

PERSFIX provides a simple post-hoc correction for personalized style imitation, a setting in which existing detectors can degrade severely or produce below-chance rankings. It requires neither retraining nor internal modification of the underlying detector and yields large empirical gains, particularly in the fine-tuned literary setting. Its transfer across authors and generator models within this setting suggests that the method may be useful as a lightweight correction rather than merely as a detector-specific adjustment.

  1. An actionable response to a known failure mode, supported by extensive empirical analyses.

This paper moves beyond documenting feature inversion by extracting a cross-domain class-contrast direction and using its sample-level projection to adjust detector scores. This method is evaluated across both training-based and training-free detectors, multiple personalization settings, proxy models, layers, and probe-set sizes. Comparisons with random, domain, and generic MGT directions indicate that the proposed direction contains predictive information beyond an arbitrary score perturbation. The Appendix E direction comparisons are explicitly upper-bound analyses because α\alphaα is selected on the test set; they should not be read as held-out performance comparisons.

Critical weaknesses
  1. The representational interpretation is not adequately validated.

The claim that PERSFIX identifies a consistent machine-generation feature relies on a final-token residual-stream representation from GPT-2 layer 10. This state is not trained as a document representation and may be sensitive to the suffix, termination or truncation convention, sequence length, and other local properties. The probing domains differ simultaneously in genre, author style, historical period, and generation task, including rewrite versus continuation. This paper compares layers, modules, proxy models, and extraction datasets, but does not compare token aggregation schemes or control for sequence length, suffix effects, or simple stylometric features. The projections show same-direction HWT–MGT separation on the two probe datasets used to define the construction. They do not isolate whether the direction is stable with respect to machine generation itself, as opposed to the particular combination of source corpus, generation task, author, period, length, and terminal-token convention in those probes. This weakens the causal interpretation attached to the score correction.

  1. The mathematical formulation and experimental specification contain consequential inconsistencies or ambiguities.

Equations 5–6 use cross-domain difference vectors paired by a common index kk, whereas Algorithm 1 constructs each within-domain difference set using all MGT–HWT combinations and then takes the Cartesian product Vg×VsV_g \times V_s. These define materially different estimators. If Algorithm 1 is implemented literally, MM depends only on the sums, and hence on the class-mean contrasts up to scale, of the two domain-wise difference sets. It does not retain the sample-level cross-domain contrast structure suggested by Equations (5)–(6). The matrix has rank at most two, and all nonzero eigenvectors lie in the span of the two domain-level mean contrast vectors. Section 4.2 restricts the search for α\alpha to [−1,1][-1, 1], yet Table 2 reports mean ∣α∗∣\vert{}\alpha^*\vert{} values as high as 3.00; Table 2 is described as using development-set selection, whereas the separate oracle-tuned experiments are introduced only in Appendix E. The current text therefore does not explain how the Table 2 values were obtained. These discrepancies affect the effective estimator, the strength of the correction, and the composition of the headline results.

Questions for authors
  1. Estimator Construction:

Which exact formulation of matrix MM was used in the reported experiments: the sample-paired formulation in Equations 5–6, the all-pairs Cartesian-product formulation in Algorithm 1, or another implementation? Please describe both the within-domain MGT–HWT pairing and the cross-domain aggregation procedure. If Algorithm 1 was used literally, our reading is that the matrix factorizes through the two domain-wise sums of MGT–HWT difference vectors and has rank at most two. How should the resulting eigenspace and the claimed feature-duality interpretation be understood?

  1. Hyperparameter and evaluation setup:

How can the mean ∣α∗∣\vert{}\alpha^*\vert{} values of 2.90–3.00 in Table 2 arise from the candidate range [−1,1][-1, 1] specified in Section 4.2? Was a different search range, score normalization, or oracle-tuning procedure used for Table 2? Please report the exact range and selection data used for this table, and clarify whether the headline averages include nine or eleven detectors and identify the included detectors.

  1. Representation robustness:

Are there results showing whether the main improvements persist under alternative decoder-compatible extraction methods, such as mean pooling over later token positions or EOS pooling with a consistently appended EOS token, and after controlling for sequence length, suffix conventions, and simple stylometric baselines? If not, what evidence supports interpreting the extracted direction as a domain-consistent machine-generation feature rather than as a benchmark-specific auxiliary detector?

4 comments
AuthorsOpenReview ↗

Response 1/3 — W1 + Q3

We thank the reviewer for the careful reading and the technical questions.

W1 + Q3: What VconV_{con} means, and controls for length and surface features

Our definition of a consistent feature is operational. It is a direction in representation space along which human-written and machine-generated texts separate with the same sign in both the general and the personalized domain. Section 3.1 gives this definition and Section 4.1 gives the extraction objective. We do not intend a causal claim, and we do not claim that VconV_{con} represents "machine generation itself."

We acknowledge that some sentences in the paper read more strongly than this. For example, the introduction says that the inverted feature "causes" the trap and that the adjustment "suppresses" inversion. We reword these sentences in the revision. What the paper claims is empirical: adjusting detector scores along this direction repairs detection in personalized settings. This holds across 11 detectors and 45 groups, and under cross-model and cross-scenario transfer.

Ablations already in the paper argue against simple surface explanations. Figure 5a varies the GPT-2 layer and module. Figure 5c replaces the reference model and extracts VconV_{con} again from 19 other probe pairs with different authors and styles. Appendix E replaces VconV_{con} with a domain direction, an MGT direction, or random directions. If the improvement came only from surface cues such as length or style, random and domain directions would not perform so much worse than VconV_{con}. The 19 different probe datasets would also not give similar improvements.

We also tested sequence length directly. We used the representative Fine-tuned subset from Appendix E.1, which is Jane Austen with Qwen3-4B and Lastde++. We measured the correlation between the VconV_{con} projection and the token count. It is −0.01-0.01 for human-written texts and −0.07-0.07 for machine-generated texts, so both are near zero. We then split the samples of this subset into short, medium, and long groups. Detection with the VconV_{con} projection alone reaches 0.90, 0.88, and 0.87 AUROC in the three groups. On this subset, the projection does not depend on length.

Two further facts argue against seeing VconV_{con} as an extra detector tuned to one benchmark. First, in all main and transfer experiments, VconV_{con} is extracted once from 300 probe pairs, 150 per domain. It is then applied unchanged to 3 benchmarks, 45 groups, and 11 detectors, with no tuning per benchmark. Second, when Figure 5c extracts VconV_{con} again from 19 other probe datasets, the improvement stays similar. In the revision, we add the length control as Appendix E.5.

AuthorsOpenReview ↗

Response 2/3 — W2 Part 1 + Q1

W2, Part 1 + Q1: Exact form of the matrix MM and the geometry of feature duality

Our experiments use the index-paired form in Equations 5–6 with N=150N = 150. Appendix D.1 describes the pairs. In the general domain, each pair is a human XSum text gk−g_k^- and its LLaMA-3-8B rewrite gk+g_k^+. In the personalized domain, each pair is a human Jane Austen text sk−s_k^- and its LLaMA-3-8B continuation sk+s_k^+. Thus, vg,k=gk+−gk−v_{g,k} = g_k^+ - g_k^- and vs,k=sk+−sk−v_{s,k} = s_k^+ - s_k^- each pair a human text with the machine text generated from it. Across the two domains, we pair these difference vectors by the index kk and compute M=12∑k=1N(vg,kvs,kT+vs,kvg,kT)M = \frac{1}{2}\sum_{k=1}^{N} (v_{g,k} v_{s,k}^T + v_{s,k} v_{g,k}^T). Algorithm 1 was mistakenly written in all-pairs form. We have corrected it to match Equations 5–6.

We agree with the reviewer that Equations 5–6 and the all-pairs form in Algorithm 1 give different matrices for a finite sample. The rank-two derivation given by the reviewer shows how they are related. Let a=1N∑k=1Nvg,ka = \frac{1}{N}\sum_{k=1}^N v_{g,k} and b=1N∑k=1Nvs,kb = \frac{1}{N}\sum_{k=1}^N v_{s,k} be the mean difference vectors of the two domains. The all-pairs sum equals N2⋅12(abT+baT)N^2 \cdot \frac{1}{2}(a b^T + b a^T). It has rank two when aa and bb are nonzero and not collinear.

The XSum and Jane Austen probe samples have no correspondence, so the index kk across the two domains is arbitrary. The cross-domain pairing can therefore be treated as random. Averaged over uniformly random pairings, the index-paired MM equals N⋅12(abT+baT)N \cdot \frac{1}{2}(a b^T + b a^T). This is the same matrix up to a scale factor, so it has the same eigenvectors. The index-paired MM differs from it by a noise term with zero mean, which comes from the finite sample.

When aa and bb are nonzero and not collinear, the two nonzero eigendirections of 12(abT+baT)\frac{1}{2}(a b^T + b a^T) have closed forms. The positive eigenvalue belongs to a∥a∥+b∥b∥\frac{a}{\|a\|} + \frac{b}{\|b\|}. This is the bisector of the two mean directions, and it corresponds to VconV_{con}. The negative eigenvalue belongs to a∥a∥−b∥b∥\frac{a}{\|a\|} - \frac{b}{\|b\|}. This is their difference, and it corresponds to VinvV_{inv}. In other words, the mean "machine minus human" directions of the two domains form an angle. Their bisector is the part shared by both domains, and their difference is the part that reverses between domains.

We checked this on our probe data. The angle between aa and bb is 103∘103^\circ. The index-paired VconV_{con} has cosine similarity 0.89 with the bisector, and VinvV_{inv} has cosine similarity 0.95 with the difference vector. On the representative Fine-tuned subset, which is Jane Austen with Qwen3-4B and Lastde++, PersFix reaches 0.815 AUROC with the index-paired VconV_{con} and 0.833 with the bisector. On the representative General subset, both reach 0.842. The two forms are therefore close in direction on the probe data and close in effect on these two subsets. In the revision, we fix Algorithm 1 and add this derivation to Appendix F.4.

AuthorsOpenReview ↗

Response 3/3 — W2 Part 2 + Q2

W2, Part 2 + Q2: Range of ∣α∗∣|\alpha^*| in Table 2, selection data, and detector counts

As Section 6.1 states, α\alpha for the combined variants in Table 2 is chosen on the development set. The oracle upper-bound analysis is a separate experiment in Appendix E.

The ∣α∗∣|\alpha^*| values in Table 2 are standardized weights. When we built Table 2, we z-scored the detector score and the VconV_{con} projection separately before combining them with α\alpha. As a result, ∣α∗∣|\alpha^*| is in units of standard deviation and is not limited to [−1,1][-1, 1]. We did this so that the adjustment strength could be averaged across detectors whose raw scores have very different scales, such as log-likelihoods and probabilities. Equation 7 applies α\alpha to the raw scores. The two are different scalings of the same linear combination. The caption of Table 2 did not state this, and we apologize for the omission. In the revision, the caption says that the last column is the mean selected standardized strength, chosen on the development set.

We also checked how the raw-score setup of Section 4.2 behaves in each range of baseline AUROC. We computed the five ranges on all 405 main-experiment subsets with Equation 7 exactly as written, with α∈[−1,1]\alpha \in [-1, 1] chosen on 200 development pairs. We compared this with a wider range, α∈[−3,3]\alpha \in [-3, 3]:

Unadjusted detector AUROCSubsetsUnadjusted detectorSign-flipped on devVconV_{con} alonePersFix, α∈[−1,1]\alpha \in [-1, 1]PersFix, α∈[−3,3]\alpha \in [-3, 3]
<0.5< 0.51550.2920.7050.8110.7920.802
0.5–0.650.5\text{–}0.65690.5640.5570.7900.7900.793
0.65–0.80.65\text{–}0.8620.7230.7230.7990.8440.843
0.8–0.90.8\text{–}0.9330.8630.8630.7540.9010.901
>0.9> 0.9860.9740.9740.7550.9780.978
All4050.5960.7530.7890.8480.852

Where the baseline is below 0.5, the VconV_{con} projection alone is the strongest variant. It reaches 0.811, against 0.792 for PersFix with [−1,1][-1, 1] and 0.802 with [−3,3][-3, 3]. From 0.5 to 0.65, PersFix equals VconV_{con} alone at 0.790. From 0.65 to 0.8 and from 0.8 to 0.9, combining the detector with VconV_{con} is better than either part alone. The values are 0.844 against 0.723 and 0.799, and 0.901 against 0.863 and 0.754. Above 0.9, PersFix stays at the level of the detector, with 0.978 against 0.974. Widening the range from [−1,1][-1, 1] to [−3,3][-3, 3] changes the overall mean from 0.848 to 0.852. In the revision, Table 2 stays in the main text with the corrected caption. The 405-subset table is added as Appendix E.4 and referred to in Section 6.1.

Regarding the number and names of the detectors, we clarify as follows. The paper evaluates eleven detectors, as listed in Table 1. They are RoBERTa, DeTeCtive, Entropy, Log-Likelihood, LogRank, Fast-DetectGPT, DetectLRR, Lastde, Lastde++, DNA-DetectLLM, and SpecDetect++. The text said nine because we added DNA-DetectLLM and SpecDetect++ later and did not update it. In the revision, the text says eleven detectors, and the mean values in the abstract and introduction are computed over all eleven. In the Fine-tuned setting, the mean AUROC rises from 0.466 to 0.839. Some of the additional analyses in this response cover a subset of these detectors.

Thank you for the detailed response.

Authors' response resolves the implementation and hyperparameter ambiguities, but only partially addresses the representational concern. Although the empirical utility of PERSFIX becomes clearer, the feature-duality interpretation remains insufficiently validated.

Reviewer_qT4xOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

In this paper, the authors address the problem of style-transfer attack against existing AI-generated text detectors. The authors hypothesize that the observed drop in performance on personalized texts is caused by the so-called "feature inversion" - situation when relative distributions (natural vs. machine-generated) of certain components of the text representation flip. Building on this, authors propose PERSFIX, a logit adjustment that augments the score of an existing detector with a sample's projection onto a cross-domain consistent direction, essentially trying to "un-do" the reversion. They perform experiments with nine detectors across three settings (general, prompted, fine-tuned) and report significant improvement in performance from the addition of PERSFIX.

Critical strengths

S1. This paper addresses a highly relevant problem - quality of model-generated texts is constantly improving, and the style transfer is one of the strongest adversarial attacks against the existing detectors.

S2. The idea to decompose the hidden state into low-level domain-specific and high-level task-specific features aligns with the previous works in this field.

Critical weaknesses

W1. The proposed method isn't entirely train-free as advertised. It requires some amount of calibration data (200 samples as in the experiments) from the personalized domain, which may be unavailable when the target author is unknown. Also, the choice of certain parameters of the proposed method (β=−50\beta = -50, hidden states from the 10-th layer) isn't properly explained; no sensitivity analysis was reported.

W2. The evaluation setup is severely limited; the results aren't sufficient enough to properly support the main claims:

  • This paper does not study simultaneous shift of both model and domain in cross-domain setting.

  • Training-free detectors are built entirely on the GPT-J-6B model. It is rather dated (released in 2021), which raises concern regarding the artificial lowering of the baseline, and may interfere with the GPT-2 which is used as the backbone for the proposed PERSFIX.

  • The range of baselines is rather limited. Experiments show that addition of PERSFIX improves performance of the relatively old methods for the AI-text detection; they do not actually compare PERSFIX with modern competitive methods.

  • The methods with which PERSFIX are threshold-based; it is quite interesting to see how their performance would change if they were recalibrated on the same development data on which PERSFIX was trained.

  • Results of the cross-domain evaluation are reported only by absolute gain of AUROC over the baseline; no baseline values were given. This may be highly misleading, because if the baseline AUROC is very low (e.g., way below 0.5), improving it (to, say, 0.5) does not guarantee a good classifier.

  • No adversarial attack analysis beyond style transfer (e.g., paraphrasing).

Questions for authors
  • Please, report actual AUROC values (not just the increase) for the baselines and PERSFIX in cross-domain study.
  • Conduct comparison with recalibrated baselines

Some formatting issues:

  • On Figure 4 (and similar figures in the appendix), some labels are unreadable due to the small font-size.

  • Increase Figure 3 - right now it is hard to read.

4 comments
AuthorsOpenReview ↗

Response 1/3 — W1, W2 Point 1

We thank the reviewer for the detailed review and constructive questions.

W1, Part 1: Development data and unknown target authors

We do not claim that PersFix is training-free. The abstract says that it works "without modifying detector architectures," and Section 4.2 says that α\alpha is chosen on a development set. PersFix fits only one scalar, α\alpha.

The unknown-author case is the cross-scenario transfer in Figure 4b. There, α\alpha is chosen on one author and applied to another. In the Fine-tuned setting, this gives a mean AUROC of 0.829. Choosing α\alpha on the target author's own development data gives 0.836, so the difference is less than 1 point. When both the generator and the author change, the mean is 0.826. The unadjusted mean is 0.451. Therefore, when no data from the target author is available, an α\alpha chosen on other authors can be reused.

In general, only a small amount of development data is needed. Across all 405 subsets, where a subset is one detector on one group, 50 development pairs give a mean AUROC of 0.845. With 100 pairs it is 0.849, and with 200 pairs it is 0.848. One fixed α\alpha shared by all groups gives 0.796, while the unadjusted mean is 0.596.

The General setting is different. A reused α\alpha is not reliable across subdomains, as we show under W2, Point 1 below. There we recommend choosing α\alpha on a small labeled set from the target subdomain. In the revision, we state this in Section 4.2 and give the details in Appendix D.7.

W1, Part 2: Choice of β=−50\beta = -50 and of layer 10

In Equation 7, αβ\alpha\beta is a constant added to the score of every sample in a group. It does not change the order of any two samples. AUROC and TPR@5%FPR depend only on this order, so β\beta has no effect on either metric. We checked β∈{0,−25,−50,−100}\beta \in \{0, -25, -50, -100\} on all 405 subsets. The chosen α\alpha, the test AUROC, and the test TPR@5%FPR are the same in every case. We set β=−50\beta = -50 only to keep the adjusted scores in a range close to the original scores.

We chose layer 10 from Figure 5a. This figure compares the residual stream, the attention output, and the MLP output in all 12 layers of GPT-2. The residual stream in the middle and late layers gives the largest and most stable AUROC improvement, and the best layer is layer 10. Figure 5c also shows similar improvements when VconV_{con} is extracted with other reference models, namely Qwen2.5 and Llama-3.1-8B. In the revision, we add both explanations to Section 4.2.

W2, Point 1: Generator and domain change at the same time

In the Fine-tuned setting, the improvement remains when the generator and the scenario change together. We chose α\alpha on the development set of generator A and scenario X. We then tested it on generator B and scenario Y, with A≠BA \neq B and X≠YX \neq Y. The scenario is the author in Fine-tuned and the subdomain in General. Prompted has only one scenario, so it cannot be tested this way. There are 2268 such transfer pairs in Fine-tuned and 2160 in General. The mean test AUROC is:

Target test AUROCBaselinePersFix, target devGenerator changesScenario changesBoth change
Fine-tuned0.4510.8360.8320.8290.826
General0.7430.8740.7790.7490.699

In the Fine-tuned setting, changing both gives 0.826. This is about 1 point below choosing α\alpha on the target group, which gives 0.836. The result is below the baseline in only 5 of the 2268 pairs. This matches Figure 4: in the personalized setting, it matters little which group is used to choose α\alpha.

In the General setting, a reused α\alpha gives 0.699 on average when both change. This is below the unadjusted baseline of 0.743. Most of the loss already appears when only the subdomain changes. The mean is then 0.749, against 0.874 with target data.

The reason is the same as in Section 5.2.2 and Table 2: the suitable α\alpha depends on how strong the base detector already is. In the General setting, most detectors already work well. The suitable α\alpha is therefore small and differs between subdomains, and an α\alpha taken from another group is too large. Grouping the detectors by baseline strength shows this. Weak detectors still improve when both change: RoBERTa goes from 0.499 to 0.584, DeTeCtive from 0.666 to 0.713, and Ent from 0.349 to 0.434. Detectors that start around 0.8 or higher lose 0.04 to 0.15. For example, LL falls from 0.798 to 0.649.

In the General setting, α\alpha should therefore be chosen on the target subdomain. In the revision, we add this table and analysis to Appendix D.5.

AuthorsOpenReview ↗

Response 2/3 — W2 Points 2–4 + Q2

W2, Point 2: GPT-J-6B as the scoring model

We would like to clarify three points. First, we use GPT-J-6B because Lastde++ by Xu et al. at ICLR 2025 uses it, and we wanted our numbers to be directly comparable. Appendix D.3 gives the details. Second, the baselines are not unusually low. In the General setting, Lastde++ reaches 90.6, DNA-DetectLLM reaches 92.5, and SpecDetect++ reaches 90.9 in Table 1. These match their usual performance. The drop appears only in the personalized settings. Third, GPT-J-6B and GPT-2 have separate roles. GPT-J-6B only computes the base detector scores, and GPT-2 only computes the projections onto VconV_{con}.

To test a newer scoring model directly, we replaced GPT-J-6B with Qwen2.5-7B, which was released in 2024. We did this for LL, LR, and Ent on all 45 groups and kept all other settings fixed. Each group uses 200 development pairs and the first 800 test pairs. Our scoring code reproduces the paper's GPT-J scores with a correlation of 1.0000.

SettingDetectorBaseline, GPT-JBaseline, Qwen2.5-7BPersFix, GPT-JPersFix, Qwen2.5-7B
Fine-tunedLL0.2570.2950.8280.828
Fine-tunedLR0.2730.3140.8300.829
Fine-tunedEnt0.6110.6930.8530.875
PromptedLL0.7170.7650.8300.846
PromptedLR0.6940.7520.8280.849
PromptedEnt0.3660.2180.7050.697
GeneralLL0.7980.8330.8920.905
GeneralLR0.8290.8580.9080.916
GeneralEnt0.3480.2070.7320.712

The conclusions hold with Qwen2.5-7B. In the Fine-tuned setting, unadjusted LL and LR still average 0.295 and 0.314. AUROC is below 0.5 in 19 of 21 groups for LL and in 18 of 21 groups for LR. Score inversion is therefore not caused by GPT-J-6B. With PersFix, all of these groups rise above 0.7 and average 0.83, the same as with GPT-J-6B.

In the Prompted and General settings, Qwen2.5-7B raises the LL and LR baselines by 3 to 6 points. Unadjusted Ent becomes weaker. It falls from 0.366 to 0.218 in Prompted and from 0.348 to 0.207 in General. PersFix still improves LL and LR, and it lifts Ent to about 0.70. Across all 135 pairs of group and detector with Qwen2.5-7B, PersFix is below the baseline in only 2 cases. In the revision, we add this table as Appendix D.8.

W2, Point 3: Comparison with recent strong methods

Table 1 already includes recent methods from 2024 to 2026. These are Lastde++ from ICLR 2025, DNA-DetectLLM from NeurIPS 2025, SpecDetect++ from AAAI 2026, and the supervised method DeTeCtive from NeurIPS 2024. PersFix is a score adjustment applied on top of an existing detector. It does not replace the detector. The question is therefore whether it improves these recent methods, and Table 1 shows that it does. In the General setting, DNA-DetectLLM improves from 92.5 to 96.0 and SpecDetect++ from 90.9 to 93.9. In the Fine-tuned setting, these detectors become much weaker: DNA-DetectLLM is inverted at 36.6 and SpecDetect++ falls to 70.0. PersFix raises them to 83.5 and 87.1.

W2, Point 4 + Q2: Comparison with baselines recalibrated on the development set

AUROC and TPR@5%FPR both depend only on the order of the test scores within a group. AUROC covers all thresholds, and TPR@5%FPR is read from the test ROC curve at a 5% false-positive rate. Therefore, any strictly increasing recalibration fitted on the development set leaves every number in Table 1 unchanged. Examples are a new threshold, temperature scaling, and Platt scaling with a positive slope.

Among simple recalibrations, only one choice changes the order: the sign of the score. We can keep sD(x)s_D(x) or use −sD(x)-s_D(x), depending on which is better on the development set. We tested this sign-flip baseline on all 405 subsets:

SettingUnadjustedSign-flipped on devPersFix
Fine-tuned0.4510.6890.836
Prompted0.6230.7280.783
General0.7430.8250.874
All0.5960.7530.848

Flipping the sign on the development set recovers part of the drop. This supports the view that score inversion is real. PersFix is still about 10 points higher overall, at 0.848 against 0.753. This shows that the VconV_{con} projection adds information beyond reversing the sign of the detector. In the revision, we report this sign-flip baseline in the new Appendix E.4, both per setting and per range of baseline AUROC. We also refer to it in Section 6.1.

Continued in the next comment.

AuthorsOpenReview ↗

Response 3/3 — W2 Points 5–6 + Q1, Q3

W2, Point 5 + Q1: Absolute AUROC values in the cross-domain study

The cross-domain improvement does not come from lifting very low baselines to only about 0.5. The table below gives the absolute test AUROC of each detector in the Fine-tuned setting. In the last three columns, α\alpha is chosen on another group and tested on the test set of the target group.

DetectorBaselinePersFix, target devGenerator changesAuthor changesBoth change
RoBERTa0.5030.8380.8380.8380.838
DeT0.7050.8750.8740.8730.873
Ent0.6110.8540.8360.8490.837
LL0.2560.8290.8280.8280.827
LR0.2730.8300.8290.8300.829
SD0.2020.7950.7930.7920.790
LRR0.3540.8290.8210.8230.816
Ld0.6600.8490.8470.8280.829
Ld++0.4920.8270.8220.7960.793

The unadjusted baselines range from 0.202 to 0.705. PersFix with a reused α\alpha reaches 0.790 to 0.874, with an average of about 0.83. This matches the result with α\alpha chosen on the target group itself. In the revision, we add the absolute AUROC tables for Fine-tuned, Prompted, and General to Appendix D.5. We also mark the unadjusted baseline AUROC values on the transfer heatmaps.

W2, Point 6: Paraphrasing and simulated light editing

We used the three representative subsets from our ablation in Appendix E.1, one per setting. On each, we took 500 test pairs and changed only the machine-generated texts, in two ways. The first is paraphrasing the full passage with Qwen2.5-7B-Instruct. The second is simulated light editing, which replaces 10% of the words with synonyms and swaps one pair of neighboring sentences. We tested four detectors: RoBERTa, and LL, LR, and Ent scored with GPT-J. For the changed texts, we chose α\alpha on 100 of the 500 pairs and tested on the other 400. For the original texts, we chose α\alpha on the usual 200-pair development set and tested on the same 400 pairs. Each cell shows the unadjusted AUROC first and the AUROC with PersFix second.

SubsetDetectorOriginalParaphrasedLightly edited
Fine-tuned: Austen / Qwen3-4BLL0.018 → 0.8180.079 → 0.9940.000 → 0.785
LR0.025 → 0.8240.070 → 0.9940.000 → 0.797
Ent0.691 → 0.8630.853 → 0.9970.986 → 0.991
RoBERTa0.830 → 0.8790.833 → 0.9970.984 → 0.975
Prompted: Blogs / Claude-3.7LL0.795 → 0.8090.764 → 0.7000.078 → 0.736
LR0.777 → 0.7950.736 → 0.7440.082 → 0.743
Ent0.317 → 0.7050.370 → 0.6060.895 → 0.913
RoBERTa0.634 → 0.7090.586 → 0.6520.887 → 0.887
General: WikiHow / DavinciLL0.839 → 0.8350.740 → 0.8750.213 → 0.552
LR0.832 → 0.8320.708 → 0.8610.215 → 0.556
Ent0.235 → 0.4850.371 → 0.7370.839 → 0.839
RoBERTa0.685 → 0.6850.527 → 0.8030.666 → 0.666

The effect of editing on unadjusted detectors depends on the detector. Under light editing, LL falls from 0.795 to 0.078 on Blogs and from 0.839 to 0.213 on WikiHow, and LR behaves similarly. Unadjusted Ent rises instead, for example from 0.317 to 0.895 on Blogs.

Across the 24 changed cells, PersFix raises AUROC in 19, ties in 3, and lowers it in 2. One is LL on paraphrased Blogs, from 0.764 to 0.700, probably because α\alpha was chosen on only 100 pairs. The other is RoBERTa on lightly edited Austen, from 0.984 to 0.975, where the baseline is already close to 1.

On the Fine-tuned subset with LL, the original α\alpha works without a new selection. It gives 0.994 under paraphrasing and 0.785 under light editing. The Prompted and General subsets need a new α\alpha chosen on the 100 changed pairs. We did not test attacks designed against VconV_{con}. Such attacks need knowledge of VconV_{con} and of the reference model. In the revision, we add this table as Appendix D.9 and note such attacks in the Limitations in Appendix A.1.

Q3: Font sizes in Figure 3, Figure 4, and the appendix figures

We thank the reviewer for this suggestion. In the revision, we enlarge Figure 3. We also stack the panels of Figure 4 and of the appendix transfer heatmaps in Appendix D.5. Each panel is now drawn larger and its labels are readable.

Thank you for providing a detailed response to my review; I appreciate new experiments and improvements to the paper's text.

Regarding W2, p.3 (limited baseline number) - my concern was rather about comparison with alternative methods of re-calibration of MGT-detectors. The problem of cross-domain instability and feature inversion in personalized domains was realized quite before (even though sometimes it was not directly named so). In the literature there were described some previous attempts to increase cross-domain robustness by recalibrating existing detectors -- an approach not unlike what PERSFIX does. One example is FairOPT introduced by M. Jung and others in "Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection" ( ArXiv:2502.04528 ) - which is actual cited in the submitted paper but no comparison with FairOPT provided. Other methods of group-adaptive thresholding can be applied to the task of MGT detection as well.

Reviewer_8FnCOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the sharp performance changes that machine-generated-text detectors can experience when moved from general writing to personalized writing. The proposed method, PERSFIX, extracts a consistent direction from paired human- and machine-written representations in two domains, then adds each sample’s projection onto that direction to an existing detector’s score. The authors report substantial gains across personalization settings and transfer experiments, while also reporting improvement on the general-domain benchmark.

Critical strengths

Meaningful empirical gains and a useful transfer evaluation. The paper reports a large mean personalized-AUROC increase

Critical weaknesses

The practical setup requires labeled paired data and development-set tuning. Although PERSFIX treats the detector as a black box, extracting its correction direction requires human- and machine-written representations from both general and personalized domains.

The reported improvements are not uniformly robust, and the detector count is unclear. The paper acknowledges a general-domain target where adjustment consistently reduces AUROC, showing that the extracted direction can conflict with some subdomains.

Questions for authors

How much supervision and target-domain access does deployment require?

What happens under active evasion or ordinary editing? Since the paper notes that feature directions could inform evasion, have the authors tested paraphrasing, human editing of generated text, or adversarial changes designed to disrupt the consistent-feature projection?

3 comments
AuthorsOpenReview ↗

Response 1/3 — W1 + Q1

Thank you for your thoughtful comments and questions.

W1 + Q1: Supervision and target-domain data needed at deployment

PersFix uses two kinds of data. The first is the probe data, which we use once to extract VconV_{con}. The second is a small labeled set from the target, which we use to choose the scalar α\alpha.

First, we extract VconV_{con} once from 300 public text pairs. Of these, 150 come from XSum and 150 come from Jane Austen, as described in Appendix D.1. We keep this vector fixed in all main and transfer experiments. Only the sensitivity study in Figure 5 extracts it again on purpose. Figure 5b shows that the AUROC improvement stops growing at around 150 probe pairs per domain. So the extraction cost is small and fixed, and a new target domain does not need a new extraction.

Jane Austen is also one of the 7 authors in the Fine-tuned setting. We therefore removed her from the evaluation as a check. This leaves 162 subsets from the other 6 authors. On these subsets, mean AUROC rises from 0.477 to 0.842. The mean over all 7 authors rises from 0.451 to 0.836, so the two results are close. The Fine-tuned improvement therefore also holds for authors that were not in the probe data.

Second, a new target domain needs only one parameter, the scalar α\alpha. The table below shows the mean test AUROC over all 405 subsets, which are 9 detectors times 45 groups. The columns differ in how much development data from the target domain is used.

SettingUnadjustedOne shared α\alpha for all groups50 dev pairs100 dev pairs200 dev pairs, as in the paper
Mean test AUROC, 405 subsets0.5960.7960.8450.8490.848

Reducing the development set from 200 to 50 pairs barely changes the mean AUROC, which goes from 0.848 to 0.845. A single α\alpha shared by all groups uses no target-domain data and still reaches 0.796. This is 0.20 above the unadjusted 0.596.

Whether α\alpha can be reused without target labels depends on the setting. In the Fine-tuned setting, an α\alpha chosen on another generator or author works almost as well as one chosen on the target group. Mean AUROC is 0.832 when the generator changes, 0.829 when the author changes, and 0.826 when both change. With development data from the target group it is 0.836, and without adjustment it is 0.451. These numbers match Figure 4.

The General setting is different. Base detectors are mostly strong there, so the suitable α\alpha is small and differs between subdomains. A reused α\alpha is not reliable in this case. Mean AUROC falls from 0.874 to 0.749 when the subdomain changes and to 0.699 when both change. The last value is below the unadjusted 0.743.

So our advice is as follows. In the personalized setting, an α\alpha chosen elsewhere can be reused without target labels. For a general-domain target, we recommend choosing α\alpha on 50 to 200 labeled target pairs. In the revision, we add this advice to Section 4.2. We also add the table above and a plot of AUROC against development-set size in the new Appendix D.7.

AuthorsOpenReview ↗

Response 2/3 — W2

W2: Uneven improvements and the number of detectors

The drop noted by the reviewer is the General-domain case in Section 5.2.2, which reports "a consistent drop of about −12-12 for RoBERTa". It occurs in the M4 WikiHow subdomain with ChatGPT texts and the RoBERTa detector. On this group, unadjusted RoBERTa has an AUROC of 0.696. VconV_{con} alone reaches only 0.571, so its signal is weak here.

We choose α\alpha on the development set with a score that combines AUROC and TPR@5%FPR, as in Equations 8–9. On this group, the chosen α\alpha raises development TPR@5%FPR from 0.105 to 0.185, but it lowers development AUROC from 0.623 to 0.535. It has the highest combined score on the development set. On the test set, it lowers AUROC from 0.696 to 0.572. If we choose α\alpha by development AUROC alone, we get α=0\alpha = 0 and the drop does not occur. This agrees with Table 2: a stronger base detector and a weaker VconV_{con} call for a smaller α\alpha.

Across all 405 subsets, PersFix raises AUROC in 361 cases, leaves it unchanged in 21, and lowers it in 23. Only 5 of these decreases are larger than 0.01, and only the WikiHow group above is larger than 0.05. In the Fine-tuned setting, all 189 subsets improve by at least 0.02. Observation 1 in Section 5.2.1 was about the average of each setting, but its wording was too broad. In the revision, we reword it to give the 361 / 21 / 23 counts. We also name the WikiHow / ChatGPT / RoBERTa group in Section 5.2.2 and give the explanation above.

Regarding the number of detectors, we thank the reviewer for noticing this inconsistency. The text said nine, while Table 1 lists eleven. We added DNA-DetectLLM and SpecDetect++ after writing the text and did not update it. In the revision, the text says eleven detectors, and the mean values in the abstract and introduction are computed over all eleven. They are 0.466 to 0.839 for Fine-tuned and 0.774 to 0.888 for General.

AuthorsOpenReview ↗

Response 3/3 — Q2

Q2: Robustness to paraphrasing, simulated editing, and adversarial changes

Paraphrasing and light editing affect unadjusted detectors in different ways, and PersFix improves AUROC on the changed texts in most cases. We did not test attacks that are designed against VconV_{con}.

We used the three representative subsets from our ablation in Appendix E.1, one per setting. On each, we took 500 test pairs and changed only the machine-generated texts, in two ways. The first is paraphrasing the full passage with Qwen2.5-7B-Instruct. The second is simulated light editing, which replaces 10% of the words with synonyms and swaps one pair of neighboring sentences. We tested four detectors: RoBERTa, and LL, LR, and Ent scored with GPT-J. For the changed texts, we chose α\alpha with Equations 8–9 on 100 of the 500 pairs and tested on the other 400. For the original texts, we chose α\alpha on the usual 200-pair development set and tested on the same 400 pairs. Each cell shows the unadjusted AUROC first and the AUROC with PersFix second.

SubsetDetectorOriginalParaphrasedLightly edited
Fine-tuned: Austen / Qwen3-4BLL0.018 → 0.8180.079 → 0.9940.000 → 0.785
LR0.025 → 0.8240.070 → 0.9940.000 → 0.797
Ent0.691 → 0.8630.853 → 0.9970.986 → 0.991
RoBERTa0.830 → 0.8790.833 → 0.9970.984 → 0.975
Prompted: Blogs / Claude-3.7LL0.795 → 0.8090.764 → 0.7000.078 → 0.736
LR0.777 → 0.7950.736 → 0.7440.082 → 0.743
Ent0.317 → 0.7050.370 → 0.6060.895 → 0.913
RoBERTa0.634 → 0.7090.586 → 0.6520.887 → 0.887
General: WikiHow / DavinciLL0.839 → 0.8350.740 → 0.8750.213 → 0.552
LR0.832 → 0.8320.708 → 0.8610.215 → 0.556
Ent0.235 → 0.4850.371 → 0.7370.839 → 0.839
RoBERTa0.685 → 0.6850.527 → 0.8030.666 → 0.666

We draw four observations from this table.

First, the changes affect unadjusted detectors in opposite ways. Under simulated light editing, unadjusted LL falls from 0.795 to 0.078 on Blogs and from 0.839 to 0.213 on WikiHow, and LR falls in a similar way. Unadjusted Ent rises instead: from 0.317 to 0.895 on Blogs, from 0.691 to 0.986 on Austen, and from 0.235 to 0.839 on WikiHow.

Second, the Paraphrased and Lightly edited columns contain 24 cells. PersFix raises AUROC in 19 of them, ties in 3, and lowers it in 2. One decrease is LL on paraphrased Blogs, from 0.764 to 0.700. A likely reason is that α\alpha was chosen on only 100 pairs, which makes the choice less stable. The other decrease is RoBERTa on lightly edited Austen, from 0.984 to 0.975. Here the unadjusted score is already close to 1, and there is little room to improve.

Third, whether α\alpha must be chosen again on the changed texts depends on the setting. On the Fine-tuned subset with LL, the original α\alpha remains effective without a new selection. It gives 0.994 under paraphrasing and 0.785 under light editing, the same as the newly chosen α\alpha. In the Prompted and General subsets, a new selection is needed. Reusing the original α\alpha is unstable there. For example, LL on paraphrased Blogs falls from 0.764 to 0.648. A likely reason is that paraphrasing shifts the distribution of the projection on VconV_{con}, while the base detector already performs well and needs only a small α\alpha. For this reason, the table reports results with α\alpha chosen on the 100 changed pairs.

Fourth, we did not test white-box attacks that edit a text to disturb its projection onto VconV_{con}. Such an attack needs access to VconV_{con} and to the GPT-2 reference model. Detectors with a public scoring function share this weakness. An attacker can, for example, optimize a text directly against LL.

In the revision, we add the paraphrasing and light-editing experiments as Appendix D.9. We also note targeted attacks against VconV_{con} in the Limitations in Appendix A.1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.