acceptodds
Under review as a conference paper at ICLR 2027

Self-Evolving Private Evolution for Differentially Private Synthetic Text Generation

Abstract

Differentially private synthetic text generation enables data sharing and text mining applications while providing formal privacy guarantees. Recent advances leverage large language models (LLMs) as strong generative priors for producing fluent and semantically rich synthetic text. Among these methods, Private Evolution (PE) is a promising approach that incorporates LLM inference APIs to generate and refine synthetic samples without expensive fine-tuning. PE iteratively improves synthetic data by generating variations, evaluating candidates through private voting, and selecting high-quality samples for the next iteration. However, existing PE methods discard these privatized evaluation signals after each iteration, despite their potential utility for future rounds. We propose SelfEvo-PE, which reuses available privatized signals as supervision to train lightweight evaluators that guide future evolution steps without additional privacy cost. This mechanism allows SelfEvo-PE to perform more evolution steps under the same privacy budget. Our experiments on three datasets show that SelfEvo-PE outperforms the conventional PE (Aug-PE) under limited-data and strict-privacy settings, improving FID and MAUVE by up to 15% and 20%, respectively.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
differential privacy, synthetic data, synthetic text, private evolution

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_9GsGOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new method for private synthetic text generation using LLMs. The problem is timely and important. The proposed method builds on Aug-PE and proposes a lightweight evaluator that uses previously privatized voting results to guide additional evolution steps without further access to the private corpus. This enables more iterations without additional privacy cost and improves generation quality under the same privacy budget.

Critical strengths
  1. The paper is well-presented, easy to follow and intuitive.
  2. The proposed method meaningfully improves Aug-PE while being simple and computationally efficient.
  3. Experiments include diverse datasets and informative ablations.
Critical weaknesses
  1. My biggest concern is the choice of baselines. The paper mostly compares against older methods such as Aug-PE and CTCL, but leave out many newer methods in both inference-only and fine-tuning branches. For example, see [1-6] for the methods that should have been compared against. Also see [7] for a survey. With the current status, I lean toward rejecting the paper. If the authors provide a comprehensive comparison with these baselines, I will increase my score. Otherwise, I am unsure that the survey of previous methods are informative enough for acceptance.

  2. What are hyperparameters used to compute MAUVE? MAUVE is known to be highly sensitive to hyperparameters such as scaling factors or number of clusters. For example, [2] and [3] show that under MAUVE scaling factor 5, many of methods based on instruction-tuned models show very low MAUVE score, reflecting their low diversity. Also in Figure 6, the authors should clarify if the number of clusters is growing with the data size or is fixed.

  3. What is the reason behind choosing a relatively old LLM, i.e., Llama 3.2 3B-Instruct, as the main model in experiments? How do the results change if a newer reasoning model is used instead?

  4. What is the privacy-utility trade-off of the method? In particular, how does SelfEvo-PE perform in ϵ=0\epsilon=0 and ϵ=∞\epsilon=\infty cases?

[1] Private prediction for large-scale synthetic text generation

[2] Clustering and Median Aggregation Improve Differentially Private Inference

[3] EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors

[4] ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Control

[5] MAPLE: Metadata Augmented Private Language Evolution

[6] DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning

[7] How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy

Questions for authors

Please see weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 9GsG (1/2)

Thank you for taking the time to review our work and share your suggestions.

Weakness 1: Baselines

We thank the reviewer for the suggestion. Unfortunately, most of the mentioned papers [1,2,5,6] do not release their code. Our paper already cited and discussed [1], [2], [4], [5], [6], and [7]. We have updated the paper to include [3] as well. The survey [7] categorizes the methods into three main branches based on the setting differences and their strenghts: (1) Private Inference [1,2], (2) Private Evolution [5], and (3) Fine-tuning [4,6,7] (please see Table 10, Page 62).

First of all, our method is built on the private evolution branch, which leverages black-box LLM APIs only and does not require any access to the model weights or fine-tuning. Therefore, the fine-tuning methods [4,6,7] are not directly comparable to our method. While MAPLE [5] is a private evolution method, it enhances the random_api which is orthogonal to our method. Additionally, MAPLE requires an additional assumption of a small public (donated) in-distribution dataset, which is not required in our method. [3] also does not require fine tuning, but it requires the control of inference (forward pass) to manipulate the hidden states of the model, which is not available in most standard LLM APIs.

We implement the Private Inference methods [1,2] only, as they share the closest setting with our method. Private Inference was designed to generate a small number of synthetic samples (e.g., <25 samples) mainly for in-context learning. These methods add noise to the softmax output of each token, so the required noise scale is increased with both the number and the length of the generated samples. However, we found that to generate a few hundred samples, the noise is too large that makes each token probability distribution close to uniform. The below table shows the results of [1] and [2] on the BioRxiv dataset with ϵ=1.0\epsilon = 1.0. We added this new result to Appendix C.7.5.

MethodFIDMAUVEAcc
Private Inference [1]130.000.00657.45
Private Inference [2]135.240.00661.88
Aug-PE40.620.66769.62
SelfEvo-PE34.930.71871.23

[1] Private prediction for large-scale synthetic text generation

[2] Clustering and Median Aggregation Improve Differentially Private Inference

[3] EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors

[4] ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Control

[5] MAPLE: Metadata Augmented Private Language Evolution

[6] DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning

[7] How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy

AuthorsOpenReview ↗

Response to Reviewer 9GsG (2/2)

Weakness 2: MAUVE hyperparameters

We use the default hyperparameters of the mauve-text library [1]. In particular, the scaling factor is set to a constant value at 5. This configuration is consistently used to evaluated all methods in our experiments.

[1] https://pypi.org/project/mauve-text/

LLM backbone

The reason we chose a relatively old LLM because the cutoff date of its training data is before the date of the datasets we used in our experiments. This ensures that the LLM has not seen any of the private data during its pretraining, which is important for the DP guarantee.

In Appendix C.7.3, we presented the result with a slightly larger LLM (Qwen-2.5-7B), where SelfEvo-PE outperforms Aug-PE across different privacy budgets.

Additionally, we conduct a new experiment using a closed-source model (GPT6-luna) on the BioRxiv dataset.

MethodFIDMAUVEAcc
Aug-PE67.370.2480.625
SelfEvo-PE60.170.2790.670

SelfEvo-PE also outperforms Aug-PE in this case. This confirms that the gain of SelfEvo-PE is from the better noise reduction and selection performance, regardless of the LLM backbone. We added this new result to Appendix C.7.4.

Performance without DP

While the main point of SelfEvo-PE is to enhance PE through better selection and noise reduction, we do not expect SelfEvo-PE to improve over Aug-PE in the non-private setting. We added this new result to Appendix C.5.3.

DatasetMethodFIDMAUVEAcc
BioRxivAug-PE28.910.95473.253
BioRxivSelfEvo-PE27.710.93173.919
Congressional SpeechAug-PE34.920.95260.067
Congressional SpeechSelfEvo-PE36.540.85757.367
OpenReviewAug-PE42.210.46143.126
OpenReviewSelfEvo-PE42.870.35441.637

Private Evolution is not able to perform with ϵ=0.0\epsilon = 0.0, as there is no signal to guide the evolution. Instead, we can consider the zero-shot baseline that was already included in the main paper.

Dear Reviewer,

Thank you very much for your time and helpful suggestions. We hope our response addresses your concerns. As we are approaching the rebuttal deadline, please let us know if you have any further questions or concerns.

Sincerely, Authors of SelfEvo-PE

Reviewer_3bq2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SelfEvo-PE treats the differential privacy voting signals from Private Evolution—which are typically discarded after each round—as reusable supervisory data. By employing a lightweight evaluator, it guides additional evolution steps without incurring any extra privacy cost. Consequently, this approach improves the FID, MAUVE, and downstream classification performance of DP-generated synthetic text, even when faced with limited private data and a strict privacy budget.

Critical strengths
  1. Propose SelfEvo-PE, which treats the noisy DP voting results generated in each round of PE as persistent resources, rather than as transient signals to be discarded after a single round of selection.
  2. Train a lightweight evaluator using pairwise LTR (Learning-to-Rank), density-aware features, and an XGBoost ensemble to guide the self-evolving iterations, without requiring any additional access to private data.
Critical weaknesses
  1. The paper lacks experiments using large-scale datasets, such as Yelp. Furthermore, it does not include ablation studies on the quantity of synthetic data—specifically, how performance varies when larger amounts of synthetic data are used.

  2. A key advantage of the "PE" line of research is its ability to leverage the capabilities of closed-source models. This paper lacks experiments in this regard, and the LLM backbone employed is relatively weak; both factors undermine the reliability of the experimental results presented.

Questions for authors

See Critical Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 3bq2 (1/1)

Thank you for taking the time to review our work and share your feedback!

Large-scale datasets

Our method is designed for small-data (a few thousand samples) and strict-privacy regimes. This setting is common in many real-world applications. We pick randomly 4K+ datasets on Huggingface and analyze their dataset size [1].

PercentileNumber of samples
10th13
25th209
50th (median)1,842
75th26,152
90th247,714

More than half of the datasets have less than 2K samples. This demonstrates the importance of our setting in practice.

Additionally, in Sec. 4.2.3 and Appx. C.4, we presented an analysis that shows the regime where SelfEvo-PE is effective. If the dataset is large (e.g., 100K samples), the noise unlikely affects the selection, and SelfEvo-PE is not needed. In this case, Aug-PE is the right choice, we do not expect SelfEvo-PE to improve over Aug-PE.

LLM backbone

PE's advantage is API-only access, for both closed and open models. We aggree that reaching closed-source models is one benefit of PE. However, we also would like to emphasize that API-only access is a key advantage because clients may not have resources to fine-tune a LLM in many applications. PE is designed for this scenario. For example, Apple has used PE for their privacy-preserving applications [2]. Therefore, PE tackles a more general and practical scenario of API-only access, regardless of whether the model is closed or open.

In Appendix C.7.3, we presented the results with Qwen-2.5-7B, where SelfEvo-PE outperforms Aug-PE across different privacy budgets.

We provide additional results on a closed-source model (GPT6-luna) on BioRxiv dataset.

MethodFIDMAUVEAcc
Aug-PE67.370.2480.625
SelfEvo-PE60.170.2790.670

SelfEvo-PE also outperforms Aug-PE. This confirms that the gain of SelfEvo-PE is from the better noise reduction and selection performance, regardless of the LLM backbone. We added this new result to Appendix C.7.4.

[1] https://anonymous.4open.science/r/selfevo-pe-0133/hf_text_only_datasets.csv [2] https://machinelearning.apple.com/research/differential-privacy-aggregate-trends

Dear Reviewer,

Thank you very much for your time and helpful suggestions. We hope our response addresses your concerns. As we are approaching the rebuttal deadline, please let us know if you have any further questions or concerns.

Sincerely, Authors of SelfEvo-PE

Reviewer_UJRfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SelfEvo-PE, an inference-only framework for differentially private synthetic text generation. It reuses noisy voting histograms from previous Private Evolution rounds to train lightweight ranking evaluators. These evaluators guide additional evolution rounds without further private-data access. The method combines pairwise ranking, neighborhood features, evaluator ensembles, and diversity-preserving selection. Experiments on three datasets show improvements over Aug-PE, particularly with limited private data and strict privacy budgets.

Critical strengths

(1) Practical motivation and significance. Reusing already privatized supervision is a sensible way to improve the privacy–utility trade-off. The approach preserves black-box access to the generator and avoids expensive LLM fine-tuning, making it practically attractive. (2)Substantial empirical investigation. The evaluation covers multiple privacy budgets, datasets, data-size ratios, and an additional Qwen backbone. The appendix reports variability across three runs and examines selection strategies, diversity regularization, and evolution schedules. The authors also acknowledge settings where the advantage diminishes. (3)Generally clear presentation. The motivation, overview figure, and highlighted pseudocode communicate the central mechanism effectively. Separating inference-only and fine-tuning methods is helpful for interpreting practical constraints.

Critical weaknesses

(1) Section 3 uses the total noisy votes of neighboring samples as an input feature. However, newly generated populations in self-evolving rounds have no fresh voting histogram. The paper does not clearly specify whether neighbors come from a historical archive, how their votes are combined across rounds, or how training and inference features remain consistent. This omission affects reproducibility and verification of the post-processing claim. (2) Algorithm 2 computes class-dependent sample counts from exact private class proportions, while the proof states that voting is the only private-data access. The algorithm explicitly assumes these proportions are public, but this assumption should also appear in the theorem, with a precise treatment of neighboring datasets and public metadata. Hyperparameter selection using private-data utility measurements also requires clarification. These are gaps in the end-to-end specification rather than evidence that post-processing itself is invalid.

Questions for authors

(1) How exactly are neighborhood-vote features constructed for unseen candidates without new private votes? (2) Can you state the complete privacy assumptions, including public class proportions, dataset size, and hyperparameter selection? (3) How much improvement remains against simple historical-vote predictors under matched generation budgets and selection controls? (4) Are ranking pairs restricted to the same class and iteration, given that vote counts depend on population composition? (5) Can you provide stronger evidence of downstream utility and evaluator generalization across iterations, beyond distributional similarity metrics?

3 comments
AuthorsOpenReview ↗

Response to Reviewer UJRf (1/2)

We sincerely thank the reviewer for their time and valuable feedback!

Weakness 1 / Question 1 Neighborhood features.

As we presented in the paper, the vote count of a sample depends on both the sample quality itself and how crowded its neighborhood is. Our density-aware features aim to capture the neighborhood information. The density-aware features include:

  • the distance from the sample to its k-nearest neighbors
  • the number of neighbors (not their votes) within a global radius.

We fixed the description in Sec. 3.

Weakness 2 / Question 2 Complete privacy statement.

We have moved the class proportions to be an input to the SelfEvo-PE algorithm. We also added a remark after the theorem (in Appendix A.1) to suggest the theorem can extend to cover the case where the class proportions are required to be protected. In our experiments, we assume the class proportions are public and we do not account privacy budget for hyperparameter tuning. We would like to emphasize that all the methods include the baselines and SelfEvo-PE are implemented with these two assumptions. Therefore, our comparison is fair. Additionally, the two assumptions follow the well-established setting and are widely used in the DP synthetic data literature [1-7]

Question 3 Simple historical-vote predictors

In Section 4.2.2, we presented the performance of selection with different evaluator designs, including a vote-count-prediction model (named "Regression"). Our proposed evaluator can select subsets that carry more clean votes compared to simple historical-vote predictors. Additionally, we would like to emphasize that our evaluator design is already simple and computationally efficient. In particular, our evaluator was implemented with the XGBoost library [8], which is widely used in industry.

Question 4: Are ranking pairs restricted to the same class and iteration?

Yes, we form pairs only inside each (class, voting round) block because the vote counts are only comparable within the same population.


[1] Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe (ACL'23)

[2] Differentially Private Synthetic Data via Foundation Model APIs 1: Images (ICLR'24)

[3] Differentially Private Knowledge Distillation via Synthetic Text Generation (ACL'24)

[4] Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains (ACL'24)

[5] Differentially Private Synthetic Data via Foundation Model APIs 2: Text (ICML'24)

[6] SynBench: A Benchmark for Differentially Private Text Generation (arXiv'25)

[7] Differentially Private Synthetic Data via APIs 4: Tabular Data. (ICML'26)

[8] https://xgboost.readthedocs.io/en/stable/index.html

AuthorsOpenReview ↗

Response to Reviewer UJRf (2/2)

Question 5: Downstream utility and evaluator generalization

Downstream utility

In the main paper, SelfEvo-PE has the best classification accuracy among inference-only methods in 9/9 settings of Table 1. We provide additional results for next token prediction tasks.

We present additional results on next-token predictions tasks. We consider two pipeline settings: (1) train from scratch, (2) fine-tune from a pre-trained model. We train the model on the synthetic datasets and test on the real samples. The results are shown in the table below. We can see that SelfEvo-PE outperforms Aug-PE in most cases for both settings. We added these new results to Appendix C.3.

Table 1 Training GPT-2 from scratch

DatasetTraining dataε = 0.2ε = 0.5ε = 1.0
BioRxivNo training0.000.000.00
Aug-PE10.76 ±\pm 0.1011.10 ±\pm 0.3511.37 ±\pm 0.61
SelfEvo-PE11.90 ±\pm 0.6112.14 ±\pm 0.2712.26 ±\pm 0.63
Private, no DP17.62 ±\pm 0.1117.62 ±\pm 0.1117.62 ±\pm 0.11
Congressional SpeechNo training0.010.010.01
Aug-PE9.60 ±\pm 0.129.30 ±\pm 0.229.26 ±\pm 0.86
SelfEvo-PE9.25 ±\pm 0.419.46 ±\pm 0.099.60 ±\pm 0.07
Private, no DP13.09 ±\pm 0.2413.09 ±\pm 0.2413.09 ±\pm 0.24
OpenReviewNo training0.710.710.71
Aug-PE13.40 ±\pm 0.6714.00 ±\pm 0.2313.84 ±\pm 0.11
SelfEvo-PE13.79 ±\pm 0.0514.11 ±\pm 0.0514.19 ±\pm 0.12
Private, no DP20.89 ±\pm 0.0620.89 ±\pm 0.0620.89 ±\pm 0.06

Table 2 Fine-tuning GPT-2

DatasetTraining dataε = 0.2ε = 0.5ε = 1.0
BioRxivNo training35.61535.61535.615
Aug-PE35.610 ±\pm 0.00535.613 ±\pm 0.01235.636 ±\pm 0.011
SelfEvo-PE35.633 ±\pm 0.00935.640 ±\pm 0.00935.666 ±\pm 0.002
Private, no DP37.725 ±\pm 0.04137.725 ±\pm 0.04137.725 ±\pm 0.041
Congressional SpeechNo training33.09233.09233.092
Aug-PE33.112 ±\pm 0.01233.132 ±\pm 0.01133.119 ±\pm 0.024
SelfEvo-PE33.138 ±\pm 0.01333.130 ±\pm 0.00733.132 ±\pm 0.028
Private, no DP34.613 ±\pm 0.03534.613 ±\pm 0.03534.613 ±\pm 0.035
OpenReviewNo training33.87633.87633.876
Aug-PE33.962 ±\pm 0.07834.069 ±\pm 0.09734.084 ±\pm 0.053
SelfEvo-PE33.936 ±\pm 0.14034.085 ±\pm 0.04434.116 ±\pm 0.084
Private, no DP37.595 ±\pm 0.01637.595 ±\pm 0.01637.595 ±\pm 0.016

Evaluator generalization

By design, the evaluator is not needed to generalize across several iterations. By default (Table 1), we only configure τevo=1\tau_{evo} = 1, which means that the evaluator is only used for one self-evolving iteration. After that, the evaluator is discarded and re-trained with the newly released votes.

Furthermore, in Figure 17, Appendix C.3, we vary τevo\tau_{evo} from 1 to 4. Each extra self-evolving step moves the population further toward what the evaluator has learned from the released votes. Even with τevo=4\tau_{evo} = 4, the performance of SelfEvo-PE is still stable and better than Aug-PE.

Additionally, we have evaluated the evaluator selection performance in Sec. 4.2.2, where the proposed LTR can select subsets that carry the most clean votes.

Dear Reviewer,

Thank you very much for your time and helpful suggestions. We hope our response addresses your concerns. As we are approaching the rebuttal deadline, please let us know if you have any further questions or concerns.

Sincerely, Authors of SelfEvo-PE

Reviewer_HQrSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SelfEvo-PE, an extension of Private Evolution (Aug-PE) for DP synthetic text. Standard PE throws away each iteration's noisy nearest-neighbor vote histogram once it has been used. SelfEvo-PE keeps the sample noisy vote pairs and trains a lightweight evaluator (ensemble of XGBoost rankers) on them. The trained evaluator is used to rank samples during the evolution rounds without consuming the privacy budget. As a result, SelfEvo-PE allows more iterations for a given privacy budget, or noise reduction for a given number of iterations.

To increase diversity, SelfEvo-PE adds two strategies during sample selection phase: 1) a KNN-distance term in addition to the evaluator’s score; and 2) top-K early on and Top1PerFamily later. Experiments over 3 datasets show that SelfEvo-PE gives gains in the small-data and strict-privacy regimes.

Critical strengths
  1. Simple idea and low overhead. This method is a simple add-on to PE algorithm. It is orthogonal to other variants like metadata-based or RL, with about 2% extra wall-clock time (Fig. 5).
  2. Design choices supported by reasonable ablations. Various sampling strategies and evaluator training strategies are stidufied in Sec. 4 and appendix.
  3. Limitations and reproducibility. Studies of SNR in Sec. 4 and larger epsilon regimes in appendix shows the gains are confined to small-data, strict-privacy regimes, which is useful guidance for practitioners. Code is released (anonymous repo).
Critical weaknesses
  1. Unclear how much gain comes from the learned evaluator and whether other simpler evaluators can be used. Two types of baselines are worth comparison:

    • Aug-PE + Diversity-regularized score + Top1PerFamily selection: this helps isolate the gain from a better selection method v.s. a learned evaluator in Table 1.
    • Other evaluator options: keep the same schedule and noise as SelfEvo-PE, but replace the learned evaluator with (i) a constant (pure diversity selection), (ii) random selection, and (iii) a trivial evaluator, e.g. each child inherits its parent's noisy vote (plus diversity score), or a kNN-regression of the previous votes (i.e., each example gets votes by summing its k-nearest neighbors’ previous votes) plus diversity score. This helps justify if the proposed XGBoost/LTR machinery is necessary.
  2. Congressional Speech accuracy looks uninformative. In Table 1, despite different MAUVE and FID metrics, Aug-PE, ICL-PE, CTCL and DP-FT all report exactly 54.300. This looks like the majority-class rate, and if so, reporting the majority prediction baseline is useful. POPri reports very low accuracy 31.233 despite good MAUVE score, which is strange. Please explain. Besides, Appx. B.2 mentions that POPri was run for 10 rounds instead of 30 and does not tune hyperparameters, which may hinder its performance.

  3. Limitation on downstream task types. The experiments only cover the classification tasks not the next token prediction tasks. This may disadvantage fine-tuning-based methods. Clearly stating this limitation or adding next token prediction experiments can help guide practitioners in understanding and choosing different methods.

  4. Missing comparison of API calls. Appx. B.2 says "All the PE methods share the same hyperparameters (number of variations, sampling configurations) and prompt designs…" However, it is unclear whether SelfEvo-PE and Aug-PE use the same total number of iterations in Table 1: if yes, then their API cost is equal; and if no, it is worth mentioning in the table and show additional comparison given the same API budget.

  5. The motivating observation may come from same selected examples. Fig. 2 compares the top-5% of iteration t−1 with the selected/unselected samples of iteration t. But the iteration-t population consists of the selected samples of t−1. The absolute gap is also small (e.g., 0.05 vs. 0.045). A more convincing motivation would control for parentage (compare siblings, or exclude each sample's own parent from the reference set). If most of the gap comes from the existing parent examples, then using additional evaluator to rank seems not necessary (connected to the #1 weakness on comparing w/ other evaluator options).

Questions for authors
  1. Weakness #1: can we isolate the gain from the learned evaluator and the diversity selection? do we have to use XGBoost/LTR and does other simpler method also work here?
  2. Weakness #2: can we understand what's happening for the Congressional Speech accuracy?
  3. Weakness #3: does the proposed method work for next token prediction tasks?
  4. Weakness #4: does the proposed method need more API calls?
  5. Weakness #5: does the overlap of top selected examples b/t two iterations come from the same selected examples?
6 comments
AuthorsOpenReview ↗

Response to Reviewer HQrS (1/n)

We sincerely thank the reviewer for their time and the helpful suggestions.

Weakness 1: Performance gain from learned evaluator

In Section 4.2.2, we already provided a study on selection performance of different evaluator designs. Our proposed evaluator can select subsets that carry more clean votes compared to the other evaluator designs. We would like to emphasize that our evaluator design is also quite simple and computationally efficient. In particular, our evaluator was implemented with the XGBoost library [1], which is widely used in industry.

Thank you the reviewer again for the suggestion. We provide additional results on the BioRxiv dataset. In particular, we compare SelfEvo-PE with Aug-PE + diversity + Top1PerFamily, which uses the same selection strategy of SelfEvo-PE but without self-evolving iterations. We also consider SelfEvo-PE with random selection that does not employ an evaluator for the self-evolving iterations.

MethodFIDMAUVEAcc
Aug-PE41.14 ±\pm 1.230.609 ±\pm 0.0980.704 ±\pm 0.010
Aug-PE w/ DR + Top1PerFamily38.14 ±\pm 0.960.510 ±\pm 0.0080.710 ±\pm 0.006
SelfEvo-PE w/ random selection41.65 ±\pm 0.500.534 ±\pm 0.0530.701 ±\pm 0.009
SelfEvo-PE w/ evaluator selection (default)34.60 ±\pm 0.730.711 ±\pm 0.0260.712 ±\pm 0.002

We can see that DR and Top1PerFamily does not close the gap between Aug-PE and SelfEvo-PE. SelfEvo-PE w/ random selection is not better than Aug-PE because selecting wrong samples actually hurts the evolution process. These results together confirm that the performance gain is significantly from self-evolving iterations with the learned evaluators. We added this new result to Appendix C.6.1.

Weakness 2: Congressional Speech accuracy

The value of 54.300 is the majority-class prediction accuracy. This setting is challenging because the number of private samples is small (1K samples). The downstream classifier is trained on only 250 synthetic samples (25% of the private samples). In most cases, only SelfEvo-PE is able to generate synthetic samples with classification signal that transfers to real text. The other methods fall back to the class distribution prior and predict the majority class. In Appendix C.7.1., we present additional results on this dataset with a looser privacy budget at ϵ=4.0\epsilon=4.0. In this case, some methods can perform better than the majority-class prediction accuracy.

Regarding POPri, we would like to emphasize that POPri is a fine-tuning method which is not a direct baseline of our inference-only method. POPri is designed for the larger scale settings with the dataset size of 72K–133K (Hou et al., 2025, Table 3), which is far from our 1K–3K regime. Under the small-data and strict-privacy regime, POPri does not exhibit good performance, its performance is mostly not better than DP-FT. This is consistent with the findings in [2]. Thank you very much for catching the accuracy of POPri. POPri also produces classifiers that predict the majority class. There was a bug of encoding that caused mislabeling of the class index in our implementation of POPri evaluation for this dataset. This led to the reported accuracy is another class count which is not the majority class. We corrected the numbers in the updated version.


[1] https://xgboost.readthedocs.io/en/stable/index.html

[2] Zou et al. PE-SGD: Differentially Private Deep Learning via Evolution of Gradient Subspace for Text. ICLR'26.

AuthorsOpenReview ↗

Response to Reviewer HQrS (2/n)

Weakness 3: Next-token prediction

We present additional results on next-token predictions tasks. We consider two pipeline settings: (1) train from scratch, (2) fine-tune from a pre-trained model. We train the model on the synthetic datasets and test on the real samples. The results are shown in the table below. We can see that SelfEvo-PE outperforms Aug-PE in most cases for both settings. We added these new results to Appendix C.3.

Table 1 Training GPT-2 from scratch

DatasetTraining dataε = 0.2ε = 0.5ε = 1.0
BioRxivNo training0.000.000.00
Aug-PE10.76 ±\pm 0.1011.10 ±\pm 0.3511.37 ±\pm 0.61
SelfEvo-PE11.90 ±\pm 0.6112.14 ±\pm 0.2712.26 ±\pm 0.63
Private, no DP17.62 ±\pm 0.1117.62 ±\pm 0.1117.62 ±\pm 0.11
Congressional SpeechNo training0.010.010.01
Aug-PE9.60 ±\pm 0.129.30 ±\pm 0.229.26 ±\pm 0.86
SelfEvo-PE9.25 ±\pm 0.419.46 ±\pm 0.099.60 ±\pm 0.07
Private, no DP13.09 ±\pm 0.2413.09 ±\pm 0.2413.09 ±\pm 0.24
OpenReviewNo training0.710.710.71
Aug-PE13.40 ±\pm 0.6714.00 ±\pm 0.2313.84 ±\pm 0.11
SelfEvo-PE13.79 ±\pm 0.0514.11 ±\pm 0.0514.19 ±\pm 0.12
Private, no DP20.89 ±\pm 0.0620.89 ±\pm 0.0620.89 ±\pm 0.06

Table 2 Fine-tuning GPT-2

DatasetTraining dataε = 0.2ε = 0.5ε = 1.0
BioRxivNo training35.61535.61535.615
Aug-PE35.610 ±\pm 0.00535.613 ±\pm 0.01235.636 ±\pm 0.011
SelfEvo-PE35.633 ±\pm 0.00935.640 ±\pm 0.00935.666 ±\pm 0.002
Private, no DP37.725 ±\pm 0.04137.725 ±\pm 0.04137.725 ±\pm 0.041
Congressional SpeechNo training33.09233.09233.092
Aug-PE33.112 ±\pm 0.01233.132 ±\pm 0.01133.119 ±\pm 0.024
SelfEvo-PE33.138 ±\pm 0.01333.130 ±\pm 0.00733.132 ±\pm 0.028
Private, no DP34.613 ±\pm 0.03534.613 ±\pm 0.03534.613 ±\pm 0.035
OpenReviewNo training33.87633.87633.876
Aug-PE33.962 ±\pm 0.07834.069 ±\pm 0.09734.084 ±\pm 0.053
SelfEvo-PE33.936 ±\pm 0.14034.085 ±\pm 0.04434.116 ±\pm 0.084
Private, no DP37.595 ±\pm 0.01637.595 ±\pm 0.01637.595 ±\pm 0.016
AuthorsOpenReview ↗

Response to Reviewer HQrS (n/n)

Weakness 4: API calls

We confirm that Aug-PE and SelfEvo-PE are at equal API budget. Aug-PE and SelfEvo-PE use the same number of iterations, the same number of candidates per iteration, and the same prompts, so the total number of API calls is the same. ICL-PE also uses the same number of iterations and candidates but with longer prompts because it includes the in-context examples.

Weakness 5: Motivating observation

We calculate the Jaccard similarity from the top 5% samples of iteration t−1t-1 to the selected and unselected samples of iteration tt for both cases: (1) the full population of iteration tt; (2) the subset of iteration tt that excludes the parent samples from iteration t−1t-1.

The reported values are the average similarity across all iterations of BioRxiv.

population ttselectedunselecedgap
full0.05100.04060.0104
excl. parent0.04840.04030.0081

The results show that there is still a gap between the selected and unselected samples even after excluding the parent samples. This indicates that the intuition is still valid with unseen samples.

Additionally, we would like to emphasize that the small "absolute gap" is not a concern, as the selection process is a relative ranking problem. Also, the gap is quite significant if we consider the relative gap: (0.0510-0.0406)/0.0406 = 25.6%; (0.0484 - 0.0403)/0.0403 = 20.1%.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.