acceptodds
Under review as a conference paper at ICLR 2027

Agentic Private Evolution: Differentially Private Synthesis of Agent Workspace Data

Abstract

Evaluating enterprise agents requires realistic workspace data, but private email threads, conversations, and documents cannot be freely shared. Private Evolution (PE) offers differentially private (DP) synthesis without training the generator. Applying PE to workspace data, however, presents three challenges: independent generation lacks shared context whose rules vary across data types; semantic similarity does not ensure structural validity; and vote-based selection can lose coverage of requested sample types. To address these challenges, we propose Agentic Private Evolution (Agentic-PE), a framework for DP synthesis of agent environment data. Starting from a DP attribute table that specifies the samples to generate, Agentic-PE introduces three complementary modules: a constraint module provides shared participants and data-type-specific context rules; a validation agent checks and repairs candidates before private voting; and constrained selection retains one feasible candidate per table row to preserve coverage. The modules operate on DP outputs and synthetic samples, adding no privacy cost. On email at a matched privacy budget, Agentic-PE achieves 6.4 times Aug-PE’s semantic recall at lower Fréchet Inception Distance (FID) and raises structural validity from 62.8% to 93.1%. Its corpus improves downstream training over a DP fine-tuning baseline and yields positively correlated retrieval and reply-model rankings on real and synthetic benchmarks. The framework also improves recall and FID over matched-budget Aug-PE on chat, meetings, and internal documents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Differential Privacy, Synthetic Data Generation, Private Evolution, Large Language Models, Agent Evaluation, Workspace Data

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_5BQaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Agentic Private Evolution for differentially private synthesis of structured workspace data such as email, chat, meetings, and documents. It extends Private Evolution with three components: a shared constraint module for maintaining consistent participants and context, a validation-and-repair agent for enforcing structural correctness, and constrained selection that preserves coverage of the released attribute table. The method is evaluated primarily on email and additionally on three other workspace modalities, using semantic fidelity, structural validity, downstream fine-tuning utility, and agreement between model/retrieval rankings on real and synthetic benchmarks.

Critical strengths
  1. The paper addresses an important limitation of existing DP text synthesis: sample-level semantic similarity is insufficient for structured agent environments. And the resulting improvements are substantial in benchmarks.
  2. The empirical validation goes beyond intrinsic similarity metrics and tests practical downstream utility. The paper includes matched-budget ablations, downstream fine-tuning on held-out real email, real-vs.-synthetic system-ranking comparisons, and transfer to chat, meetings, and documents.
Critical weaknesses
  1. Missing comparison with MAPLE. MAPLE is the most closely related prior work and the paper's method is based on MAPLE. The paper itself identifies it as particularly relevant. Without a matched-budget comparison, the empirical novelty of Agentic-PE is difficult to assess.
  2. Gap between theory and implementation lies on idealized assumptions such as Gaussian multi-scale variations and retained parents. The actual system uses LLM variations and does not retain parents the paper section 3 claims.
  3. Methodological novelty is one of my concern, as the paper's method seems like combining MAPLE + AugPE together with some tricks, which leads less novelty credit.
Questions for authors
  1. Could the authors provide a matched-privacy-budget comparison against MAPLE? This would materially affect my assessment of whether Agentic-PE provides a substantial improvement over the closest prior method rather than primarily over the older Aug-PE baseline.
  2. Could the authors clarify which parts of Theorem 2 are intended to apply to the implemented Agentic-PE system, given the differences in variation, parent retention, and admissibility assumptions? If the theorem is only intended as an analysis of an idealized variant, the claims should be scoped accordingly.
1 comment
AuthorsOpenReview ↗

Response to reviewer 5BQa

Thank you for focusing on the two points that matter most for the paper's claims.

Q1. Comparison with MAPLE

Yes. At the same privacy budget, Agentic-PE improves substantially on a MAPLE-style baseline as well as on Aug-PE.

Setup. We ran the requested comparison at the same privacy budget as Table 1.

  • Pipeline. A DP release of the attribute marginals, an initial population conditioned on it with public exemplars, then the plain Aug-PE loop.
  • Difference from MAPLE. It uses one Gaussian marginal release in place of AIM.
  • Held fixed. The generator, voting representation and candidate budget are the same as in Table 1.
Corpus (ten-round means)PRFIDValidityContradictory pairs
MAPLE-style baseline0.5920.2920.05160.6266.0%
Agentic-PE (full)0.5920.4120.04550.9311.6%
  • Semantic and structural metrics. At equal precision, Agentic-PE's recall is 1.4× the baseline's (0.412 vs. 0.292). FID improves from 0.0516 to 0.0455, and validity from 0.626 to 0.931.
  • Over the rounds. The baseline's recall falls from 0.353 in the first round to 0.263 in the last, while Agentic-PE's rises from 0.352 to 0.414. Agentic-PE uses P in every round; MAPLE uses its metadata only to initialize.
  • Contradictions. Under an independent judge, contradictions between threads drop from 6.0% to 1.6% of pairs.
  • Source of the structural gain. Most of it comes from the constraint module. Our response to YTkJ Q1 gives an arm with the module but without validation.

You noted that this comparison would affect your assessment. We hope these results show that Agentic-PE improves substantially on the closest prior method. In the revision, we will add this baseline to Table 1, report it next to the Aug-PE comparison in the abstract, and also run MAPLE with its AIM-based release.

Q2. Which guarantees apply to the implemented system

The guarantees the method relies on in practice, privacy and row coverage, hold for the implemented system. Theorem 2 analyzes an idealized variant. Theorem 2 assumes multiscale Gaussian variations, retained parents and oracle admissibility. The implementation uses two language-model variations without retained parents.

StatementApplies to the implemented system?
Proposition 1 (privacy)Yes. It needs only the DP releases and post-processing.
Proposition 2 (row coverage)Yes, in every round where each row has an admissible candidate. In the full arm, every row kept a survivor in every round (Figure 2a).
Theorem 2 (convergence)No. It is a utility analysis of the idealized variant under (M1)–(M6).

The implemented system moves in the direction the idealized analysis describes. From the first round to the last, recall rises from 0.352 to 0.414 and FID falls from 0.0493 to 0.0456. We will state the theorem's scope in the main text where §3.2 introduces the theory.

Thanks again for your review!

Reviewer_m77fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies DP synthetic enterprise workspace data, e.g., email threads, conversations, and documents. The authors identify three challenges when directly applying PE: independent generation lacks shared context whose rules vary across data types; semantic similarity does not ensure structural validity; and vote-based selection can lose coverage of requested sample types. To address these issues, they propose Agentic-PE, which builds upon the PE line of work. The phase A privately releases a DP attribute table that specifies the samples to generate and design a constraint module provides shared participants and data-type-specific context rules; and in Phase B, the method runs PE-style evolution with two additional modules: a validation agent checks and repairs candidates before private voting, and constrained selection retains one feasible candidate per table row to preserve coverage. . The method significantly outperforms AugPE in terms of validity and recall.

Critical strengths

The paper tackles an important problem of DP synthetic data for workspace data, which is essential for agent trainings. It extends the PE series on images, text, and tabular data to a more important setting where shared context is critical.

Critical weaknesses
  1. The presentation of section 3.1 and 3.2 is really hard to follow. I tried to internalize it as much as I could, and I would like to make sure I'm reading it correctly: phase A is essentially constructing MAPLE-style metadata (with other agentic task specific modifications) and phase B is the AugPE style evolution with task specific engineering (like the validity checks and voting details)
  • If I understand it correctly, I wonder why you chose tabPE instead of AIM, since AIM is likely to be cheaper right? Also, I wonder if there are any measurements of the quality of PP, as this may be important for the future generation step.
  1. In line 256-257 you stated, "We fix (ϵ,δ)(\epsilon,\delta) and solve for a shared σ\sigma". However, your choices of ϵ\epsilon being 5.2985 and 5.293 suggest to me that you didn't calibrate the noise multiplier with the privacy budget, but rather that you fix a noise multiplier up front and then solve for the ϵ\epsilon. Is this correct? I did some simple calculations manually and I still don't fully understand how the numbers were computed. I would like to see more details on the exact accounting of these steps

  2. I'm a bit curious on the DP-SGD recipe. You stated "100 Poisson-subsampled steps" on 2k examples, but didn't state what the sampling ratio is. To me, the outperformance in recall and fid could be due to the undertuned recipe (or insufficient amount of data.) More importantly, DP-SGD trained model being worse than untrained model in "valid" (and their precisions being exactly the same) suggests that there may be something wrong with the recipe.

  3. There are lots of inconsistencies in the reported numbers, for example,

  • in Tab 1, the mean precisions of "Agentic-PE (full)" and "Agentic-PE, random selection" are 0.592 and 0.582, but in Fig 6, the means are 0.657 and 0.582. The recall scores of Tab 1 do not match with that in Fig 6 either. Is there a difference in the setup?
  • the test loss in fig 2b and Tab 3 don't match up. Is there a difference in the setup?
  1. You cited [1] for tabPE, which should be [2]

[1] Differentially private synthetic data via foundation model APIs 1: Images

[2] Differentially Private Synthetic Data via APIs 4: Tabular Data

Questions for authors

See above.

2 comments
AuthorsOpenReview ↗

Response to reviewer m77f (1)

Thank you for the close reading and comments. We will address your concerns step by step:

Q1. The two phases, Tabular PE, and the quality of P

Your reading is correct, with one key difference from MAPLE: Agentic-PE uses P in every round.

  • Phase A. It releases the attribute table P with Tabular PE, then builds the constraint module from P.
  • Phase B. It is Aug-PE-style evolution with three changes: validation before voting, one survivor per row of P, and the module's shared context in every prompt.
  • Difference from MAPLE. P is used in every round, while MAPLE uses its metadata only to initialize.

This difference matters in practice. Against a MAPLE-style baseline at the same budget, Agentic-PE's recall is 1.4× higher, and it keeps rising over the rounds while the baseline's falls (our response to YTkJ Q1). We will rewrite §3.1–3.2 around one running example, and add a notation table.

Tabular PE is the cheaper option. Tran et al. [1] report that it runs 28× faster than AIM. We will discuss the choice of release mechanism in the revision.

P carries most of the attribute structure. We compared P with the private attribute table, using total-variation distance averaged over all 1-way and 2-way marginals (lower is better):

Table1-way TVD2-way TVD
400 rows sampled from the private table (sampling floor)0.0280.063
Released P (Tabular PE, 15 steps)0.1420.242
Uniform rows0.4820.647
  • Measured gap. From uniform rows to the sampling floor, P closes 75% of the gap on 1-way marginals and 69% on 2-way marginals.
  • Effect on generation. P is useful for the generation step: conditioning on it lifts recall from 0.064 (Aug-PE) to 0.412 (Table 1).

Q2. Exact privacy accounting

Yes, your reading is correct. Every PE arm spends the same budget, and exact accounting puts it at ε = 4.38, below the 5.2985 stated in the paper.

  • How the level was set. We fixed the effective noise level first, σ/√N = 1. Each arm then uses σ = √N for its own number of Gaussian steps N, so every arm spends the same budget. The level was chosen independently of the data, so the guarantee is unaffected. We will rewrite lines 256–257 to say this.
  • Exact accounting. N Gaussian mechanisms with noise σ compose exactly to μ-GDP with μ = √N/σ = 1, which gives ε = 4.38 at δ = 10⁻⁵. The Private Evolution library uses this GDP accounting, and the revision will report it for every PE arm.
  • The paper's 5.2985. It is the looser RDP bound, ε = min over α of [Nα / (2σ²) + log(1/δ) / (α − 1)], minimized at order 5.8.
ConfigurationGaussian steps Nσσ/√Nε (GDP)ε (RDP, in the paper)
Full email (1 pre-release + 15 release + 10 votes)265.0990221.0004.385.2985233
Random-selection arm (no votes)164.0000221.0004.385.2984942
Seven-attribute arm255.0000001.0004.385.2985261
Budget-matched Aug-PE103.1623001.0004.385.2984852
  • DP-SGD. It uses the PRV accountant for Poisson-subsampled Gaussian steps; with σ = 1.3818359375 its ε is 5.2931. With GDP accounting for the PE arms, DP-SGD therefore has the larger budget in our comparison (5.29 vs. 4.38).
  • Revision. We will list steps, σ and accountant for every configuration.

Reference

[1] Tran et al. Differentially Private Synthetic Data via APIs 4: Tabular Data. ICML 2026.

AuthorsOpenReview ↗

Response to reviewer m77f (2)

(Continued from part 1.)

Q3. DP-SGD sampling, tuning, and the validity of its output

The DP-SGD recipe was tuned and works as intended. Its weaker results come from the noise DP training needs on 2,000 records, not from an untuned recipe.

Sampling. The sampling probability is q = 256/2,000 = 0.128, so each record appears in 12.8 of the 100 updates in expectation.

ParameterValue
ModelQwen2.5-7B-Instruct, LoRA r = 8 (no lm_head adapter), chat template with a format-describing prompt
SamplingPoisson, q = 0.128, expected batch 256, 100 updates
Clipping / noiseC = 1.0, σ = 1.3818359375
OptimizerAdamW, lr 2e-4, cosine schedule, 10 warmup steps
Length2,048 tokens; 332 of 2,000 records truncated

Tuning. We first developed the baseline on the smaller Qwen2.5-1.5B, where each run is cheaper, and moved to Qwen2.5-7B-Instruct for Table 2 once the recipe worked. Every DP run below spends the same budget (ε ≤ 5.299). The measure is the share of samples that are well-formed email threads.

RunWhat it testsWell-formed threads
Qwen2.5-1.5B basestarting recipe0.75%
Same recipe, DP offwhether the recipe can learn the format91%
1.5B base, expected batch 512 (from 256)larger batch2.0%
1.5B base, learning rate 1e-3 (from 2e-4)higher learning rate4.0%
Qwen2.5-1.5B-Instruct, format promptinstruct model, prompt that describes the format17%
Qwen2.5-7B-Instruct, format prompt (Table 2)larger instruct model99%
  • The recipe can learn the format. With DP off, the same 1.5B recipe reaches 91%, so the low rate under DP comes from the noise.
  • What helped. At a fixed ε, a larger batch forces a larger σ and helps little, and a higher learning rate helps a little. Moving the format into the model and the prompt helps most, and costs no privacy budget.
  • 7B. The untrained 7B-Instruct model with the same prompt also writes well-formed threads (99%), so structure no longer separates the runs. The useful comparison there is semantic; we give it below.
  • Clipping. On 7B, the median per-example gradient norm is about 0.30, so C = 1.0 clips only 1.25% of examples. On Qwen3-4B, a smaller clipping norm (C = 0.3 with lr 6e-4) lowered the share of well-formed threads from 1.00 to 0.73.

Neither the validity nor the precision result signals a problem with the recipe.

  • Semantics. On 7B, DP training moves the output slightly closer to the private corpus: recall 0.183 vs. 0.140, FID 0.0839 vs. 0.0900.
  • Surface quality. More unfilled placeholders in the DP output (3.0% vs. 1.0% in our probe) are one reason for its lower structural validity (0.555 vs. 0.682).
  • Identical precision. Both corpora place 353 of 400 samples inside the estimated private manifold, so precision is 0.8825 for both. The two corpora share no identical texts.

The comparison with DP-SGD in the paper therefore stands. We will report these probes in the appendix.

Q4a. Table 1 vs. Figure 6

Both sets of numbers are correct; the two exhibits compare different arms. Figure 6 compares the greedy vote-ranked arm (precision 0.657) with random selection (0.582). These two arms differ in two ways: whether votes are used, and whether one candidate per row is kept (random selection keeps one per row; greedy does not). The matched comparison is the full method against random selection, since both keep one candidate per row:

ArmPRFID
Full (constrained, uses votes)0.5920.4120.0455
Random selection (constrained, no votes)0.5820.3350.0471

The conclusion of §4.2 holds on this matched pair: private votes add 0.077 recall and 0.010 precision. We will base §4.2 and Figure 6 on this pair, and label the greedy arm separately.

Q4b. Figure 2b vs. Table 3

Both are correct; they use different downstream models. Figure 2b uses Qwen2.5-1.5B and Table 3 uses Qwen2.5-7B-Instruct, both evaluated on the same 174 held-out real threads. We will name the model in each caption.

Q5. Tabular PE citation

Thank you. In §3.3 and Appendix B, we will cite Tran et al. [1] for Tabular PE. Lin et al. (2024) remains the citation for Private Evolution.

We hope these answers resolve the questions about presentation and accounting.

Reference

[1] Tran et al. Differentially Private Synthetic Data via APIs 4: Tabular Data. ICML 2026.

Reviewer_YTkJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Agentic-PE extends Private Evolution to structured workspace data such as email, chat, meeting records, and documents. A differentially private attribute table supplies generation targets. A shared fictional context, validation and repair, and constrained selection aim to improve consistency and preserve coverage during evolution. The privacy accounting combines attribute releases and noisy similarity votes; the remaining stages operate on their outputs. Experiments report improved semantic coverage and structural validity, downstream email-training benefits, and positive agreement between system rankings on real and synthetic corpora.

Critical strengths
  1. The paper addresses a practical gap in private text synthesis. A collection of individually plausible records can still fail as a workspace if participants, reply structure, or facts are inconsistent. The shared context and validation mechanism directly address this issue. Table 1 shows a meaningful joint validity improvement from 0.786 to 0.931 when these components are added to the nine-attribute configuration.
  2. The privacy analysis pays attention to a subtle dependency: a sender-frequency attribute depends on other records. Appendix C handles this through a separately accounted noisy frequency release and conditional sensitivity. The evaluation also goes beyond embedding similarity by examining downstream training and system rankings. These are useful choices for this application.
Critical weaknesses
  1. The comparison does not establish the incremental contribution over a strong metadata-conditioned PE pipeline. MAPLE is identified as particularly relevant, but is not evaluated. In Table 1, the nine-attribute variant without the constraint module already reaches recall 0.391, compared with 0.412 for the full method and 0.064 for Aug-PE. The structural gains are meaningful, but the headline 6.4-fold recall improvement does not show how much comes from the proposed modules.
  2. A matched comparison should control the metadata, generator, voting representation, and repair/selection budget. Shared context and validation are currently varied together, so their respective contributions also remain unclear. The evidence for using the synthetic corpus as a substitute evaluation benchmark is still limited. Reply-model ranking uses only ten real and ten synthetic prefixes; Appendix I reports a correlation of 0.733 with a wide 95% interval of [0.006, 0.843]. Three judges give positive correlations, but they rescore the same small set of replies. This addresses evaluator sensitivity without establishing stability across sampled tasks.
  3. The retrieval results add useful evidence, but neither ranking study compares against a simpler synthetic-data baseline. It therefore remains unclear whether the proposed workspace-consistency machinery is what preserves model rankings. This is important because reliable comparative evaluation is a main motivation for the method.
Questions for authors
  1. How does Agentic-PE compare with MAPLE, or a comparably strong metadata-conditioned PE baseline, using the same generator, attribute schema, voting representation, and generation budget? Which gains remain when shared context and validation are separated?
  2. How stable are the reply and retrieval rankings across independently sampled tasks and synthesis runs? Can you report ranking agreement for the no-module or simpler PE baselines as well as the full pipeline? This would directly test whether the proposed additions improve benchmark fidelity.
  3. Can you quantify consistency across records independently of the rules used during generation, or show its effect on retrieval questions that require combining related records? The deterministic validity scores establish useful properties, but do not by themselves establish the broader workspace-consistency benefit.
2 comments
AuthorsOpenReview ↗

Response to reviewer YTkJ (1)

Thank you for the detailed questions. We ran new experiments for each of them.

Q1. Comparison with MAPLE, and separating context from validation

Agentic-PE's gains hold against a MAPLE-style baseline at the same budget, and both the constraint module and validation contribute to them.

Setup. We ran two new arms. Both use the same privacy budget, generator, voting representation and candidate budget as Table 1.

  • MAPLE-style baseline. The attribute table is released with one Gaussian marginal release (in place of MAPLE's AIM). The initial population is conditioned on it with public exemplars. Evolution then follows the plain Aug-PE loop: top-voted selection, no module, no validation.
  • Module, no validation. This arm uses the full arm's released table, org graph and selection. Only the consistency audit, the repair and the plan-adherence regeneration are switched off.
Corpus (ten-round means)PRFIDValidity
Aug-PE0.5860.0640.10090.628
MAPLE-style baseline (new)0.5920.2920.05160.626
Metadata-conditioned, no module0.5460.3910.04720.786
Module, no validation (new)0.6310.3930.04310.921
Agentic-PE (full)0.5920.4120.04550.931

Against MAPLE, the gain does not come from attribute conditioning alone.

  • Overall. The 6.4× headline is measured against Aug-PE, which uses no attributes. The MAPLE-style baseline uses DP attributes too, and Agentic-PE still reaches 1.4× its recall (0.412 vs. 0.292) at equal precision. FID improves from 0.0516 to 0.0455, and validity from 0.626 to 0.931.
  • Over the rounds. The gap grows: the baseline's recall falls from 0.353 in the first round to 0.263 in the last, while Agentic-PE's rises from 0.352 to 0.414.
  • Why. This matches the main design difference from MAPLE. Agentic-PE keeps using P in every round, through its prompts and its per-row selection; MAPLE uses its metadata only to initialize.

The constraint module carries most of the structural gain, and validation adds to it.

  • Module without validation. It raises validity from 0.786 to 0.921 and precision from 0.546 to 0.631, and lowers FID from 0.0472 to 0.0431. Cross-record contradictions fall from 5.7% to 1.5% of thread pairs (Q3).
  • Adding validation. It raises the ten-round mean validity from 0.921 to 0.931 and recall from 0.393 to 0.412. The validity gain grows over the rounds: it is higher in every round from the fourth on, and in the last round it is 0.948 against 0.918.

Generation budget. All arms spend the same privacy budget; model calls differ.

  • What validation adds. One consistency-audit call per candidate. It also regenerates about 39% of candidates once or twice, when they miss their row's length bucket.
  • Total. The full arm makes roughly 1.8× the calls of the no-module arm.
  • A cheaper option. The module-only arm skips both steps and keeps most of the structural gain.

Coverage. Constrained selection is part of the method because it keeps coverage that vote-ranked selection loses. By Proposition 2, it keeps all 400 rows, while greedy selection with the same votes keeps 78. As a result, recall is 0.412 vs. 0.314, and the corpus has 81 vs. 41 distinct senders.

In the revision, we will:

  • add both arms to Table 1;
  • report the MAPLE-style comparison next to the Aug-PE headline in the abstract;
  • report call counts;
  • also run MAPLE with its AIM-based release.
AuthorsOpenReview ↗

Response to reviewer YTkJ (2)

(Continued from part 1.)

Q2. Stability of the rankings

The reply-model ranking stays stable when the task sample is tripled.

We repeated the study on the Table 1 corpus with 30 thread prefixes per side (the paper used 10) and the same four judges.

  • Models. Gemma-3-27B did not return a reply for every prefix. We therefore rank the nine models with complete replies, so both leaderboards are built on the same prefixes. The last column adds Gemma-3-27B, as a check.
  • What the table shows. Spearman ρ between the real and synthetic leaderboards, with 95% bootstrap intervals over prefixes.
Judgeρ (9 models)95% intervalSame top modelρ (10 models, with Gemma-3-27B)
GPT-5.4 (pre-registered; not a candidate)0.820[0.28, 0.93]No0.669
GPT-5.6 Sol0.933[0.57, 0.97]Yes0.915
Mistral Large 30.667[0.15, 0.88]Yes0.636
Kimi K2.60.500[−0.07, 0.82]Yes0.564
  • Agreement. Under the pre-registered judge, ρ is 0.820 with interval [0.28, 0.93]. With 10 prefixes per side, the interval was [0.006, 0.843]. Three of the four intervals now exclude zero.
  • Top models. They also agree closely. GPT-5.6 Sol is first on real prefixes and second on synthetic ones. Kimi K2.6 is first on synthetic prefixes and tied for second on real ones.
  • Primary judge. Two judges that are also candidates rank themselves first, so we treat GPT-5.4 as the primary judge.

In the revision, we will report these results with the full rankings.

Q3. Consistency measured independently of the generation rules

Under a judge that never saw our rules, Agentic-PE's corpora contain about a quarter of the cross-record contradictions of the arms without the module.

Independent contradiction audit. We asked a judge model that generated none of the data (GPT-5.4) to compare pairs of round-10 threads that share a participant. The judge does not see the roster or the module's rules.

  • What counts. The judge lists statements in one thread that cannot be true together with the other. Only four kinds count: a person's identity or role, the date of the same event, a number for the same item, or a decision on the same matter.
  • Sample. We sampled 200 pairs per corpus. For 13 of the 1,200 pairs the judge returned no usable answer; these are excluded.
CorpusPairs judgedPairs with a contradiction
Agentic-PE (full)1933 (1.6%)
Module, no validation2003 (1.5%)
No module19411 (5.7%)
MAPLE-style baseline20012 (6.0%)
Aug-PE2002 (1.0%)
Private reference20031 (15.5%)
  • Arms with the module. They have 6 contradictory pairs out of 393, against 23 out of 394 for the two arms without it (Fisher exact p = 0.002). In the arms without the module, most contradictions concern identity.
  • Aug-PE. Its threads rarely refer to the same people or events: 551 distinct senders, each in 2.2 threads on average. A sampled pair is usually linked by a single recurring name, so there is little to contradict.
  • Private reference. It has the most contradictions, all on identity: real mail contains people with several addresses or titles. The audit measures whether a corpus describes one organization consistently, which is the property the module is designed for.

Identity consistency in the text (round-10 corpora; private reference at the same size):

CorpusAddresses with >1 nameNames with >1 address
Aug-PE2.6%0.5%
No module4.4%5.1%
Full0.0%0.0%
Private reference1.4%4.0%

In the module arms, the roster fixes names and addresses, so these conflicts are removed by design. The audit above confirms the effect without using the roster.

Conclusion. The constraint module delivers the cross-record consistency that workspace data needs, and the effect holds under an independent check. In the revision, we will add both measurements and state the consistency claims in terms of them.

We hope these results address your concerns about the contribution over metadata-conditioned PE. Thanks again for your review!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.