acceptodds
Under review as a conference paper at ICLR 2027

Private Long-Form Generation via Semantic Release

Abstract

Reports based on private records often ask different questions about the same underlying quantities. Acquiring those quantities separately for each question divides the privacy budget across repeated computations. In this paper, we propose Semantic Release, which acquires shared task states once and reuses them throughout a report. Public question dependencies specify the required states. Record-level mechanisms release them into a typed ledger, a public compiler computes answers, and a writer expresses those answers without further private access. Conditional zCDP composition bounds the privacy cost of the resulting transcript by the acquisition budget. We evaluate a Gaussian-view instance on LongWeave Sales Report 8K. Under the same statistical scope and total budget, report correctness increases from 20.96% with question-wise acquisition to 24.74% with shared releases, while answered rate remains similar. A budget sweep improves both acquisition and report correctness. Exact-view controls and a fidelity analysis show where gains are lost: questions can require unreleased statistics, noisy views can produce wrong answers, and the writer can fail to express a correct answer.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model, Private LLM decoding, Privacy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DDK2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Semantic Release generates private reports by releasing shared statistics once, compiling answers from them, and passing those answers to an LLM writer. The paper provides a transcript-level zCDP guarantee and evaluates a nine-view Gaussian implementation on LongWeave SR8K. Across 48 paired reports, correctness increases from 20.96% to 24.74% relative to question-wise reacquisition at the same privacy budget.

Critical strengths

The separation of private acquisition, answer compilation, and writing provides an explicit privacy boundary. The exact-view controls are useful for distinguishing missing statistics from perturbation error, while the compiled-answer evaluation separates acquisition quality from writing quality.

Critical weaknesses
  1. The incremental contribution is not established. The main baseline independently reacquires each view for every dependent question (Section 6.2), incurring the variance penalty derived in Section 5. This isolates the benefit of reuse. Reusable measurements already underpin the matrix mechanism and HDMM. Theorem 1 applies standard composition and post-processing, while the tested system acquires all views before writing. The paper provides no matched release-once statistics-to-text baseline establishing what the typed ledger and compiler contribute beyond conventional reuse.

  2. The report-level gain is not validated against the judging protocol. The main evaluator retains context across reports, but the bootstrap excludes this dependence and evaluator variability (Appendix D). Its interval [0.82, 6.76] therefore leaves order and context effects untested. The local evaluator gives a gain of 1.97 points with interval [-0.62, 4.53], and rates the non-private control at 41.50% correctness versus 26.97% in the main evaluation. Fresh-context evaluation and adjudication of disagreements are needed to establish the robustness of the central report-level result.

  3. The chosen release scope lacks a matched utility comparison. Nine of 21 views are released. Restricting exact statistics to these views reduces structured correct coverage from 93.16% to 60.35% (Figure 3). The budget sweep holds scope fixed, and the development configurations change scope, allocation, and granularity together (Appendix B.1). Neither establishes how the selected workload compares with broader releases at the same total budget. Since scope restriction is the largest measured source of lost coverage, this missing comparison leaves a consequential design choice unsupported.

  4. The core compiler is insufficiently specified for reproduction. Equation 3 defines answer functions abstractly, and Appendix D states that dependencies come from question text. The manuscript does not provide the complete question-to-view mapping, executable answer rules, or concrete abstention thresholds. These determine coverage and the answers supplied to the writer. The listed Gaussian views and budgets are insufficient to reconstruct the reported acquisition and report results.

Questions for authors
  1. What measurable benefit does the ledger/compiler design add over a release-once statistics-to-text pipeline at the same scope and budget? Which comparison isolates that benefit?
  2. How were reports and conditions ordered and grouped in evaluator streams? Does the gain persist with fresh contexts, blinded condition labels, and independent adjudication of disputed ratings?
  3. What selection objective produced the nine-view workload, and how does it compare with broader workloads at the same total privacy budget under the same evaluation?
  4. Can you provide the full dependency mapping, answer functions, abstention thresholds, and structured-answer correctness criteria needed to reproduce Tables 2 and 4?
1 comment
AuthorsOpenReview ↗

Response to Reviewer DDK2

Thank you for the detailed review. We address your four concerns below:

Q1 / W1 — Answer compilation

Answer compilation improved report correctness by 7.44 percentage points at the same privacy budget. Both methods used the same DP releases, writer settings, and scoring protocol. The only change was whether the writer received the released statistics or the answers computed from them.

Table 1. Answer compilation.

MethodWriter inputReport correctness
Direct release-onceReleased statistics17.30%
Semantic ReleaseCompiled answers24.74%
Gain with Semantic Release+7.44 pp

Q2 / W2 — Report scoring

We hid the method labels and sorted reports from both methods together by their anonymous IDs. We assigned consecutive groups of four reports in turn to three evaluator sessions and kept the order within each group.

We then scored each report in a fresh session using the same rubric. We hid the method labels and had an independent adjudicator resolve disputed ratings. Semantic Release still led by 2.40 percentage points.

Table 2. Independent re-evaluation.

MethodMain evaluationIndependent re-evaluation
Question-wise acquisition20.96%21.40%
Semantic Release24.74%23.80%
Gain with Semantic Release+3.78 pp+2.40 pp

Q3 / W3 — Release scope

On the SR8K development set, we searched allocations over 21 candidate views at ρ=4\rho=4. We chose the allocation with the largest minimum margin across our preset targets for coverage, precision, correct coverage, monetary error, and recall improvement. That allocation gave positive budget to nine views.

We compared it with an equal split across all 21 views, using the same total budget, development questions, and structured-answer scoring rules.

Table 3. Release scope.

AllocationCoveragePrecisionCorrect coverage
21 views, equal allocation90.13%47.93%43.20%
9 views, selected allocation58.94%71.23%41.98%
Nine views − 21 views−31.19 pp+23.30 pp−1.22 pp

The nine-view allocation answered fewer questions and made fewer mistakes among the answers it gave. The share of questions answered correctly was similar.

Q4 / W4 — Compiler and scoring rules

Dependency mapping. question_view_dependencies.json lists the views each question needs. This entry uses product revenue and total revenue.

{
  "family": "product_top1_revenue",
  "question": "Which product generated the most revenue and what was its contribution?",
  "status": "SUPPORTED_SCHEMA",
  "views": ["G_REV", "P_REV"],
  "scope": "UNCHANGED"
}

Answer functions. For this question, the compiler finds the top product in P_REV and divides its revenue by the total in G_REV to compute its share. The answer functions give the calculations for the other question families.

Abstention rules. The compiler returns UNRESOLVED when a required view is missing, when an estimated denominator is zero or negative, or when a comparison fails its stability check. For a qualitative comparison, the absolute difference must exceed 1.645 propagated standard errors. The compiler code gives the checks for each question family.

Correctness criteria. For Tables 2 and 4 of the paper, categories and top-three sets must match exactly, and comparisons must match the reference direction. The tolerances are USD 5,000 or 15% of the absolute reference amount for monetary answers, whichever is larger, and 5 pp for percentages. For counts, the tolerance is 3 or 15% of the reference count, whichever is larger. Every required field must pass for a compound answer to count as correct.

Reviewer_SYQkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a new method, namely semantic release, which utilizes differential privacy to protect the privacy of data records when assisting with LLMs. Experiments demonstrate that the proposed Semantic Release can achieve a better utility-privacy tradeoff compared with existing methods.

Critical strengths
  • This paper addresses a vital topic in LLM privacy.
  • Both theoretical and empirical analyses are provided for the proposed method.
Critical weaknesses
  • The writing is terrible. I think the whole paper requires a significant revision for almost all parts.
  • Many of the math symbols lack explanations.
  • There are no recent baselines introduced in the experiments.
Questions for authors

None.

3 comments
AuthorsOpenReview ↗

Response to Reviewer SYQk

Thank you for the review. We clarify the three concerns below:

W1 — Writing

We explain the method through the shared-statistics example in Figure 1 and the acquisition–compilation–writing pipeline in Figure 2. The central idea is that several report questions can use the same released statistics. The compiler turns those statistics into answers, which the writer then expresses in the report. Section 4 follows this sequence and defines each stage.

Could you point us to the passages you found difficult to follow? This would help us address your presentation concern more precisely.

W2 — Mathematical notation

In the manuscript, we introduce mathematical symbols and explain their meanings at first use, rather than listing all notation at the beginning. This may have made the notation harder for you to follow across sections. We summarize the main symbols and their roles below to provide a clearer explanation.

Table 5. Main notation.

SymbolMeaning and role
ϕj\phi_jMaps each record to its contribution to an aggregate.
SjS_jThe aggregate statistic before private release.
ZjZ_jThe privately released value stored for later use.
Bℓ,gℓB_\ell,g_\ellBℓB_\ell specifies the states needed by question ℓ\ell, and gℓg_\ell computes its answer from those released states.
L\mathcal LThe ledger that stores released states for reuse across questions.
ρj,ρ⋆\rho_j,\rho_\starThe privacy budget assigned to acquisition jj and the total budget.

W3 — Recent baselines

Our experiments include the private decoding method InvisibleInk [1] in Appendix E. We implemented its fixed-budget residual decoding and further improved it with SM-FID, which uses a position-dependent budget schedule to support decoding beyond a prespecified horizon.

On GovReport, our SM-FID extension improves Fact@512 from 0.1172 to 0.1448 over InvisibleInk. On Yelp, it improves TopicAcc@100 from 0.10 to 0.18.

We further compare our method with both baselines in the positional acquisition experiment, using the same reference count, total privacy budget, and sampler. At position 8192, our method achieves an acquisition score of 64.36%, compared with 11.44% for InvisibleInk and 11.14% for SM-FID, showing higher acquisition utility when private information is needed late in generation.

[1] Vishnu Vinod, Krishna Pillutla, and Abhradeep Guha Thakurta. InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy. Advances in Neural Information Processing Systems (NeurIPS), 2025.

Dear authors,

Thanks for your reply, and apologize for my fault to neglect your comparison in Appendix.

I suggest a revision in future to: 1. append the notations of math symbols into the manuscript (this is vital); 2. relocate the experiments especially the comparisons; and consider to enlarge the introduction section and revise the methodology part.

Thanks.

Thank you for revisiting the InvisibleInk comparison in Appendix E. We agree that incorporating the notation table into the manuscript and highlighting the baseline results in the main text would improve the clarity and accessibility of our presentation. We hope that these clarifications address your concerns, and would be grateful if you would consider whether they warrant an adjustment to your overall rating.

Reviewer_qEGfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework called Semantic Release to improve privacy-budget efficiency in long-form report generation from private records. To address redundant privacy-budget consumption in conventional question-wise acquisition, the authors introduce a method that acquires shared statistics (task states) only once, stores them in a typed ledger, and reuses them throughout the generation process. Theoretically, the paper establishes a formal privacy guarantee (Theorem 1) based on zero-concentrated differential privacy (zCDP). Empirically, experiments on a synthetic sales dataset demonstrate that the proposed method significantly improves report correctness compared to the baseline.

Critical strengths
  • S1: Well-Motivated Framework with Formal Privacy Guarantees: The paper identifies an important inefficiency in privacy-preserving report generation: redundant privacy-budget consumption from repeated access to shared information. The proposed framework addresses this problem through reusable task states while providing a rigorous zCDP guarantee for the entire generated transcript. This combination of a practical problem formulation and formal privacy guarantees is a key strength supporting acceptance.

  • S2: Empirically Demonstrated Improvement in Privacy–Utility Trade-off: Under identical privacy budgets and statistical scopes, Semantic Release achieves statistically significant improvements in report correctness over question-wise acquisition. These results provide evidence that the proposed framework offers tangible benefits beyond its theoretical formulation, strengthening the case for acceptance.

Critical weaknesses
  • W1. Limited Generalizability Beyond Synthetic Data: The evaluation relies entirely on synthetic data, leaving uncertainty about the framework's effectiveness on real-world datasets with noise, missing values, and complex dependencies. In particular, it is unclear whether ledger construction and public compilation would remain reliable under such conditions. This limitation weakens the empirical evidence for the practical applicability of the proposed approach.
  • W2. Limited End-to-End Reliability: Despite statistically significant improvements over the baseline, overall report correctness remains relatively low. The framework also relies heavily on the public compiler and subsequent LLM-based generation, but their failure modes and impact on report quality are not sufficiently analyzed. Consequently, it remains unclear whether improved privacy-budget efficiency translates into practically reliable long-form reporting, limiting the overall impact of the work.
Questions for authors
  • Q1: Robustness to Real-World Data Imperfections: The evaluation relies entirely on synthetic data. How does the framework handle real-world characteristics such as noise, missing values, and complex dependencies? Specifically, what mechanisms or design choices ensure that ledger construction and public compilation remain reliable when faced with such imperfections?

  • Q2: End-to-End Reliability and Failure Handling: Despite relative improvements, absolute report correctness remains relatively low. Given the pipeline’s heavy reliance on the public compiler and subsequent LLM generation, how does the framework address failure modes (e.g., unresolved queries, compounding noise, or compiler misalignment)? What fallback strategies or constraints are in place to ensure that improved privacy-budget efficiency actually translates into practically reliable long-form reporting?

2 comments
AuthorsOpenReview ↗

Response to Reviewer qEGf

Thank you for the insightful review. We address your two questions below:

Q1 / W1 — Real-world data

To reduce the impact of noise, missing values, and complex dependencies in real-world data, our framework controls how much an individual record can affect each aggregate. We extract the quantities needed for the task from each record and cap their magnitude before aggregation through the record-to-state mapping in Eq. (4).

We define missing-value handling within the same mapping. To compute a mean, a valid record contributes its clipped value and a count of one, (xi,1)(x_i,1), while a record missing a required field contributes (0,0)(0,0). This keeps the sum and count aligned and distinguishes missing entries from valid zeros. Both components are released under a mechanism calibrated to their joint sensitivity, and the compiler computes the mean from the released values.

For calculations involving several fields, we record a common validity rule in the scope of the related statistics. Their numerators and denominators then refer to the same eligible population. The question-to-view mapping and answer functions in Eq. (3) specify which states each answer needs and how to combine them.

Q2 / W2 — Reliability and failure handling

In our method, the compiler marks unresolved queries explicitly and passes these markers to the writer alongside the available answers (Sections 4.1 and 4.3, Eqs. (3) and (6)). We instruct the writer to preserve this uncertainty and avoid filling in unsupported answers.

For noise, overlapping questions reuse the same released values (Section 4.3), and the uncertainty analysis accounts for their shared error (Section 5, Eqs. (9)–(10)). During compilation, a nonpositive denominator makes the answer unresolved. For a qualitative comparison, the estimated difference must exceed 1.645 propagated standard errors in absolute value to produce a definite answer.

To address compiler misalignment, we specify the required views and answer function for each question (Section 4.1, Eq. (3)). State types identify the quantities being used, and scopes identify the populations they describe. The writer receives the computed answers, reducing the calculations it must perform while generating the report.

Table 4. Reliability.

AnalysisMetricResult
Exact full scope, Figure 3(b)Structured correct coverage93.16%
Exact active scope, Figure 3(b)Structured correct coverage60.35%
Private active scope, Figure 3(b)Structured correct coverage40.67%
Realization fidelity, Section 6.6Report correctness given a correct compiled answer (local)63.41%
Direct release-onceReport correctness17.30%
Semantic ReleaseReport correctness24.74%

Using the same DP releases, explicit compilation improves final report correctness by 7.44 pp.

Thank you for the detailed response and clarifications. I appreciate the authors' efforts to address my concerns.

I have already given the paper a positive rating (Weak Accept), recognizing its contributions. However, while the response provides useful clarifications, I do not find the overall contribution sufficiently strong to justify a clear acceptance.

Therefore, I will maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.