acceptodds
Withdrawn from ICLR 2027

What Does a Copyright Budget Certify? An Exact Threshold for Per-Token Meters, and a Length-Independent Certificate for Selection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Certified inference-time copyright defences publish a divergence budget between the served distribution and a safe model, the anchor, trained without the protected works. First, a budget of nats says nothing about a work whose surprisal under the safe model is at most , at every R\'enyi order. Anchored decoding charges its budget per token, so it grows with the output, and at its authors' setting it is vacuous for all protected passages. Second, spent on choosing among draws from the safe model, by any rule, an adversary's included, the budget is a pathwise certificate of exactly nats at any output length, small enough that a rights-holder can bound near-verbatim copying by sampling the safe model alone. Third, with the risky model served as a text continuer at temperature , neither of two judges separates the one metered budget that covers a -token window, , from the safe model. At matched certificates, best-of- and its blockwise variant beat every meter we ran there, under one of two judges once empty answers count as losses; at the authors' temperature the meter at ties selection. Where a meter's certificate is void, selection wins against the authors' B pair and against their byte-level decoder, whose base model is weak on instruction prompts, but loses to the B at their temperature or served as a chat assistant. Against the chat assistant a windowed meter wins too, at window budgets too large to audit by sampling, and leaks near-verbatim text at nats, where both judges prefer it. With no judge, majority vote lifts GSM8K from to . Selection never consults the risky model, so no rescues an anchor that cannot do the task; served from the anchor alone its draws cost a metered decode, and it loses at matched compute. Its leakage, zero at any or at , is relative to the anchor, which must be vetted, as must the scorer unless queries are capped.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Copyright protection for language models, certified decoding, near-access-freeness, divergence budgets, best-of- 𝑛 n selection, memorization and extraction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_cP4AOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Background: This paper studies decoding algorithms to reduce the risk of copyright infringement in LLMs. Typical copyright infringment decoding has two LLMs (a risky LLM and safer anchor LLM). While decoding from the risky LLM, the algorithm ensures the divergence between the risky/anchor LLM is below a threshold.

This paper argues that as generations get longer, the divergence between the risky LLM and the anchor LLM will grow, and this will make it harder to certify that the generation is copyright-safe. This paper instead proposes generating n=64 full sequences in parallel from the anchor LLM, and then scoring them using a reward model. This ensures that the probability of copyright infringement is bounded by a low threshold of np, where p is the anchor LLM's probability of generating the copyright text. Unlike previous methods, this bound does not grow with the generation length.

The main experiments use a 1.8B anchor with 8B / 70B risky models as baselines in math, creative writing, and Wikipedia biography domains. Selection improves LLM-judged quality in some configurations and produces no observed near-verbatim copying in the clean anchor setup. However the quality gains depend on the experimental setup and do not always help factual accuracy (perhaps due to the usage of a much smaller anchor LLM). The authors additionally study the computational tradeoff between the options.

(note: the paper is very dense and hard to read, so the summary above is a best-effort after spending ~2 hours on it, I might have missed details)

Critical strengths

None

Critical weaknesses
  1. The paper is confusing and very hard to understand. The exposition is unnecessarily dense and makes the work sound more substantial than it actually is. It keeps mixing what the math guarantees, what actually happens in experiments, and whether the answers are useful. I strongly urge the authors to rewrite the paper in a more accessible way for the standard ICLR readers.

  2. Generating n=64 generations from an anchor LLM, followed by a rescoring step seems impractical. While I agree a lot of it can be parallelized, it needs an order of magnitude more compute to sustain the same QPS, despite sharing prefill. Why not instead have a copyright checker at inference, which can block decoding / individual tokens once they cross an n-gram thereshold of similarity against copyright corpora?

  3. Training anchor LLMs at fronter LLM scale is probably not feasible. Frontier LLM pretraining is an extremely complex and expensive engineering process (millions of dollars at the minimum) and involve web-scale corpora of text. At this scale, it's hard to guarantee the the training is completely copyright-free --- even if the sources are copyright-free, there could be several derived web sources which leak content. Given these constraints, I don't think assuming access to an equal quality anchor LLM is a realistic assumption for frontier LLMs. I would much rather see significantly cheaper post-training strategies which can minimize recitation.

  4. Related to (2), (3), I find it hard to imagine that significantly smaller anchor LLMs can match the quality of a frontier LLM when rescoring n=64 (unless they distill from the risky LLM, which increases copyright risk). I believe this gap will especially grow for longer agentic rollouts / complex tasks where a smaller LLM might not be able to land on the correct solution in the larger search space. Even the experiments in the paper show mixed results in quality parity, so I'm a bit skeptical overall. The selection accuracy is also fundamentally bounded by the quality of the reward model / selector judge.

Questions for authors

N/A, see above

Reviewer_QAKsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies inference-time copyright guarantees relative to a safe reference model psp_s. For an event with reference probability e−Se^{-S}, it shows that a Rényi-divergence budget of at least SS allows the event to occur with probability one, at every order considered. A budget accumulated per token can therefore become uninformative about a protected event. The paper then studies selecting one of nn samples from the safe model. Any such rule satisfies q(y)≤nps(y)q(y)\le n p_s(y), giving a whole-output max-divergence bound of log⁡n\log n, independent of output length. The paper also considers adaptive queries, bounded-budget sparsity, blockwise selection, and a sliding-window meter, and compares leakage, judged utility, and computation across several settings. Its central message is to evaluate a budget by the protected events it actually constrains.

Critical strengths
  1. The paper makes a useful distinction between an uninformative certificate and observed leakage. Proposition 1 uses standard divergence arguments, but applying the threshold to deployed budgets reveals a practical issue: a formally valid budget may impose no useful restriction on the protected event. Below the threshold, different Rényi orders give different event bounds, so equal numerical budgets need not provide equal protection. The authors also report that a meter can offer empirical protection after its certificate becomes uninformative. This is a careful and useful treatment of formal guarantees and measured outcomes.

  2. The whole-output selection guarantee is simple and robust to the scoring rule. The bound q(y)≤nps(y)q(y)\le n p_s(y) holds for any rule that serves one of the draws. It does not depend on the scorer being accurate or reliable. The paper also reports important limits: a contaminated anchor can leak more under selection, the quality of the anchor limits utility, serving templates can reverse rankings, and selection loses at matched compute. These results support the value of selection as a component with a clear relative guarantee, while giving readers evidence about its costs and limits.

Critical weaknesses
  1. The claimed prefix-conditional window guarantee for blockwise selection is false in general. Section 3 and Appendix G, including Table 33, assign a budget of (⌈w/L⌉+1)log⁡n(\lceil w/L\rceil+1)\log n, capped by the number of blocks, to any ww-token window given the preceding text. Proposition 2 supports a whole-block bound given a block-boundary history. Conditioning again on a rare prefix inside the selected block introduces a normalization factor that is not bounded by the argument. A two-token example makes this explicit. Let

    ps(00)=1−a,ps(10)=a(1−b),ps(11)=ab.p_s(00)=1-a,\qquad p_s(10)=a(1-b),\qquad p_s(11)=ab.

    Draw two candidates and select according to 11>00>1011>00>10. Then

    q(10)=[a(1−b)]2,q(11)=1−(1−ab)2.q(10)=[a(1-b)]^2,\qquad q(11)=1-(1-ab)^2.

    With a=10−6a=10^{-6} and b=10−3b=10^{-3}, the conditional probability of a second token equal to 1, given a first token equal to 1, rises from 0.0010.001 under psp_s to about 0.9995010.999501 under qq. This is a factor of about 999.5999.5, exceeding even the factor of 4 permitted by two charges of log⁡2\log 2. Exact rational enumeration confirms that the whole-output likelihood ratio remains at most 2. The same example can be the first block of a two-block process, with the second block sampled from the anchor; therefore the issue also affects the general installment claim. The supplied audit reports that analysis/blockwise_selection.py returns the claimed formula and that its selected tests check the returned values. The mathematical counterexample is independent of those implementation reports. Appendix G, correctly explains why a joint guarantee does not control a conditional span after a rare prefix. The main text and Tables 2, 31, 33, and 35 should be reconciled with that distinction. The whole-output guarantee and guarantees for windows starting at a block boundary remain useful; the separately constructed sliding-window meter is not refuted by this example.

  2. The empirical utility conclusion should be organized around a clearly specified setting and endpoint. The main judged workload uses a templateless text continuer, while changing the serving template, temperature, scorer, or task can change the comparison. The authors disclose these sensitivities, which is a strength. One important example is empty answers: counting them as losses reduces the original headline difference to +0.0155+0.0155, with a 95% interval of [−0.0245,+0.0555][-0.0245,+0.0555]. However, the paper also reports that selecting the best non-empty draw retains the same log⁡n\log n guarantee and gives +0.056+0.056 with interval [+0.0195,+0.093][+0.0195,+0.093] under judge B. That favorable result should be considered alongside the original rule's sensitivity. FActScore and matched-compute results identify further limits, rather than disproving the selection bound. My concern is that the large set of comparisons makes the central empirical claim hard to identify, especially while some of the window certificates used to match methods require correction. A compact comparison that fixes the serving setup, output rule, endpoint, guarantee type, and compute budget would make the evidence easier to assess.

Questions for authors
  1. Please check the counterexample above. Are the windows in Table 33 restricted to start at block boundaries? If arbitrary within-block starts are allowed, what bounds the additional denominator introduced by conditioning? Please make the guarantee types consistent across Section 3 and Tables 2, 31, 33, and 35, and revise any matched-certificate conclusions affected by this correction.

  2. Could you identify one primary endpoint and present the key comparisons with a fixed serving template, temperature, empty-answer rule, guarantee type, and computational budget? Please include the non-empty selection result and distinguish an inconclusive difference, a formal equivalence result, and a reliably positive difference. The paper already provides many of these ingredients; a concise consolidated analysis would clarify the main conclusion.

  3. Please state explicitly which prompt-conditioned distribution and protected event the 3n/N3n/N sampling guarantee covers, including the assumption of zero observed hits. What assumptions would extend it to an adversarially selected prompt family or pooled adaptive queries? The condition that a vetted scorer neither contains the work nor follows user instructions also needs a precise interpretation: does it support a formal guarantee without a query cap, or is it an operational condition to be checked empirically?

The threshold analysis and whole-output selection certificate are useful, and the paper reports its empirical limits carefully. The conditional-window extension affects a claimed guarantee and part of the comparative interpretation, and needs correction. With that issue resolved and the empirical conclusion more clearly organized, I would reconsider acceptance.

Reviewer_TS7wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines the practical meaning of divergence-based copyright certificates for language generation. It characterizes when such certificates become vacuous for protected-text reproduction events and proposes a selection-based approach that provides a length-independent, pathwise divergence bound of log⁡n\log n by choosing among nn completions from a safe model. The paper evaluates certificate strength, generation utility, copyright leakage, and computational costs against existing metered decoding methods under various settings.

Critical strengths

S1. Systematic analysis of certificate guarantees.

The paper carefully distinguishes whole-sequence, window-level, near-verbatim reproduction, and multi-query settings. Rather than comparing divergence budgets alone, it examines what each certificate actually guarantees and when the resulting bounds are informative. This provides a useful framework for evaluating copyright protection mechanisms under different protection requirements.

S2. Empirical auditing of existing certificates.

The paper compares published divergence budgets with actual expenditure and examines extraction behavior across multiple model pairs. These experiments provide useful evidence about the practical informativeness of existing certificates, beyond their formal validity.

S3. Simple and general selection guarantee with extensive evaluation.

The proposed selection mechanism provides a pathwise log⁡n\log n certificate independent of output length and the scoring function. The paper also evaluates different anchors, scorers, decoding configurations, and adversarial settings, including cases where selection performs poorly. This broad evaluation and the reporting of unfavorable results strengthen the empirical contribution.

Critical weaknesses

W1. Sensitivity of the empirical utility advantage to serving configurations and evaluation choices

The headline comparison uses Llama-3.1-8B-Instruct as a text continuer at temperature 1.0, where selection performs favorably (Section 4, Table 2). However, the advantage is substantially configuration-dependent. Under the metering authors' recommended temperature 0.7 and repetition penalty 1.1, selection is statistically indistinguishable from the meter even at k=0.5k = 0.5, where the window certificate remains informative (meter minus selection: −0.0245 [−0.057, +0.0095]). With the chat template, the meter instead outperforms selection by 0.138 at k=10k = 10 (Appendix C, Table 10), and the advantage also reverses on AlpacaEval and MT-Bench at larger budgets (Appendix C). These reversals generally occur where the meter's certificate is already uninformative, but the results nonetheless indicate that the practical utility advantage depends strongly on the serving configuration and task.

The evaluation is also sensitive to the judge and scorer. Judge B agrees with itself on only 29–37% of items across the two presentation orders (Section 4, Appendix B). Replacing the Qwen scorer with Gemma makes the headline difference inconclusive under Judge B (+0.015 [−0.0185, +0.0485]), while counting empty answers as losses similarly reduces the advantage to +0.0155 [−0.0245, +0.0555] (Appendix B). The authors provide valuable robustness checks with independent judges and a non-empty selection rule, and several advantages remain significant. Nevertheless, these sensitivities limit how broadly the headline utility improvement can be interpreted.

W2. Practical limitations of the joint protection–utility–compute tradeoff

Selection provides a length-independent certificate per response, but maintaining meaningful protection across repeated queries introduces additional constraints. Section 2 and Table 1 show that, at n=64n = 64, the transcript-level certificate for a protected 50-token window remains informative for only 38 adaptive queries when an unvetted scorer may collude with users who pool their transcripts. This does not invalidate the much smaller per-response reproduction bound, but it limits protection against piecewise reconstruction across queries. The authors propose a vetted scorer as a remedy, yet their evaluated Qwen scorer is unvetted. More importantly, the prompt-blind Comma-7B scorer tested in Appendix B provides much smaller gains (+0.0405 under Judge B and +0.0120 [−0.0230, +0.0475] under Judge G), leaving the combination of effective scoring and unrestricted query access unresolved. Per-work auditing also incurs additional cost: a 1% near-verbatim bound at n=64n = 64 requires approximately 19,200 clean anchor samples per work (Appendix E).

Computational efficiency presents a separate limitation. Best-of-64 selection incurs 21.8 times the throughput cost of a metered decode in the evaluated server configuration (Section 4.2, Appendix F), although batching yields lower single-request latency (0.51 times the 8B meter and 0.22 times the 70B meter). Under approximately matched compute (n=2n = 2, at 0.68 times the meter's cost), selection loses its utility advantage, with a difference of −0.0330 [−0.0625, −0.0025]. This suggests that selection's reported utility gains come at a substantial computational cost, raising questions about its practical scalability.

The authors transparently report these limitations. Nevertheless, the experiments do not clearly identify operating conditions under which selection simultaneously provides auditable protection, a practical multi-query horizon, competitive utility, and acceptable throughput cost. This leaves its practical advantage over metered decoding insufficiently established.

Questions for authors

Q1. Comparability of matched certificates

Table 2 matches numerical divergence budgets despite differences in divergence types and protection scopes. Can the authors clarify which comparisons remain favorable to selection when evaluated under the same event-specific probability bound, rather than the same numerical budget? This would help establish whether the reported advantage reflects a meaningful improvement in the utility–protection tradeoff.

Q2. Significance of Propositions 3 and 4

Propositions 3 and 4 appear to follow from standard KL divergence properties and the chain rule. What additional insight do these results establish beyond characterizing the divergence expenditure of particular decoding strategies? In particular, to what extent do they support general conclusions about the utility–certificate tradeoff?

Q3. Robustness and practical significance of the empirical advantage

Selection's advantage weakens or reverses under certain decoding configurations, benchmarks, and matched-compute settings, while some results are sensitive to judge reliability and empty-answer handling. Could the authors clarify the intended scope of their empirical claims and identify which configurations demonstrate a statistically reliable advantage under practically relevant protection and computational constraints?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.