acceptodds
Under review as a conference paper at ICLR 2027

Watermarking Values, Not Tokens: Distribution-Free Verification for LLM Time-Series Forecasts

Abstract

Large language models can generate probabilistic time-series forecasts as numerical ensembles, allowing downstream users to account for predictive uncertainty rather than rely on a single point forecast. When such forecasts are released and reused, verifying their provenance becomes important. Existing watermarking methods, however, are poorly aligned with this setting in two respects. First, watermark evidence is typically defined over generated tokens, whereas forecast releases are manipulated as numerical values through operations such as rounding, reordering, subsampling, and cropping. Second, watermarking an autoregressive forecast can feed modified values back into later predictions, causing local edits to propagate through the forecast horizon. To address these limitations, we propose V-band, a generation-time value-space watermarking framework for frozen autoregressive language-model forecasts. V-band uses the owner’s secret key to label numerical bands at each forecast step and tilts the model’s conditional digit probabilities so that sampled values are more likely to fall within preferred bands. We further introduce value-feedback-free steering, which preserves unmarked history across forecast steps while conditioning lower-order digits within each value on the higher-order digits that will appear in the released value. The resulting watermark is verified directly from released numerical values by ranking the owner key against independent decoy keys. We establish finite-sample false-positive control under key exchangeability, exact invariance to row reordering, and theoretical connections between numerical displacement, accessible band diversity, and watermark retention. Experiments on 1,455 units across five evaluation sets show evidence retention under the nine registered release operations with small pooled CRPS changes. A separate study of 26 compatible checkpoints across six model families examines transfer under the same band-width rule and verifier.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Watermarking, Time-Series Forecasting, Numeric Generation, Distribution-Free Verification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WMvdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes V-band for watermarking numerical ensembles from autoregressive LLM forecasters. It combines keyed numerical bands, value-feedback-free digit steering, and decoy-key verification. Experiments evaluate detection, numerical-reuse robustness, forecast quality, and transfer across datasets and compatible generators.

Critical strengths
  1. Numerical-value watermarking is well motivated by forecast reuse operations that can disrupt token-based evidence while preserving useful numerical information.
  2. Decoy-key verification provides finite-sample false-positive control under explicit assumptions, accommodating dependence among forecast values. VFF additionally provides a deterministic displacement bound.
  3. The evaluation is extensive, covering five evaluation sets and 26 compatible checkpoints, with component ablations, numerical-space comparators, and transparent reporting of limitations.
Critical weaknesses
  1. The benefit of model-probability-guided insertion over simpler numerical editing is not fully isolated. Existing comparisons change both the insertion procedure and carrier configuration. An output-only baseline using the same keyed bands, editable digits, and displacement budget would better establish the contribution of conditional digit steering (Section 5.2 and Appendix H).
  2. The small forecasting-cost conclusion depends substantially on aggregation. At the default strength, pooled CRPS increases by 0.26%, whereas the mean per-unit relative increase is 7.41%, or 2.55% after excluding near-zero controls. Although disclosed, this heterogeneity warrants upper-tail degradation statistics and evaluation under a prespecified utility constraint (Appendix I.1).
  3. Forecast-quality evaluation primarily measures stepwise marginal accuracy and coverage. Since VFF conditions future predictions on unpublished control values, these metrics do not establish preservation of temporal dependence in released trajectories. A trajectory-sensitive evaluation, such as cumulative-outcome accuracy or a variogram score, would strengthen the utility assessment (Sections 3.2 and 5.1).
Questions for authors

See above.

Reviewer_zQeXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes V-band, a watermarking framework for numerical forecast ensembles generated by frozen autoregressive language models, enabling provenance verification directly from released values rather than tokens. It combines secret-key numerical bands with digit-probability steering and a value-feedback-free mechanism that prevents watermarked values from propagating into subsequent forecast steps. Its theoretical contributions include finite-sample false-positive control under key exchangeability, invariance to trajectory reordering, and analysis linking numerical displacement to watermark retention. Validation covers 1,455 evaluation units across five sets and 26 compatible checkpoints from six model families, reporting substantial watermark retention under nine release operations with small pooled changes in forecasting CRPS.

Critical strengths
  1. V-band embeds and verifies evidence in numerical values, addressing the mismatch between token-based watermarking and operations such as rounding, trajectory reordering, and subsampling. This makes the contribution directly relevant to how forecast ensembles are actually shared and reused.

  2. The paper establishes finite-sample false-positive control under explicit key-exchangeability assumptions and introduces value-feedback-free steering to prevent watermark modifications from propagating across forecast steps. These address two consequential requirements: reliable attribution and preservation of forecasting utility.

  3. Evaluation spans 1,455 units across five sets, complemented by 26 compatible checkpoints from six model families and nine release operations. The reported retention of watermark evidence alongside small pooled CRPS changes provides substantial empirical support for applicability beyond a single model or dataset.

Critical weaknesses
  1. Comparative advantages require stronger validation. Comparisons are not matched for distortion, and Waltz is represented by a simplified stand-in. Since WGTD achieves higher survival at greater forecasting cost, a matched robustness–utility comparison would better establish V-band’s advantage.

  2. Forecasting costs vary across tasks. The pooled CRPS increase is 0.26%, whereas the mean per-unit increase is 7.41%, or 2.55% after excluding near-zero-error cases. Although transparently reported, this variation warrants further analysis before concluding that utility loss is consistently small.

  3. Robustness to numerical reuse remains uneven. Joint survival falls from 75.9% overall to 45.8% on Traffic, mainly due to coarse rounding. Dependence on metadata and original step indices, together with limited rescaling support, also constrains practical reuse.

  4. Security guarantees cover a limited threat model. False-positive control relies on key-independence assumptions, and deliberate watermark removal is explicitly outside scope. While reasonable, these boundaries leave robustness against adaptive removal or forgery unestablished.

Questions for authors
  1. At matched forecasting distortion and false-positive levels, does V-band retain an advantage over competing value-space methods, particularly WGTD? Could the authors provide a robustness–utility comparison across operating points to clarify whether the reported benefit persists beyond the selected configurations?

  2. What characterizes the forecasting units with substantial CRPS degradation, and how frequently does this occur in absolute as well as relative terms? Clarifying whether these failures can be identified before release would materially affect the assessment of the method’s practical utility.

Reviewer_MrX7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes V-band, a watermarking framework for probabilistic forecasts generated by frozen, digit-serialized autoregressive language models. Rather than assigning watermark evidence to token identities or sequence positions, it assigns keyed binary labels to numerical intervals at each forecast step. The sender biases low-order digit probabilities toward preferred intervals.

The proposed value-feedback-free, or VFF, sender retains the unmarked trajectory as the context for subsequent predictions while conditioning the units digit on the published tens digit. Verification operates on released numerical values and public metadata, ranking the owner’s score against independent decoy keys.

On the 984-unit main evaluation, the default configuration achieves 99.6% identity detection and 75.9% survival across all nine separately applied release operations, with a reported pooled CRPS increase of 0.26%. Additional evaluations cover four other sets and 26 compatible model checkpoints.

The central idea is well motivated: watermark evidence should align with the numerical objects that users manipulate. The paper’s strongest contributions are the integration of this carrier with model-conditioned digit sampling, explicit isolation of autoregressive feedback, and a verifier that accommodates dependent forecast cells.

Critical strengths

The problem formulation captures a meaningful mismatch in conventional watermarking. Rounding and reordering are ordinary operations on forecast ensembles. Evaluating their consequences for token-based evidence is more informative than assuming that text-watermark robustness transfers to numerical outputs.

The construction is technically coherent. Step-indexed numerical bands make row-order invariance immediate, while the choice of low-order editable digits yields a transparent displacement budget. VFF also addresses a genuine distinction between modifying a published value and allowing that modification to change subsequent model inputs.

The decoy-key verifier handles dependent observations appropriately. Forecast cells occupying the same band reuse the same random bit. The paper recognizes why treating them as independent Bernoulli observations can produce badly miscalibrated z-scores. The exchangeability argument avoids that assumption.

The theoretical claims are generally well scoped. Appendix B explicitly distinguishes exact rank validity and displacement bounds from the single-tilt analysis and conditional retention approximation. The paper also acknowledges that its operational steering strengths lie outside the informative weak-tilt regime.

Critical weaknesses

Marginal forecast metrics do not establish preservation of joint trajectory quality. The released objects are trajectories, but the main utility metrics—step-averaged CRPS and interval coverage—primarily assess marginal distributions.

The feedback intervention convincingly shows that feeding a local edit into autoregressive history can substantially change later trajectories. However, the CRPS differences in Table 34 have confidence intervals containing zero. The controlled sender comparison also finds no significant CRPS difference between coupled, FF, and VFF generation.

Theorem 1 conditions on the release and metadata while averaging over independent, identically distributed key functions. This is a valid and useful result. It does not guarantee a particular false-positive frequency across releases for every fixed owner key.

Questions for authors

Are CRPS values normalized before cross-dataset pooling? What are the scale-normalized macro-average and upper-tail degradation rates?

How do VFF releases affect cumulative forecasts, temporal dependence, and path-dependent event probabilities?

What key/nonce deployment protocol is assumed, and how variable are false-positive rates across fixed owner keys?

What performance is obtained by a model-free bounded editor using exactly the same bands, verifier, and editable digits?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.