acceptodds
Under review as a conference paper at ICLR 2027

Audit Frontiers and Finite Budget Rank Decisions under Partial Observations

Abstract

Partial observations can prove a predictive law wrong without saying what should replace it. We score updates of a finite predictive law by their worst case log score improvement over every latent world consistent with the observed law. A trust budget limits divergence from the reference; an audit budget limits divergence from the least committal law with the same observations. On a compact subanalytic set of available updates, the gain exponent, as a function of the audit precision, is eventually constant exactly when positive gain survives a zero audit. The comparison behind this criterion specializes preparation theory and bounds the gain by a finite maximum of minima of monomials in the two budgets, with constants independent of their ratio. In a conditional logit module, the vanishing orders of singular values determine the correction rank needed at each audit precision, and in a sixteen state tied network, bounds over every admissible action give exact minimum rank decisions at finite budgets. We also give certificates that hold uniformly over updates fitted to the same observations and a hardness construction whose instances share a second order jet. On a rain and sprinkler example, four of five deployed language models, told only how often the ground was wet, changed the unidentified split between rain and the sprinkler in almost every revision. In trained adapter families of two open weight models, that split moves at approximately first order over the measured edit scales, outside the module's second order assumption. This audit tests no theorem. All guarantees concern this finite interface, and no result is validated on a language model.

Primary area:
learning theory
Keywords:
partial observations, subanalytic geometry, low rank adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_B3WROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study how to revise a predictive distribution when new observations invalidate the original model but do not uniquely identify the underlying latent distribution. The paper measures improvements using the worst-case expected log-score gain over all latent distributions consistent with the observations. Two KL-based constraints are introduced: a trust budget limiting deviation from the reference distribution and an audit budget limiting unsupported changes beyond the least-committal distribution with the same observable predictions. Under compact subanalytic assumptions, the paper establishes a finite monomial description of the two-budget gain frontier and characterizes its scaling as the audit budget tightens. In a conditional logit model, the orders at which singular values vanish determine the correction rank needed to meet a given audit precision. The paper also presents diagnostic experiments with language models and low-rank adapters, while explicitly noting that these do not validate the theoretical guarantees.

Critical strengths
  1. The worst-case gain, trust budget, and audit budget form a coherent framework for model updates under partial identification. This formalization captures the important distinction between correcting observed errors and making unsupported changes to latent predictions.

  2. The paper provides nontrivial mathematical results, including a two-budget asymptotic frontier and finite-budget rank certificates. The analysis connects the singular-value vanishing orders of hidden logit changes to the rank needed for an audited correction.

Critical weaknesses
  1. The general frontier theorem relies heavily on established subanalytic preparation machinery. It does not provide directly computable exponents, practical optimization algorithms, or broadly applicable finite-budget constants.

  2. The explicit rank conclusions rely on a highly engineered sixteen-state tied model and higher-order cancellation assumptions.

Questions for authors

Can the authors demonstrate a real model-editing setting in which an audit precision requirement meaningfully predicts the minimum useful LoRA rank, rather than only showing the behavior of a specially constructed finite model?

Reviewer_byBnOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies how to revise a predictive distribution when only partial observations are available. It analyzes the trade-off between guaranteed improvement and constraints on the update, considers rank-restricted corrections in a designed model, and reports language-model experiments.

Critical strengths

The paper is explicit about the scope and limitations of its claims. It distinguishes the general asymptotic results from the finite-budget guarantees for the designed network, and acknowledges that the language-model measurements do not validate the theoretical rank law. The math also appears to be sound.

Critical weaknesses

The writing is confusing, particularly in the introduction. It opens with a rain-and-sprinkler example and quickly introduces worst-world gain and the trust/audit budgets, followed by a list of technical contributions. The paper does not explain how the example leads to the general problem being studied, what existing work leaves unresolved, or why the problem itself is worth studying. Without sufficient context, the reader is completely in the dark about what contribution "two independently shrinking budgets or the resulting gain exponents" is supposed to be.

On significance, I found it difficult to understand what the results add to our understanding of the problem. The paper presents precise statements about gain exponents, zero-audit limits and rank thresholds, but gives little intuition for what one should expect, which conclusions are surprising, or why they matter. The discussion of prior work identifies mathematical tools and formal differences, but I could not extract a clear explanation of what new insight the paper provides. For at least one central result, I would like to understand what was previously unclear, what the result resolves, and how this changes our understanding of model editing under partial observations. As written, I do not have a clear basis for assessing the significance of the theoretical contributions.

The technical overview also needs more explanation of the main ideas. The sketch following Theorem 3.1 refers to uniformization, normal crossings, coordinate faces and dual vertices, but I could not follow how these ingredients turn the original optimization problem into the claimed max–min formula. A simple worked example showing the costs and gain becoming monomials, the resulting linear program, and how its dual yields the comparison would help considerably. Similarly, the rank result would benefit from an intuitive explanation of how the order of the remaining singular values determines the audit cost and, in turn, the attainable gain. These intermediate connections are essential for understanding the results, but are difficult to extract from the main text.

Questions for authors

Could you use a concrete example to explain the significance of one central result? What was previously unclear, what does the result resolve, and why does this further our understanding of model editing under partial observations? In particular, what additional purpose does the separate audit budget serve when the worst-world gain already penalizes unsupported changes?

Reviewer_NxZSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies probability updates when observations identify only part of a latent distribution. It measures an update by its worst-case improvement in expected log-loss score over all latent distributions compatible with the observations. Updates satisfy two constraints: a trust budget measuring the KL divergence from the reference distribution, and an audit budget measuring the KL divergence from the information projection with the update’s own observations.

For compact action sets, the paper establishes a uniform comparison of attainable gain with a finite max–min envelope of monomials in the two budgets. It uses this representation to characterize when positive gain remains possible at zero audit. Further results relate correction rank to singular-value vanishing orders in a conditional-logit module, provide finite-budget minimum-rank certificates for a designed network, establish computational hardness and limitations of second-order information, and derive certificates from observed data. Experiments illustrate the audit on prompted and parameter-edited language models.

Critical strengths
  1. The uniform two-budget analysis is technically meaningful and interesting. Theorem 3.1 handles independently shrinking budgets, including extreme budget ratios. The proof in Appendix C addresses an important feasibility issue: the resolved coordinate boxes map into the actual action set, so their monomial optimization supplies attainable lower bounds. Although the geometric machinery is classical, its specialization and the resulting endpoint characterization form a substantial theoretical development.

  2. The rank results include constructive arguments. The conditional-logit calculation identifies how singular-value orders determine audit-limited gain. In the tied network, feasible bounded factors establish sufficiency, while bounds over all admissible corrections exclude smaller ranks. This supports the minimum-rank decisions more convincingly than evaluating selected correction paths alone.

Critical weaknesses
  1. The general frontier is an existence characterization rather than a usable general procedure. Theorem 3.1 supplies neither explicit validity radii nor an efficient way to obtain the envelope for a given model. The paper appropriately states this limitation and provides a hardness result. Nevertheless, it remains unclear how one would use the general characterization to make decisions beyond structured examples where the displacement is already analytically known.

  2. The empirical evidence is illustrative and its reproducibility needs clarification. The prompted experiments concern one four-state interface, and the parameter-editing experiments use two small models and one adapter location. The output-level SVD correction is informative, but its realizability as a neural parameter correction is not established. Appendix K gives substantial procedural detail; complete prompts, parsing and rounding rules, raw responses, and the referenced scripts would be needed to reproduce the reported counts and fitted orders. I could not assess those materials from the supplied PDF alone.

Questions for authors
  1. Can you characterize a simple first-order case, (C(t)=tA+O(t^2)), and state when positive leading-order worst-case gain survives a rank constraint? This would help interpret the regime observed in both trained model families.

  2. What is the principal intended use of the terminal-plateau criterion when the action set is known only through fitted edits or local measurements? In particular, how should a practitioner interpret a measured plateau given the arbitrarily long temporary plateaus demonstrated in Appendix E?

  3. Is there a natural class of architectures or update constraints that preserves unequal singular orders under its admissible perturbations? Alternatively, can the practical interpretation be framed through finite-scale crossover bounds rather than asymptotic orders alone?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.