acceptodds
Under review as a conference paper at ICLR 2027

Decision Transparency: When Can a Black-Box Decision Reveal Its Hidden Information?

Abstract

A good decision can be correct for the wrong reason. Decision quality asks whether an action performs well. Decision transparency asks what the released action reveals about the information that shaped it. This is a missing evaluation dimension for black-box decision systems. We call this method FanOpt, a public regret-to-evidence compiler. It makes decision transparency testable from one action under a structured alternative. If the null and alternative induce the same action law, no action-only test can have power above its Type-I error. FanOpt breaks this barrier when low regret meets a public geometric contract. Before private observations arrive, the public evaluator fixes a source-independent baseline and a public distance. Under the null, low regret keeps the returned action near the baseline. Under a structured alternative, low regret forces the action away because remaining near the baseline gives up the information-dependent utility gain. A fixed distance threshold can then turn one released action into statistical evidence. We prove finite-sample Type I and Type II error bounds and an explicit certified regret radius. A marginal-load bound carries estimation and rounding error into the regret condition required by the test. We give certificates for 2-factors and the stated spanning-tree policy. Matching also fits the framework when its family certificate holds. In controlled experiments, power rises from 16.0% at q = 1 to 100.0% at q = 2, while none of 2,196 low-regret outputs falls on the wrong side of the public threshold. These results show when one black-box combinatorial decision provides valid decision-transparency evidence under a structured alternative. The test does not need the private model, objective, or solver trace.

Primary area:
interpretability and explainable AI
Keywords:
decision transparency, black-box decision systems, action-only testing, statistical identifiability, regret analysis, combinatorial optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_FQMCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces decision transparency, which investigates whether a black-box decision reveals the influence of hidden information. The authors propose FanOpt, a framework that uses a public baseline and regret-based geometric separation to construct a statistical test from a single released action. The paper establishes finite-sample error bounds, analyzes the effects of estimation error, and evaluates the framework on several combinatorial optimization problems, including matching, 2-factors, and spanning trees.

Critical strengths
  1. Novel and well-defined problem. The paper introduces decision transparency as a distinct perspective from decision quality and interpretability. Investigating whether hidden information can be inferred from a single released action is an interesting and meaningful problem.

  2. Unified theoretical framework. The regret-to-evidence formulation provides finite-sample error guarantees under explicit geometric conditions. The applications to different combinatorial optimization families demonstrate the potential generality of the framework.

Critical weaknesses
  1. Restrictive assumptions for practical applicability. The main guarantee relies on a source-independent public baseline and a geometric certificate that separates low-regret actions under the null and alternative. It is unclear how readily such certificates can be established for realistic black-box decision systems, especially when the hidden information structure is unknown.

  2. Limited validation beyond controlled settings. The experiments largely use constructed optimization problems and semi-synthetic settings designed around the theoretical framework. While they validate the proposed mechanism, they provide limited evidence of applicability to independently developed black-box decision systems.

Questions for authors
  1. Applicability: How can practitioners determine whether the required geometric separation certificate holds for a given black-box decision system without access to its private information?

  2. Generalization: To what extent do the experimental results depend on the specific structured alternatives and public baselines used in the evaluation? What evidence supports the applicability of FanOpt beyond these controlled settings?

Reviewer_hR2kOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper takes a perspective on black-box evaluation in contrast to assessing correctness alone, they propose the dimension decision transparency, which aims to reveal the information that shaped the decision. To address this, the authors propose FanOpt, a public regret-to-evidence compiler. It fixes a default choice and a distance cutoff and checks whether a single released choice is located farther from the default than the cutoff.

Critical strengths

S1: The question is clearly posed. Testing source influence from one released action, with the model, objective, and solver hidden, is interesting.

S2: The core argument is simple and transparent. Exact null regret and a lower bound on alternative regret place low-regret outputs on opposite sides of a single threshold.

Critical weaknesses

W1 (Validity depends on assumed properties): The error guarantees depend on the private service having low regret, especially under the null. The null is designed so that the baseline is optimal. It is unclear to me how the regret condition can be verified from an external perspective, when it is realistic that the baseline is optimal, or how the baseline can be chosen accordingly.

W2 (Related Work / Framing): The introduction motivates decision transparency with modern AI services, including foundation-model agents, however, the results are limited to equal-size action families and the routing example stated as a motivation beyond decision quality is a case that, according to Theorem 4.1 no action-only test can resolve. Revising the introduction to make the motivation clearer would strengthen the paper. The discussion of related work does not include auditing and is fairly limited for interpretability, although closely related.

W3(Presentation): Overloaded symbols (G,T,A,h(A)G, T, A, h(A) vs h(A,E0)h(A, E_0)) and parameters (g,h,ρg, h, \rho) that are used before they are defined make the method hard to follow in some parts.

W4 (Experiments): Parameter choices (β=a/32\beta = a/32, gate threshold 2.6n2.6\sqrt n, K, a) are not motivated. The null and alternative match the theory’s structure, so the result that 2,196 low-regret outputs stay on the certified side follows from Theorem 4.2 and is not independent evidence. There are no baseline comparisons, and the road studies are acknowledged as semi-synthetic, it is unclear how the results transfer in practice.

Questions for authors

Q1: How does the paper relate to works about black-box influence auditing, e.g., [1]?

[1] Adler, Philip, Casey Falk, Sorelle A. Friedler, Tionney Nix, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian. "Auditing black-box models for indirect influence." Knowledge and Information Systems 54, no. 1 (2018): 95-122.

Q2: Can the authors explain how the method would transfer to practice? What happens to Type-I error if the null is misspecified?

Q3: How can the baseline, threshold, and K be chosen and verified for a real service?

Q4: Can you add a comparison with tests/ methods from the literature?

Reviewer_SRvmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a single action produced by a black-box optimizer can serve as statistical evidence that a structured private source influenced it. FanOpt fixes a public baseline E0 before data arrive and rejects the null when the action’s distance from E0 exceeds a threshold t. Under a baseline-geometry certificate that holds uniformly over the alternative class, low-regret actions fall on opposite sides of t under the null and the alternative, which bounds Type-I/II error by the regret tails. A marginal load bound links estimation error to regret. The authors give constructions for 2-factors and spanning trees and a conditional extension to matching. The paper claims that in synthetic assignment experiments, power rises from 16% to 100% as the budget q_comp goes from 1 to 2.

Critical strengths
  1. The paper proposes a novel question and clearly distinguishes it from decision quality, outcome-based risk certification, and related concepts.

  2. Opposing regret envelopes turn optimization quality into test separation, with an explicit certified regret radius (rho* = beta D/(a omega)). Any action on the wrong side of the threshold has regret above rho, so evading the test costs more than rho in utility.

  3. Theorems 4.2 and 4.3 appear correct, and Appendix D shows that the regret envelope is tight given the certificate constants (Delta, L, omega). The marginal-load bound charges estimation error only to coordinates whose sensitivity set touches an erroneous entity, weighted by expected load, rather than charging every coordinate equally.

  4. The paper gives a uniform 2-factor witness and a spanning-tree policy with bounded expected load, both using the same public test form.

  5. The paper is candid about its scope. It gives counterexamples, notes that the tree policy alone lacks Type-I control, and labels the road studies semi-synthetic.

Critical weaknesses
  1. The public test’s validity rests on a premise the auditor cannot verify. Type-I control requires a small null regret tail, which cannot be checked from the action alone. For trees, Eq. (30) makes the public test identical to the private gate, and in Table 2 the ungated pipeline has 100% Type-I error.

  2. The headline 16% to 100% transition appears to reflect the private gate. A back-of-envelope spiked-matrix calculation (assuming unit-variance noise) places the gate threshold (tau = 2.6) at q_comp of about 1.13, which matches the jump. The public test was replayed only at q_comp = 2.

  3. The theory does not explain the observed power. For 2-factors, mean regret is 4-13 times the certified radius, yet power exceeds 92%. For matching, 162 of 420 rejected alternatives lie outside the certified slice. The recovery conditions require exact recovery at the K tested, yet Fig. 2(b) shows nonzero error at q_comp = 2.

  4. The headline experiments use matching, whose certificate is left conditional. The constants are also inconsistent: beta = a/32 in the experiments but a/64 in Table 1.

  5. The method assumes an exactly known null optimum, a private objective that rewards overlap with the baseline, a simple null, and a known sparsity K. No experiment involves the ``foundation-model agents'' invoked in the motivation.

  6. he real-data evidence is weak. Road-study labels and costs share the same traffic profiles, and generic one-shot baselines reach higher power in Table 9, at a 1/16 null FPR versus 0/16.

Questions for authors

Q1: Can any part of \eta_0 be verified from the public side? If not, how does a rejection differ from the service reporting its detector output?

Q2: Could you please report the public-test rejection rate for all q in Fig. 2, along with its agreement rate? How does the transition move as \tau varies?

Q3: What fraction of rejected alternatives lie inside the certified radius, per family and q? Can you verify the matching certificate for the experimental template distribution and quantify \eta_geom?

Q4: Is there a deployed system where the null optimum is publicly known, and the private objective contains \beta |A \cap E_0|? How would a strategic service that trades a small amount of regret to cross the threshold be handled?

Q5: The threshold depends on the ratio beta/a and on the sparsity K. How should an auditor set it for a real service whose objective is private? Is there a deployed system where the null optimum is publicly known and the objective rewards overlap with the baseline?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.