acceptodds
Under review as a conference paper at ICLR 2027

ROUTE-B2B: Evidence Routing for Buyer Qualification under Incomplete Information

Abstract

Global business-to-business (B2B) buyer discovery must establish qualification from unevenly observable public information while preserving missing facts as unresolved. ROUTE-B2B connects evidence acquisition, document interpretation, and buyer qualification. Partially Identified Evidence Routing (PIER) maintains qualification bounds and acquires evidence under a fixed budget, returning contact, exclude, or verify according to explicit proof obligations. The Constraint-Compatibility Interface Router (CCIR) uses programmatic evidence contracts to select free-span extraction, source-anchor selection, or agreement-based abstention. Controlled evaluations span four domains and five languages, with paired rich/sparse views preserving candidate facts. Across three frozen open-weight backbones, an exploratory routing replay finds 9.84–20.86% lower source cost for PIER than coverage acquisition, with equal or higher observed correct-resolution rates. CCIR raises exact-card F1 by 30.89 percentage points over free extraction and reduces contract-inexact admission from 21.40% to 2.53%; compatibility relative to fixed anchors varies by backbone. In a compact hosted-system comparison, PIER matches the highest common-readout resolution rate among compared methods at lower source cost than its tied comparators, using a deterministic controller beyond shared extraction. Together, these controlled results support decision-directed evidence economy and expose the distinct roles of contract fidelity, decision coverage, and acquisition cost. Commercial intent remains a separate target for prospective validation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
evidence routing, buyer qualification, large language models, partial identification, selective prediction, data curation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zK1aOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROUTE-B2B, a framework for evidence-based business-to-business (B2B) buyer qualification under incomplete information. The framework integrates two components: Partially Identified Evidence Routing (PIER), which optimizes evidence acquisition and stopping decisions using qualification bounds and budget-constrained planning, and Constraint-Compatibility Interface Router (CCIR), which selects extraction interfaces based on calibrated evidence-contract correctness. The authors construct a controlled multilingual benchmark spanning four business domains and five languages, evaluating three open-weight language models and comparing acquisition strategies against several agentic approaches using a hosted model. Experimental results demonstrate that PIER reduces evidence acquisition costs while maintaining or improving observed decision resolution, whereas CCIR substantially improves exact evidence extraction over unconstrained generation, although its performance relative to fixed anchor extraction varies across models.

Critical strengths

S1. The paper addresses an important challenge in evidence-based decision-making: distinguishing missing information from evidence of nonqualification. The explicit treatment of uncertainty and verification requirements provides a principled alternative to conventional scoring approaches.

S2. The qualification bounds, decision rules, and budget-constrained acquisition objective are formally defined. Proposition 1 establishes a conditional soundness guarantee under explicit assumptions about evidence validity and identity verification.

S3. Separating evidence acquisition (PIER) from evidence interpretation (CCIR) is conceptually useful and enables independent evaluation of the two components.

Critical weaknesses

W1. The benchmark consists entirely of programmatically generated fictional organizations and documents. Although this enables controlled experiments, it remains unclear whether the proposed methods generalize to real B2B qualification scenarios involving noisy, contradictory, outdated, or difficult-to-access information.

W2. Table 4 shows that CCIR achieves an average exact-card F1 of 62.96%, compared with 64.17% for fixed Anchor extraction. CCIR also requires more extraction calls (1.267 versus 1.000). Therefore, the experiments do not establish a consistent advantage for the proposed interface router over the strongest fixed-interface baseline.

W3. Despite improvements in acquisition efficiency, the absolute correctly resolved decision rates remain relatively low. For example, CCIR+PIER achieves only 8.33% CRDR in the open-weight evaluation, while PIER achieves 17.50% in the hosted comparison. This raises questions about practical usefulness when most candidates remain unresolved.

Questions for authors

Q1. Source acquisition costs are expressed in designed units rather than measured retrieval expenses, latency, or human verification effort. Consequently, the reported efficiency gains may not directly translate into practical cost savings.

Q2. The ablation results suggest that optional stopping accounts for much of PIER’s acquisition-cost improvement. Full-horizon planning provides only limited additional benefits over one-step planning, making the necessity of the more sophisticated planning mechanism less convincing.

Q3. The framework assumes known predicate weights, decision thresholds, source costs, and outcome distributions. The paper does not establish how reliably these quantities can be estimated in practical deployments. Although the appendix discusses prior sensitivity and implementation assumptions, the generalization of these settings remains uncertain.

Q4. Section 6.1 attributes acquisition efficiency improvements to PIER, whereas Section 7 identifies optional stopping as the clearest source of improvement. Appendix B explicitly acknowledges that GreedyCov does not have the same stopping mechanism. Although the authors provide matched stopping ablations, the main comparison does not isolate the benefit of decision-directed planning from the benefit of stopping. A matched-stopping comparison would make the algorithmic contribution clearer.

Q5. Table 3 shows that Free+PIER achieves higher CRDR (9.79%) than CCIR+PIER (8.33%), despite CCIR’s substantially higher exact-card F1. The authors discuss this distinction in Section 7, but the results leave unclear when exact evidence recovery is necessary for correct qualification and when partial evidence is sufficient.

Q6. Table 4 shows that the Point selector achieves a higher macro F1 (64.54%) than CCIR (62.96%) while requiring fewer calls. Although CCIR incorporates additional statistical calibration and agreement mechanisms, the empirical justification for preferring the complete routing policy over this simpler alternative requires further explanation.

Q7. Table 5 reports that PIER achieves 17.50% CRDR under both common and native readouts, whereas Direct Astra achieves 12.50% under the common readout but 32.50% under its native readout. The paper explains that the readouts use different evidence aggregation and decision mechanisms. Nevertheless, this substantial ranking reversal makes it difficult to establish an overall advantage for PIER over agentic alternatives.

Q8. Proposition 1 assumes that admitted predicate bounds contain their true values and that the identity gate is correct. Appendix B explicitly acknowledges that semantic extraction errors can violate these assumptions. Thus, the guarantee establishes logical correctness under valid evidence rather than end-to-end reliability of the proposed system. The distinction should be emphasized when discussing practical qualification safety.

Q9. Equation 2 requires a valid identity gate for both contact and exclude decisions. However, Appendix A states that identity is fixed as resolved in the controlled benchmark. Consequently, the experiments do not evaluate an important component of the proposed qualification framework.

Q10. The open-weight experiments use simulator-supplied truth-frequency priors, whereas the hosted experiments use a common fixed prior. Although these settings are disclosed, it remains unclear how sensitive PIER’s advantages are to inaccurate priors in realistic environments.

7 comments
AuthorsOpenReview ↗

PIER: Evidence-Backed Qualification Within a Fixed Budget (1/2)

You identified the key distinction in ROUTE-B2B: missing evidence leaves a qualification unresolved rather than turning it into an exclusion. We tested what that principle means operationally. On the hosted cohort, PIER certifies 14 of the 15 cases that could possibly be certified under budget; even with the same stopping rule it spends less than greedy acquisition. The checks below replay frozen evidence with no new model calls or field labels.

W3: What does 17.50% correct resolution mean? At budget 2.45 we enumerated every affordable source subset for each episode, supplied its true evidence, and applied Eq. 2. This gives a clairvoyant upper bound on certificate-backed decisions, not a ceiling on predicting labels.

CohortTrue contact/exclude/verify profilesAffordable certificates (rich / sparse)Correct PIER resolutions
S2: 80 profiles, 160 views24 / 24 / 3234/160 (29 / 5)CCIR: 22 Qwen, 9 Gemma, 9 Phi, out of 34 each
Hosted: 40 profiles, 80 views14 / 12 / 1415/80 (14 / 1)14/15 with Anchor

In S2, 64/160 views are intrinsically verify even with every fact known; another 62 could resolve with all facts but not with any affordable bundle set. Thus the modest absolute yield has two sources: the decision contract/budget and the system's ability to find and interpret evidence. Hosted PIER found 14 of the 15 attainable cases, each supported by gold evidence in its own acquired set. GreedyCov and common-readout MindSearch also found 14. The smaller models leave a much larger gap. This supports a triage use case under a declared standard of proof, not a forecast of real sales throughput.

Q2/Q4/Q10: Match STOP and vary the priors on the same records. We gave GreedyCov the identical Full PIER stop/continue gate at each state; on continue, only the purchase ranking differs. This is a strong control that uses the full planner for the gate. With CCIR fixed, results are correct resolutions / mean source cost:

ConditionGreedyCovGreedyCov + same STOPPIER Full + STOP
S2 original (480 backbone-episodes)32 / 2.29432 / 1.96440 / 2.001
Same S2, neutral priors and opaque IDs32 / 2.22432 / 2.09541 / 2.061
Actual hosted cohort (80 episodes)14 / 2.27614 / 2.23214 / 2.153

The hosted row isolates an ordering benefit: with all 80 decisions unchanged, PIER saves 3.56% over matched GreedyCov (difference -0.079, nominal paired 95% CI [-0.108, -0.049]; 40 profiles). Original S2 gives a different tradeoff: Full PIER resolves eight additional backbone-episodes for 0.037 more cost; the paired quality CI includes zero. Holding S2 outputs fixed and assigning (.4,.4,.2) to every offered predicate, including the distractor, yields 41 correct for Full PIER versus 32 for matched GreedyCov, at lower cost (2.061 vs 2.095). PIER itself costs more than under the original prior (2.001). At short horizon PIER retains 39 correct at cost 1.744 on original S2. Thus the measurable contribution is valuing sources and stopping against a qualification decision; the size and even direction of the cost tradeoff depend on priors and horizon.

W1/Q1/Q3/Q8/Q9: What would deployment require? Weights and cutoffs specify a decision mandate; the deploying organization sets them and encodes noncompensable requirements separately. Costs and priors instead need measurement. Actual hosted logs show 2.50 paid purchases/episode for PIER, 2.89 for original GreedyCov and 2.33 for Direct. Purchases are observable; dollar and labor savings are not. A field test should measure tariffs, retrieval times and verifier minutes, estimate source yields on earlier disjoint records, then evaluate verification burden at several budgets. A 3.56% saving in designed units is not a 3.56% saving in money or time.

The rubric had domain-practitioner input; the evaluated organizations, documents and instance labels are generated, with identity fixed resolved. In a separate public-source feasibility audit, only 7/40 seed candidates passed the high-precision operational-identity gate and none met all pre-cutoff proof requirements. That is evidence of a data-access barrier, not field accuracy or 40 negative buyers. Proposition 1 is conditional on correct identity and sound bounds, not end-to-end reliability. A transfer test should start before identity resolution, retain conflicting, unavailable and expired sources, and score false merges, incorrect exclusions, certificate yield and verifier time. We address the witness, selector and native readout comparisons separately.

AuthorsOpenReview ↗

How CCIR and PIER Connect Evidence Quality to Justified Decisions (2/2)

W2/Q6: How does CCIR's calibration change what reaches the decision rule? CCIR is the extraction policy in our two-layer system. On Qwen, it chooses Anchor in every language: 88.00 exact F1 versus Point's 80.86 at the same one call per request. On Phi, retaining Free for Japanese/Korean improves F1 from Anchor's 40.16 to 45.55. With Full PIER fixed, CCIR yields 40 correct resolutions against 35 for Anchor and 37 for Point (480 backbone-episodes; zero observed wrong resolutions for all three). These CRDR counts are descriptive (per-backbone Holm p=1). Holding CCIR fixed instead, PIER yields 40 versus GreedyCov's 32 correct, reducing designed cost from 2.294 to 2.001 and extraction calls from 6.64 to 5.57. The first comparison changes interpretation; the second changes acquisition, including stopping. They show how each module contributes to the same qualification decision. Across the three backbones CCIR gains 30.89 pp exact F1 over Free and reduces exact-contract incorrect admissions from 21.40% to 2.53%. Relative to Point, that rate is 2.53% vs 3.42%, at 1.267 vs 1.000 calls/request. Point leads on macro F1, and Anchor's incorrect-admission rate is lower still (1.79%). On Gemma, agreement produces 96.13% precision but only 38.85% recall; the sealed criterion against both fixed interfaces remains unmet. These comparisons locate the benefits and the cost of CCIR's calibration rather than inferring its decision value from a single F1 ranking.

Q5: When is the exact witness necessary? The exact metric tests whether a system can reproduce the designated passage. Eq. 2 instead uses the identity gate and aggregate bounds induced by admitted predicate states. The online check verifies a span against the source, not against the hidden gold boundary. A shorter assertion can therefore supply the needed evidence even when it fails exact-card evaluation; whether it is audit-sufficient depends on whether entity, polarity, and context are preserved.

We rescored the unchanged predictions two ways. “Full assertion” allows omission of a layout prefix but requires the whole factual assertion verbatim, including negation and punctuation. “Substring” requires the correct state and a nonempty literal substring of the same gold witness. All original admission checks are retained.

PolicyExact / full assertion / substring F1Calls/request
Free32.08 / 62.01 / 63.101.000
Anchor64.17 / 64.17 / 64.171.000
Point64.54 / 66.55 / 66.551.000
CCIR62.96 / 64.16 / 64.161.267

Of 308 admitted Free bulletin cards, 295 preserved both the correct state and the entire factual assertion despite missing the canonical boundary. We then held each policy's actual acquisitions fixed, removed only those state-correct, nonexact cards that retain the full assertion (all surfaces), and reapplied Eq. 2. Free+PIER fell from 47 to 20 correct resolutions out of 480 backbone-episodes; CCIR+PIER stayed at 40. This gold-informed diagnostic shows which evidence mattered on the observed paths, not what a replanning system would do. It explains why document F1 is not a surrogate for decision yield and why literal overlap alone is not a general entailment test.

Q7: What causes the ranking reversal under native readout? Direct produces more correct native labels on this cohort. Direct has 26/80 correct resolutions and one wrong, versus PIER's 14 correct and none wrong, at costs 1.916 vs 2.153. The new native Direct-minus-PIER CRDR difference is +15.0 pp, nominal 95% CI [6.25, 25.00] over 40 paired profiles (exploratory; Holm p=0.04395 across six native contrasts, exact two-sided sign flips). On the same Direct runs, Eq. 2 instead gives 10 correct and zero wrong. Moving from common to native retains 9 correct, turns one correct into an incorrect resolution, and turns 17 verify decisions into correct resolutions; 53 stay verify.

Two comparisons answer your question about the ranked methods. For predicting the native label, Direct's 26 correct resolutions matter. For acting on a documented qualification, a gold-card check on the sources actually bought certifies 14 of PIER's 14 correct resolutions and 9 of Direct's 26. The other 17 are correct predictions; the acquired sources do not establish them under Eq. 2. Separating these outputs is the point of ROUTE-B2B's common readout: it credits a system for prediction and separately tests whether it can justify buyer contact or exclusion from its evidence.

AuthorsOpenReview ↗

Manuscript revisions: explaining unresolved cases and component value

We have revised the manuscript to connect the component comparisons with your questions about attainable qualification yield and practical resource use. The changes below make the evidence behind our November 8 replies directly inspectable in the paper.

1. W3: Why cases remain unresolved

Appendix I, Figure 10 expands the single-budget analysis into a diagnostic of attainable support.

  • Budget and visibility: Panel (a) shows the budget curves for paired rich/sparse views, (b) their paired minimum-cost differences among eventually certifiable firms, and (c) the affordable, over-budget, and intrinsically verify categories.
  • Acquisition and interpretation: Panel (d) compares realized decisions with reference-evidence ceilings on the recorded acquisition paths and the ceiling over all affordable paths.

These views locate the remaining gaps in catalog availability, acquisition, and interpretation. Appendix H specifies how the reference-evidence evaluator is constructed.

2. Q2/Q4: What planning adds beyond stopping

Section 6.1 now separates the original whole-policy comparison from purchase ordering under a common STOP gate.

  • Shared stopping: Table 23 collects the matched controls.
  • Planning depth: Table 3(b) retains the horizon-by-STOP ablation, Figure 2 displays the Full-versus-One cost distribution, and Appendix H traces the single episode where their decisions differ.

The revision makes the observed contribution and additional cost of deeper planning concrete.

3. Q3/Q10: Sensitivity to the acquisition forecast

Appendix H specifies an added five-fold cross-fitted prior estimated from other firms' cached extraction outputs, keeping each held-out firm's two views together. Table 23 places this condition beside the simulator and neutral priors, with separate cost and resolution columns.

The accompanying text reports mixed fold directions and a descriptive aggregate, so readers can assess the estimated-prior condition as a cost-resolution tradeoff.

4. W2/Q5/Q6 and Q1: Component value and resource use

Section 7 connects interface behavior to qualification decisions and their resource requirements.

  • Interface consequences (W2/Q5/Q6): The per-backbone overlap of episodes resolved by Free and CCIR is connected to Figure 3(c,d)'s assertion rescoring and fixed-path removal diagnostic. Reading this sequence alongside Table 4 shows when interface changes affect qualification and how Anchor and Point compare.
  • Resources per supported outcome (Q1): Figure 4(b) complements these decision analyses with total cohort source spending per supported correct outcome. Section 7 specifies the designed-unit accounting and the target-source prices and latency needed to translate it into deployment costs.

The revised discussion thus links component behavior to supported outcomes and the resources used to obtain them.

Reviewer_LVFyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper frames B2B buyer qualification as decision-making over incomplete public evidence, keeping missing facts as interval bounds that map to contact / exclude / verify. It proposes PIER, a lexicographic dynamic-programming acquisition planner with value-based stopping, and CCIR, a canary-calibrated selector among free-span, source-anchor and dual-agreement extraction, evaluated on a synthetic 80-profile, four-domain, five-language benchmark with three small open-weight backbones and a compact hosted-system comparison.

Critical strengths

S1. Unresolved predicates keep [0, 1] bounds, and Proposition 1 gives conditional soundness: given valid bounds and correct identity, every resolved decision agrees with the complete-information rule. S2. With CCIR frozen, enabling STOP at full horizon cuts source cost 9.70–13.17% and calls 12.32–16.15% with all 480 paired decisions unchanged.

Critical weaknesses

W1. No decision-quality contrast reaches significance. All 21 deployable PIER-minus-GreedyCov CRDR contrasts have Holm p = 1.0000 (Table 10), and all six hosted common-readout tests have Holm p ≥ 0.20069 (§6.3). What is supported is lower cost at no detectable difference in decisions. W2. A direct agent resolves far more episodes at lower source cost. Under the native readout Direct Astra resolves 26 of 80 episodes correctly at cost 1.916, against PIER's 14 at 2.153 (Table 5); Direct does falsely exclude one of 28 gold-contact cases, where PIER excludes none. The predeclared tests run on the common readout and the native comparison is labelled descriptive, so the readout on which PIER is not the best-resolving controller is the one left out of the claims. W3. CCIR misses its sealed criterion, and simpler selectors are competitive. Against Anchor, F1 is −1.21 pp [−3.97, 1.53] and the incorrect-admission upper bound is +1.49 pp against the prespecified +1 pp margin (§6.2). Fixed Anchor itself reaches higher macro F1 (64.17 vs 62.96) at 1.000 calls against CCIR's 1.267, and Point, which drops the significance and effect gates and the fallback hierarchy, reaches 64.54 at 1.000 calls, though at higher incorrect admission (3.42 vs 2.53).

Questions for authors

Q1. Table 3(b) shows PIER with STOP off costs 2.257 against GreedyCov's 2.294, leaving almost all of the 12.8% saving to STOP, so what does the dynamic program add that a stopping rule on GreedyCov would not?

3 comments
AuthorsOpenReview ↗

What CCIR and PIER Contribute Under Matched Controls

Your proposed control lets us separate PIER's purchase ordering from its stopping rule. The clearest ordering result is on the hosted cohort: same 80 decisions, 3.56% lower source cost even after matching STOP. We also report what changes on S2, where CCIR and PIER can each be held fixed in turn.

Q1/W1: Match stopping, then compare what is bought. At each state both policies query the exact same Full PIER STOP gate. On “continue,” the comparator buys the highest-coverage bundle and PIER buys the highest decision-value bundle. The comparator receives the benefit of the full planning computation; the test isolates purchase selection under a common gate. Post-review replay of the frozen outputs gives:

Cohort (backbone-episodes)GreedyCov: correct / costGreedyCov + matched STOPPIER Full + STOP
Original S2, CCIR (480)32 / 2.29432 / 1.96440 / 2.001
Same S2, neutral opaque (480)32 / 2.22432 / 2.09541 / 2.061
Hosted, Anchor (80)14 / 2.27614 / 2.23214 / 2.153

The hosted row isolates source choice: all three policies produce identical terminal labels on all 80 episodes. Giving GreedyCov PIER's STOP saves 0.044 units; changing purchase order then saves another 0.079 (3.56%; nominal paired 95% CI [-0.108, -0.049]). On original S2, matched GreedyCov is cheaper by 0.037; Full PIER has eight additional correct resolutions (+1.67 pp, CI [-0.83, 4.38]). We will distinguish these results in §6.1: ordering can lower cost for identical decisions, or buy additional observed resolutions, depending on the catalog. CIs resample profiles within domain (40 hosted; 80 S2, keeping views and fixed backbones together).

Which part of planning matters? With the same one-step gate and no full search, decision-directed PIER resolves 39 versus GreedyCov's 32, at cost 1.744 vs 1.758 and 5.096 vs 5.227 calls/episode. Cost and CRDR intervals include zero. This comparison distinguishes valuing the next source by its effect on qualification from maximizing predicate coverage, even without a long horizon. Full planning adds one correct resolution over One on the original S2 records. At fixed Full horizon and CCIR, enabling PIER's STOP cuts source cost 9.70–13.17% and extraction calls 12.32–16.15% across backbones, with all 480 decision labels unchanged (Appendix F).

W2: Which native predictions are supported by the acquired sources? Table 5 reports two different outcomes. Direct attains 26/80 correct at cost 1.916 versus PIER's 14/80 at 2.153. To address the asymmetry in testing, we applied a separate exploratory family to all six native PIER comparisons. Direct minus PIER is +15.0 pp (nominal 95% domain-stratified paired bootstrap CI [6.25, 25.00]; exact two-sided profile sign-flip, Holm p=0.04395). This does not replace the six predeclared common-readout tests. Direct also falsely excluded one of 28 gold-contact episodes; one event cannot establish a safety difference.

Why keep the common readout, a distinction qvtX identified as useful? With gold cards from the sources each agent actually acquired, Eq. 2 certifies 14/14 of PIER's correct resolutions and 9/26 of Direct's. Of the other 17 correct Direct predictions, four had an affordable certificate on another path and 13 had none anywhere under budget. They are correct predictions, not errors. The additional correct predictions are valuable, but an organization that requires an inspectable certificate cannot treat 26 correct labels as 26 justified actions.

W3: What does CCIR add over Point or a fixed interface? With Full PIER fixed, the observed ordering on buyer decisions differs from the ordering on document F1: CCIR yields 40 correct resolutions, Point 37, and Anchor 35 across 480 backbone-episodes, with zero observed incorrect resolutions for all three. These are descriptive counts over the same 80 profiles. On Qwen, CCIR selects Anchor throughout: compared with Point it improves exact F1 from 80.86 to 88.00 and correct decisions from 19 to 22, without a second extraction call. On Phi, language-specific selection gives 45.55 F1 versus Anchor's 40.16. This is the reason to retain the router: a single calibration procedure can preserve a strong fixed choice where it works and change interfaces where it helps.

The macro averages show the tradeoff. CCIR gains 30.89 pp exact F1 over Free and lowers incorrect admissions from 21.40% to 2.53%. Relative to Point it admits fewer incorrect cards (2.53% vs 3.42%), although Point has higher macro F1 (64.54 vs 62.96) at 1 rather than 1.267 calls/request. Anchor reaches 64.17 F1 and 1.79% incorrect admission. Gemma's agreement fallback costs recall; the original joint test against both fixed interfaces remains unmet. We will report these alongside the decision counts so that the selector is evaluated both on the cards it admits and on the decisions those cards support.

AuthorsOpenReview ↗

Manuscript revisions: clearer comparisons with simpler alternatives

We have reorganized the revised results around your central concern: what each component contributes relative to simpler alternatives. The distinctions discussed in our November 8 reply are now visible in the main figures and their accompanying analysis.

1. Q1/W1: Source ordering, stopping, and horizon

Section 6.1 now presents purchase ordering under a shared STOP gate separately from the original whole-policy comparison.

  • Matched stopping: Appendix I, Table 23, collects the full controls under shared stopping.
  • Horizon tradeoff: Table 3(b), Figure 2's Full-versus-One cost strip, and the trajectory example in Appendix H make the effect of planning depth concrete.

This presentation distinguishes lower spending for identical observed decisions from additional spending for additional observed resolutions, tying each conclusion to its own control. Readers can assess the source-ordering contribution alongside the cost of extending the planning horizon.

2. W2: Direct's native advantage and acquired-source support

Section 6.3 and Figure 4 present distinct views of the same hosted comparisons.

  • Prediction and support: Panel (c) places native and source-supported correctness side by side for all seven methods; panel (d) displays the separate exploratory native contrasts. Direct's native-answer advantage is explicit in the main results.
  • Resources per supported outcome: Panel (b) adds total cohort source spending per supported correct outcome, including spending on unresolved and incorrect episodes.

Readers can therefore compare prediction yield, documentary support, and the cost of obtaining that support through distinct, clearly labeled outcomes. The efficiency panel complements the decision comparison using the same runs.

3. W3: CCIR versus Anchor and Point

Section 6.2 and Table 4 now develop the backbone-specific explanation alongside the simpler interface comparisons.

  • Assertion scoring: Figure 3(c) adds complete-assertion rescoring.
  • Decision dependence: Figure 3(d) removes the state-correct, nonexact assertions while holding each original acquisition path fixed.
  • Buyer overlap: Section 7 shows which episodes Free and CCIR resolve in common and which are specific to either interface, including Free's higher aggregate yield and the differing backbone effects.

These additions connect extraction scores to the qualification decisions they support. The revised account keeps the sealed criterion and competitive fixed-interface results alongside this explanation, allowing the role and cost of calibration to be assessed directly.

Thanks for the response and summary. After reading the comments, I tend to keep my current score.

Reviewer_qvtXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies B2B buyer qualification under incomplete public information, emphasizing that missing evidence should be represented as an unresolved interval [0,1][0,1] rather than treated as negative (false). The framework introduces two components:

  1. PIER (Partial-Identification Evidence Router, Sections 3–4.1, Eqs. (1)–(4)): For K=7K=7 weighted obligations (weights in Appendix A Table 6), PIER maintains lower and upper qualification bounds [L(E),U(E)][L(E), U(E)] (Eq. (1)), maps them to contact (L(E)≥θ+=0.72L(E) \ge \theta _ + = 0.72), exclude (U(E)≤θ−=0.35U(E) \le \theta _ - = 0.35 or authoritative contradiction C(E)=1C(E)=1), or verify (Eq. (2)), and uses a finite-horizon lexicographic Bellman recurrence (Eqs. (3)–(4)) over M=5M=5 source bundles under budget B=2.45B=2.45 with an explicit value-based STOP action.
  2. CCIR (Contract-Calibrated Interface Router, Section 4.2, Eqs. (5)–(6)): Using 24 paired canary examples per (backbone, language) slice, CCIR selects among free-span extraction (Free), source-anchor selection (Anchor), and agreement-gated backoff (Dual agreement, D).

The evaluation uses a programmatically generated synthetic benchmark across 4 domains and 5 languages (EN, ES, DE, JA, KO; 80 latent profiles ×\times 2 visibility views = 160 episodes) with three 4B-scale open-weight models (Qwen3.5-4B, Gemma 4 E4B-IT, Phi-4-mini-instruct, Tables 3–4) and a smaller 80-episode hosted study with gpt-6-astra comparing against adapted search/planning baselines (Direct, ToolTree*, ATLAS*, FoA*, MindSearch*, Table 5).

Critical strengths
  1. Rigorous statistical reporting and honest disclosure of negative confirmatory results. The authors use profile-cluster bootstrap confidence intervals and Holm-adjusted sign-flip tests across 171 hypotheses (Appendix E), openly report that CCIR fails the prespecified joint test against Anchor (Section 6.2, Figure 6, Table 8), and clearly label the full-matrix PIER replay as post-hoc exploratory (Appendix C, p. 16).

  2. Clean modular attribution via paired visibility views and readout separation. Evaluating policies over frozen extraction caches on paired rich/sparse views and separating Common readout (shared deterministic qualifier) from Native readout (end-to-end policy decision, Table 5) cleanly disentangles extraction errors, acquisition choices, and terminal decision rules.

Critical weaknesses
  1. PIER's multi-step planning (Full) provides no meaningful gain over myopic 1-step selection (One) and increases cost; the main GreedyCov comparison is confounded by STOP and simulator priors.

    • Planning horizon (Full vs. One) vs. STOP (Table 3b, Appendix F Tables 15–18): In the 2×22 \times 2 ablation across 160 episodes per backbone, extending the horizon from 1 step (One + STOP On) to full horizon (Full + STOP On) changes 0 decisions on Gemma 4 E4B-IT (5.62% vs. 5.62% CRDR) and 0 decisions on Phi-4-mini (5.62% vs. 5.62%), and adds only 1 resolved episode on Qwen3.5-4B (13.75% = 22/160 vs. 13.12% = 21/160, pHolm=1.000p _ {\text{Holm}} = 1.000), while significantly increasing source cost (2.001 vs. 1.744, pHolm=0.00225p _ {\text{Holm}} = 0.00225) and LLM calls (5.567 vs. 4.900, pHolm=0.00225p _ {\text{Holm}} = 0.00225)! Across all 21 deployable pairs in Table 10 (p. 20) and all 6 hosted pairs in Table 14 (p. 21), PIER vs. GreedyCov has pHolm=1.0000p _ {\text{Holm}} = 1.0000 on CRDR. The reported cost savings over GreedyCov come entirely from STOP On vs. STOP Off (Table 15: 100% identical CRDR across all 480 episodes), because GreedyCov is forced to keep buying sources—including zero-weight distractors (Public media & awards digest, cost 0.35, Table 6)—even after crossing θ+\theta _ + or θ−\theta _ - is mathematically impossible under the remaining budget.
    • Algebraic redundancy of λv\lambda _ v in Eq. (3)–(4) and simulator-prior asymmetry: Because Eq. (4) applies lexicographic minimization to (1{d(E)≠verify},−A(E),0)(\mathbf{1}\lbrace d(E) \ne \text{verify}\rbrace, -A(E), 0), the second coordinate −A(E)-A(E) is compared only when two actions tie exactly on P(d=verify)\mathbb{P}(d = \text{verify}), in which case the λvP(d=verify)\lambda _ v \mathbb{P}(d = \text{verify}) term in Eq. (3) cancels identically (Appendix B, p. 15, lines 774–776). Moreover, Open-weight S2 supplies PIER with simulator truth-frequency priors and distractor roles (Section 4.1 lines 167–168; Appendix A lines 723–725); when the Hosted study replaces them with uniform (0.4,0.4,0.2)(0.4, 0.4, 0.2) priors, PIER's CRDR gain over GreedyCov vanishes (17.50% vs. 17.50%, Table 5) and its cost advantage shrinks to 5.41%.
  2. CCIR fails its prespecified criterion against Anchor, degrades Gemma 4 E4B-IT by -9.03 pp, and underperforms Free + PIER on downstream CRDR; open-weight experiments are restricted to 4B-scale models on synthetic templates.

    • CCIR vs. Anchor, Point, and Free (Tables 3a, 4, 7, 8): Under the prespecified criterion (Section 5.3 lines 306–308; Figure 6), CCIR was required to beat both Free and Anchor by ≥+3 pp\ge +3\text{ pp} F1 with inexact admission upper CI ≤+1 pp\le +1\text{ pp}. Instead, CCIR underperforms fixed Anchor by -1.21 pp Macro F1 (62.96% vs. 64.17%, pHolm=0.8945p _ {\text{Holm}} = 0.8945), exceeds the inexact bound (+0.74 pp, upper CI +1.49 pp), and increases calls by +26.7% (1.267 vs. 1.000). The simpler Point selector (64.54% F1, 1.000 call) beats CCIR on both F1 and cost. On Gemma 4 E4B-IT (Table 4, Table 7), the underpowered n=24n=24 canary test falls back to Dual agreement (D) in 4 of 5 languages, crashing F1 by -9.03 pp (64.37% →\to 55.34%).
    • Metric–decision inversion in Table 3(a): Despite scoring 32.08% on Exact-card F1, Free + PIER achieves the highest downstream CRDR (9.79%) at lower cost (1.952, 4.365 calls) than both CCIR + PIER (8.33% CRDR, 2.001 cost, 5.567 calls) and Anchor + PIER (7.29% CRDR), because Exact-card F1 rejects 297 state-correct Free bulletin cards whose spans do not match exact gold boundaries (Section 6.2, line 376).
    • Model scale and external validity: Open-weight S2 tests only 3.8B–4.5B models (Qwen3.5-4B with thinking disabled, Gemma 4 E4B-IT, and 2025 Phi-4-mini-instruct), omits all external agent baselines (Direct, ToolTree*, ATLAS*, FoA*, MindSearch*) from S2, and resolves only 8.33% of episodes (with over 91.6% abstaining to verify), while Direct Astra reaches 32.50% CRDR (96.30% selective accuracy) under Native readout in Table 5.
Questions for authors
  1. Equipping GreedyCov with STOP and explaining why Full horizon costs more than One step (Tables 3b, 15–18): What are the CRDR, source cost, and call counts when GreedyCov is equipped with the exact same value-based STOP rule as PIER (stopping whenever remaining budget or unacquired sources cannot change d(E)d(E) or reduce ambiguity)? Furthermore, why does Full + STOP On incur significantly higher cost (2.001 vs. 1.744, pHolm=0.00225p _ {\text{Holm}} = 0.00225) than One + STOP On while changing only 1 decision out of 480 episodes across the three backbones?

  2. Simulator-supplied priors in S2 and oracle-resolvable CRDR ceiling: What are the exact numerical values of the simulator-supplied truth-frequency priors used in S2 (Section 4.1 line 167; Appendix A line 723), and how does PIER perform in S2 when given the same neutral (0.4,0.4,0.2)(0.4, 0.4, 0.2) prior used in the Hosted study? Additionally, what is the ground-truth label distribution (contact / exclude / verify) across the 80 profiles and the oracle-resolvable CRDR ceiling under budget B=2.45B = 2.45 with perfect extraction?

  3. Reconciling Exact-card F1 with downstream CRDR (Free + PIER beating CCIR + PIER in Table 3a) and scaling beyond 4B models: Why does Free + PIER achieve higher downstream CRDR (9.79%) and lower cost (1.952) than both CCIR + PIER (8.33%) and Anchor + PIER (7.29%) in Table 3(a)? If Exact-card F1 is relaxed to state accuracy + substring/entailment overlap, or if larger open-weight models (7B–32B) are evaluated, does CCIR outperform fixed Anchor or Point selection?

3 comments
AuthorsOpenReview ↗

PIER: Isolating the Value of Stopping and Source Selection (1/2)

The matched control you requested separates two contributions of PIER: when to stop buying evidence, and which source to buy when continuing. Both can be measured: in S2, PIER's stopping rule removes unproductive purchases; on the hosted cohort, its source choices save a further 3.56% under the same stopping rule and the same 80 decisions.

Q1. Holding STOP fixed. At each state both policies query the frozen Full PIER recurrence. On “continue,” GreedyCov buys its highest-coverage affordable bundle; PIER buys the bundle with the best expected terminal decision value. Both get the full computation and the same priors, without the candidate's label or hidden evidence. The gate is the same function, not necessarily reached at the same states. We reproduced all 960 original traces, decisions, costs and call counts as a parity check. These post-review S2 replays fix CCIR, extracts, budget 2.45 and the same 80 rich/sparse profile pairs across three backbones.

S2 sources / policyCorrect/480Cost/episodeExtraction calls/episode
Original / GreedyCov322.2946.64
Original / GreedyCov + matched STOP321.9645.61
Original / PIER Full + STOP402.0015.57
Original / PIER One + STOP391.7445.10
Neutral, opaque / GreedyCov322.2246.66
Neutral, opaque / GreedyCov + matched STOP322.0956.36
Neutral, opaque / PIER Full + STOP412.0616.13

What changes under matched stopping? On original S2, adding PIER's gate to GreedyCov eliminates all 325 distractor purchases without changing any decision. With that gate fixed, PIER's ordering resolves 40 rather than 32 backbone-episodes: +1.67 pp (nominal 95% paired CI [-0.83, 4.38]) for 0.037 more cost. The original source-cost advantage thus belongs principally to stopping; it is not a cost advantage over matched GreedyCov here. Restricting future purchases to the greedy order when valuing STOP gives the same original-catalog averages. With neutral opaque sources, PIER's ordering gains 1.88 pp [0.42, 3.96] and costs 0.034 less [-0.057, -0.012]. No wrong resolutions occurred in any row. The intervals resample profiles within domain, keeping both views and all backbones together; they are exploratory, not replacements for the published adjusted tests.

On the actual hosted cohort, GreedyCov, matched-STOP GreedyCov, and PIER made identical decisions on all 80 episodes (14 correct resolutions) at costs 2.276, 2.232, and 2.153. Here purchase ordering saves 3.56% even after matching STOP (paired cost difference -0.079, nominal 95% CI [-0.108, -0.049], 40 profiles).

What does horizon add? Eq. 4 values expected resolution, then ambiguity, then cost. On the one Qwen episode whose decision changed, Full bought fulfillment, catalog/channel, then calibration/service (cost 2.30, correct contact); One bought catalog/channel then registry (2.05, verify). The shorter PIER variant keeps the same decision objective: it recovers 39 of Full's 40 correct resolutions in aggregate, spending 0.257 less and making 0.471 fewer calls per episode (One: 5.096 calls, Table 3b). This isolates the benefit of complementary purchases and its measured price. Your algebraic observation is correct: lambda contributes a constant times P(verify), which cancels when expected resolution ties. The overlap term, not that constant, orders the remaining ambiguity.

Q2. Priors and attainable yield. S2 gives both routers the same role-aware probabilities. In buyer-role/category/market/channel/capability/fulfillment order, the support frequencies among the 20 generated profiles per domain are:

DomainSupport probabilities
Cultural IP.55, .50, .40, .70, .65, .40
Beauty.50, .45, .60, .55, .65, .60
Alcohol.55, .65, .60, .55, .60, .40
Power.45, .50, .60, .60, .55, .50

Informative triples are (p, 1-p, 0); the distractor has (0, 0, 1). These are privileged aggregate frequencies, not held-out estimates. The neutral rows above replace every offered predicate's triple, including the distractor's, by (.4,.4,.2) and mask source identifiers. They isolate prior/catalog effects on the same S2 outputs; comparing S2 with hosted scores alone cannot do so.

The 80 profiles comprise 24 contact, 24 exclude, 32 verify. Enumerating every affordable source subset with perfect extraction and the same Eq. 2 yields an upper bound of 34/160 (21.25%): 29/80 rich and 5/80 sparse. Of 160 views, 64 remain verify even with full information; 62 more have no affordable certificate. This is a clairvoyant bound, not a demonstrated online score. An authoritative contradiction fixes one predicate; exclusion still requires verified identity and the upper score bound below .35. We address the witness/scale question separately.

AuthorsOpenReview ↗

CCIR: Exact Evidence Recovery and Downstream Qualification (2/2)

Q3: What does CCIR contribute to qualification? CCIR selects how a given backbone should produce evidence: generate a span, select a source anchor, or require agreement. Anchor is an action CCIR can choose, not a component we propose replacing. Against always-Free extraction, the frozen selector gains 30.89 pp in exact-card F1 (paired 95% CI [26.64, 35.24]) and reduces incorrect admissions from 21.40% to 2.53% of requests. The backbone-specific choices explain why routing matters. On Qwen, CCIR selects Anchor in all five languages and reaches 88.00 F1 versus Point's 80.86, both with one call per request. On Phi it selects Free for Japanese/Korean and Anchor elsewhere, reaching 45.55 versus fixed Anchor's 40.16.

Does this translate into decisions? Keeping Full PIER and its budget fixed, CCIR correctly resolves 22/9/9 episodes on Qwen/Gemma/Phi, versus 22/6/7 for fixed Anchor and 19/9/9 for Point (160 episodes per backbone; the same 80 paired profiles). That is 40 vs 35 vs 37 correct resolutions, with zero observed wrong resolutions for all three. These are matched point estimates, not significant CRDR gains (all six per-backbone comparisons: Holm p=1). Gemma is informative: CCIR's agreement fallback in four languages reduces exact F1 by 9.03 pp versus Anchor (55.34 vs 64.37), yet the resulting acquisitions still yield 9 correct decisions versus 6. Across backbones, CCIR averages 1.267 calls/request versus 1 for Anchor and Point. The sealed joint F1/admission criterion was not met against Anchor: F1 difference -1.21 pp [-3.97, 1.53], excess incorrect-admission upper bound +1.49 pp versus the +1 pp margin.

Your proposed scoring control. We kept predictions, selected interfaces, admission checks and denominators fixed. An admitted card must have the correct state. “Full assertion” permits omission of the layout prefix but requires the whole factual assertion verbatim, including negation and punctuation. “Substring” permits a nonempty literal substring of the same gold witness. F1 is the equal mean across backbones:

PolicyExact F1Full assertion F1Substring F1Calls/request
Free32.0862.0163.101.000
Anchor64.1764.1764.171.000
Point64.5466.5566.551.000
CCIR62.9664.1664.161.267

Under the substring rule, CCIR and Anchor have nearly identical point estimates (difference -0.02 pp, nominal 95% paired profile-bootstrap CI [-2.69, 2.71]); Point leads CCIR by 2.39 pp [0.48, 4.41]. This answers the relaxation question without equating a lexical match with entailment. Of 308 admitted Free bulletin cards, 297 had the correct state and a proper gold substring; 295 preserved the entire assertion. An online card can therefore carry the right fact even when its boundary fails the retrospective exact-card score.

Why Free+PIER reaches 47 correct decisions versus CCIR+PIER's 40. We held each policy's actual acquisitions fixed, removed only state-correct, nonexact cards that still contain the complete assertion, and reapplied Eq. 2. Free fell from 47 to 20 correct resolutions; CCIR stayed at 40. On the original paths, 22 correct resolutions were shared, 25 Free-only and 18 CCIR-only. Free's higher original yield is real, and many of its decisions use shorter witnesses. CCIR retained its observed yield under this specific removal. This is a gold-informed path diagnostic, not a deployable filter or a replanning experiment. CCIR's additional calls reflect dual agreement; bundle costs follow the resulting acquisition path, not document F1 alone.

We used matched small open-weight backbones and five languages (EN/FR/DE/JA/KO). The interactive agent comparisons use one hosted backbone and catalog (Table 5). We have no matched 7B–32B experiment or independent entailment annotation. We will put these boundary and decision comparisons beside the sealed result so readers can see what the router adds and at what cost.

The complementary role of PIER is visible holding CCIR fixed (Table 3): moving from GreedyCov to PIER increases observed correct resolutions from 32 to 40, lowers mean source cost from 2.294 to 2.001 and reduces extraction calls from 6.64 to 5.57. This is the whole-policy contrast; our first reply isolates stopping from source ordering. Together the comparisons show separately what reaches the decision rule and how much evidence it costs to get there.

AuthorsOpenReview ↗

Manuscript revisions: planning attribution and evidence-boundary effects

We have revised the manuscript around your requests to isolate planning from stopping, examine prior sensitivity, and connect exact extraction to downstream qualification. The analyses discussed in our November 8 replies now appear beside the results they explain, with an additional cross-fitted prior control.

1. Planning attribution

Section 6.1 now separates the original whole-policy comparison from the shared-STOP purchase-ordering control.

  • Stopping and source ordering: Table 3(b) and the matched-STOP comparisons in Appendix I, Table 23, make these contributions separately assessable.
  • Planning depth: Figure 2's lower strip displays the distribution of Full-versus-One cost differences, while Appendix H traces the episode whose decision changes with the longer horizon.

Together, these changes let readers assess stopping, source ordering, and deeper planning separately, including the extra cost of Full planning.

2. Prior sensitivity and attainable support

Appendices H-I, Tables 22-23, and Figure 10 connect the acquisition forecast to the evidence available under a budget.

  • Cross-fitted prior: Beyond the simulator and neutral controls in our earlier reply, Table 23 adds a five-fold extraction-yield prior. Appendix H specifies whole-firm folds, keeps each firm's two views together, and estimates its forecast from other firms' cached outputs. The text reports mixed directions across folds and treats the aggregate descriptively; the table displays the accompanying cost-resolution tradeoff.
  • Attainable evidence: Figure 10 extends the single-budget ceiling with rich/sparse budget curves (a), paired minimum-cost differences among eventually certifiable firms (b), and recorded-path versus all-affordable-path ceilings (d).

This makes the available evidence and the controller's access to it separately inspectable.

3. Evidence boundaries and changed buyer decisions

Section 6.2, Figure 3(c,d), and Section 7 incorporate the assertion rescoring and fixed-path removal analysis promised in our previous reply.

  • Witness-level diagnostics: Figure 3 places the rescoring and removal analyses beside language and surface variation.
  • Decision-level consequences: Section 7 adds the per-backbone overlap of episodes resolved by Free and CCIR.

The revised sequence shows which part of the extraction gap comes from exact boundaries and which buyer decisions rely on the affected passages. Table 4 and its adjacent text retain the fixed-Anchor and Point comparisons, so the interpretation of CCIR follows its backbone-specific effects and resource tradeoffs.

Comments on the paper

5 comments
AuthorsOpenReview ↗

All reviewers and Area Chair - general comment 1

To all reviewers and the Area Chair: manuscript revision overview

We have revised the manuscript to strengthen the presentation of PIER, CCIR, and their contribution to evidence-supported buyer qualification. Sections 6–7 connect each component to its reported effect, while Section 8 develops the evaluation framework for financial-document reasoning and claim verification.

1. PIER: separating stopping from source ordering

Section 6.1 and the matched controls distinguish the contributions of optional stopping and purchase ordering.

  • Stopping: At full horizon, STOP reduces source cost by 9.70–13.17% across backbones while preserving the observed decisions.
  • Source ordering: On hosted traces, PIER reduces source cost by 3.56% relative to GreedyCov under the same stop/continue gate and with identical terminal decisions.

2. CCIR: connecting extraction quality to usable evidence

Section 6.2 and Table 4 report the effect of interface calibration relative to Free:

  • Exact-card F1: a +30.89 percentage-point gain.
  • Exact-contract incorrect admissions: a reduction from 21.40% to 2.53%.

Figure 3 connects language and surface behavior to assertion recovery and downstream witness sensitivity. Fixed Anchor, Point, and every selector control remain in the comparison.

3. Hosted comparisons: native predictions and source-supported decisions

Section 6.3 and Figure 4 present native predictions alongside the correct decisions supported by the sources each method acquired.

  • Direct: 32.50% native correct resolution and 11.25% correct-and-supported resolution.
  • PIER: 17.50% under both endpoints.

These complementary endpoints connect the terminal prediction to the documentary record that can establish it.

4. Module composition and attainable evidence

Table 3 and Figure 5 connect module composition to decision quality and resource use. With CCIR fixed, the original whole-policy comparison shows that PIER:

  • Raises observed correct resolution from 6.67% to 8.33%.
  • Reduces mean source cost from 2.294 to 2.001.
  • Reduces extraction calls from 6.637 to 5.567.

The appendix controls and attainable-yield analysis explain how acquisition forecasts and evidence availability shape this operating point.

5. Section 8: financial reasoning and claim verification

Section 8 and Appendix J specify a common evaluation configuration using Qwen2.5-32B-Instruct, a shared source catalog, and four paid source reads.

The reporting structure connects three questions:

  • Answer correctness — Table 6: FinQA uses execution accuracy, while AVeriTeC uses verdict accuracy. Joint accuracy additionally requires sufficient evidence in the acquired sources.
  • Component contribution — Table 7: The controls compare PIER with matched-STOP GreedyCov while retaining CCIR, and compare CCIR with Anchor and Point while retaining PIER.
  • Evidence accompanying correct answers — Table 24: Conditional support measures the share of correct answers backed by sufficient acquired evidence. The native-to-joint gap expresses the difference between native and joint accuracy in percentage points.

This structure connects financial operands and source-backed claims to the evidence requirements used in buyer qualification. It also gives Tables 6–7 and Table 24 distinct roles: the former organize overall accuracy and component comparisons, while the latter examines the support accompanying correct answers.

The expanded conclusion connects these analyses to the method's operational output: a supported recommendation or a precise account of the remaining evidence requirements.

AuthorsOpenReview ↗

All reviewers and Area Chair - general comment 2

To all reviewers and the Area Chair: the contribution of the two routing layers

ROUTE-B2B connects missing-information states, evidence acquisition, and qualification decisions in an executable framework. Its central contribution is that the evidence needed to settle a requirement guides both which source to obtain and how its contents enter the decision record.

1. How PIER and CCIR work together

  • PIER — acquisition and decision state. PIER maintains a score interval containing the weighted score of every assignment consistent with admitted evidence. Subject to correct predicate bounds and identity, a resolved interval decision agrees with the complete-information rule. Its planner uses the remaining budget to prioritize resolution, then residual ambiguity, then cost.
  • CCIR — evidence interpretation. CCIR calibrates the extraction interfaces that populate this state using a separate programmatically labeled contract suite. The deployed choices are fixed before evaluation.

Together, these layers connect an explicit evidence requirement to an executable acquisition and interpretation procedure.

2. Connecting component effects to qualification

The S2 and hosted analyses examine the two layers from three perspectives:

  • Evidence recovery: CCIR improves exact evidence recovery over Free, while the per-backbone results explain where interface calibration helps and how its effects vary.
  • Acquisition controls: Matched comparisons identify the purchases avoided by STOP and the cheaper source ordering found by PIER on hosted traces.
  • Composition and source support: These analyses relate module effects to the final answer, the sources that can establish it, and the resources used to obtain them.

3. Evaluation units and uncertainty

The evaluation comprises:

  • 6,720 evaluation generations from three frozen backbones.
  • 720 separate calibration generations.
  • A hosted comparison of seven controllers.

The independent units are 80 latent S2 profiles and 40 hosted profiles drawn from that cohort. Domain-stratified profile resampling keeps visibility pairs and method comparisons together.

Table standard errors and paired intervals quantify variation across the evaluated profiles, conditional on the fixed backbones and deployment choices.

4. Keeping each result tied to its outcome

The revised statistical presentation distinguishes:

  • Evidence quality: exact F1 and contract correctness.
  • Resource use: source cost and model calls.
  • Qualification outcomes: native resolution and source-supported resolution.

The original sealed criterion and the correction families for subsequent exploratory controls remain explicit. This lets readers assess the reported gains at their appropriate level while seeing the complete fixed-interface and agent comparisons.

5. Section 8: connecting answer accuracy to documentary support

Section 8 and Appendix J express the relationship between answer correctness and evidence support through complementary quantities:

  • Native accuracy, A: the percentage of all cases with a correct task answer.
  • Joint accuracy, J: the percentage of all cases with both a correct answer and sufficient acquired-source evidence.
  • Conditional support, H: the fraction of native-correct answers with sufficient evidence. Table 24 displays 100H = 100J/A percent, for A > 0.
  • Native-to-joint gap, G: A − J, in percentage points.

These definitions distinguish supported yield across all cases from evidence support among correct answers. They also separate a percentage-point accuracy gap from a conditional percentage.

Tables 6–7 organize the all-case and component comparisons, while Table 24 and Figure 6(c,d) connect correct answers to the evidence supporting them. This gives financial-report reasoning and public-claim verification a direct relationship to the documentary requirements of buyer qualification.

6. What the method provides for buyer qualification

We have incorporated this structure into the revised manuscript, figures, and conclusion. The resulting record is useful in both cases:

  • Resolved: the acquired evidence supports the recommendation.
  • Unresolved: the record identifies the requirement that still needs verification.

PIER and CCIR make this distinction part of the method itself.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.