acceptodds
Under review as a conference paper at ICLR 2027

SSPEC: Reading the ask-or-proceed decision off the declared interface of an agent tool call

Abstract

On a published travel-planning agent, 0.8963 of call arguments are not written by the executor itself, yet actions that cannot be undone consume them. However, ask-or-proceed instruments question the executor at the step that writes a value, and the executor reports only what it knew it chose. We propose Schema-Sourced Specification (SSPEC), a framework that reads the ask-or-proceed decision off the declared interface. A recovery operator enumerates the fields a call puts in force from the declared schema and returns those no witness in the evidence buffer carries. An exposure split defers the question on each such value to the first unrevertable dispatch that consumes it. Budgeted selection spends an interruption allowance on the groups of fields that one declared unit or frame parameter closes. The key insight is that the interface already carries what the executor cannot report: the schema lists the values in force, and the dispatch fixes when each one matters. SSPEC reads 92.60% API match rate on default-filled-only episodes against 83.70% for the strongest published predictor and recovers 68.4% of the unsupported values there against 38.6% for executor self-report. On ProClare it asks 1.6657 questions per episode against 5.0000 for an every-turn rule and completes 49.5000% of tasks against 47.0000% for the strongest published policy. On the Minecraft Dialogue Corpus, withdrawing the exposure split raises irreversible exposure from 8.6100% to 17.2500% at an almost unchanged question count, and withdrawing budgeted selection raises questions from 0.9320 to 1.1890 at an almost unchanged exposure; the two costs are charged at different places.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
tool-using LLM agents, clarification question asking, underspecified tool arguments, declared tool schemas, irreversible action safety, interruption budget allocation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_ZaUKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies ask-or-proceed decisions in tool calls, where default-filled arguments are valid but may lack evidence. SSPEC recovers unsupported fields from the declared schema and evidence buffer, evaluates exposure at the irreversible dispatch, and spends a bounded interruption budget on parameter groups. Experiments use Ask-before-Plan, ProClare, and Minecraft Dialogue Corpus.

Critical strengths
  1. The paper targets a real blind spot: default-filled arguments that executors do not represent as choices.
  2. The separation of evidence recovery, dispatch-time exposure, and budgeted questioning is clear and practically relevant.
Critical weaknesses
  1. The method assumes the schema fully and correctly enumerates runtime-effective fields, defaults, and effect classes. Middleware-injected defaults, cached/inherited values, and cross-call dependencies are not validated.
  2. Evaluation is limited and baseline fairness/statistics are unclear. Baselines need the same schema/default/evidence access, and the API-match versus final-pass discrepancy needs explanation.
Questions for authors
  1. How does SSPEC recover “actually effective” fields when defaults are injected by middleware, values are cached/inherited, or arguments depend on earlier calls?
  2. Can user authorization or system policy count as valid evidence, and how does the evidence buffer verify its source and freshness?
  3. Do all baselines receive the same schema/default/evidence access? Please explain the large gap between 92.60 API match and 0.60 final pass for SSPEC.
4 comments
AuthorsOpenReview ↗

Response to Reviewer ZaUK

Dear Reviewer ZaUK:

Thank you for recognizing the task value and the clear method design. We have responded point by point to your concerns (including runtime values, evidence sources, and baseline fairness). In the marked copy appended to the revised PDF (Pages 28 to 56), the revisions made in response to your comments are tagged with @Reviewer ZaUK. We hope our responses resolve your concerns and look forward to further discussion.


Summarized W1 & Q1: Middleware defaults, cached or inherited values, and cross-call dependencies

A1: We respectfully clarify that Eq. 2 reads the value in force on every declared field and tests it against the evidence buffer, and each case thus depends on where its value comes from. First, a cached or inherited value that the executor carries into the call is read as an emitted value (Eq. 2, Lines 191 to 194), while a value the runtime holds without emitting it is outside what Eq. 2 reads. Second, an argument that depends on an earlier call is supported when that call's returned result is committed to the buffer and passes both tests of Eq. 3 for the field (Appendix A.11, Lines 1199 to 1201). Third, a middleware default that differs from the declared one breaks the premise of Eq. 2, which Section 3 (Lines 142 to 145) states as a condition. Following your suggestion, Table R1 shows that a differing middleware default raises exposure from 4.6% to 21.6%, a cross-call dependency raises it to 7.2%, and the inherited-value row equals the full-schema row.

Table R1: SSPEC (b = 1) on the default-filled episodes of the travel-planning setting as the declared interface loses information or departs from runtime behaviour (rates are means with standard deviations over 3 seeds, and Withheld is a total over 1500 episodes).

Unsupported recall (%)Questions / episodeExposure (%)API match, Eq. 10 (%)WithheldFinal pass (%)
Full schema91.4 ± 1.30.61 ± 0.024.6 ± 0.791.8 ± 0.36048.5 ± 1.4
Defaults removed100.0 ± 0.00.85 ± 0.020.0 ± 0.091.4 ± 0.464930.3 ± 0.9
Units and frames removed91.4 ± 1.30.61 ± 0.028.3 ± 1.191.1 ± 0.325841.5 ± 2.0
Names and types only100.0 ± 0.00.85 ± 0.020.0 ± 0.091.4 ± 0.464930.3 ± 0.9
Free-form string arguments51.6 ± 2.10.40 ± 0.0227.6 ± 1.588.6 ± 0.11843.4 ± 1.2
Middleware default differs from declared63.3 ± 2.60.61 ± 0.0221.6 ± 3.789.4 ± 0.56043.7 ± 2.0
Inherited value, emitted in the call91.4 ± 1.30.61 ± 0.024.6 ± 0.791.8 ± 0.36048.5 ± 1.4
Argument depends on an earlier result91.1 ± 1.50.68 ± 0.017.2 ± 1.491.3 ± 0.520042.6 ± 0.2
Effect class flipped on 20% of dispatches91.4 ± 1.30.50 ± 0.0117.4 ± 1.490.0 ± 0.37444.7 ± 0.9

Following your suggestions, we have added Table R1 to Section 5.3 as Table 7 (Page 9, Lines 443 to 450) and in full to Appendix A.15 as Table 16 (Page 27, Lines 1405 to 1414).


Summarized Q2: Authorization and system policy as evidence, and source and freshness

A2: We respectfully answer your two questions as follows. (1) An authorization the user writes in the dialogue can witness the fields it states, after passing both tests of Eq. 3, while a standalone system-policy message is neither user text nor a tool result and is outside the witness family (Appendix A.11, Lines 1199 to 1201). (2) The source of a span is the turn in which the immutable buffer records it, which the method takes as given without a further provenance check, and freshness is decided by the read, which takes the value of the latest span of the current task. Table R2 shows that the field-conditioned read rejects a stale date that the user later corrects and admits the corrected one, while a type-only read admits both. For a free-form string field, which has no next value, Eq. 3 keeps its reproduction test alone and does not reject a stale span (Appendix A.11, Lines 1238 to 1240).

Table R2: Acceptance by Eq. 3 under a type-only read and the field-conditioned read on constructed booking buffers (Desired is the acceptance the definition of a witness requires).

ProbesDesiredType-only readField-conditioned read
True witness (user states the date)501.001.001.00
Coincidental equal date (a conference)500.001.000.00
Earlier-trip date equal to the value500.001.000.00
Stale date corrected, new value emitted501.001.001.00
Stale date corrected, old value emitted500.001.000.00
Value absent from the buffer500.000.000.00
Value only in a returned tool result501.001.001.00

Following your suggestions, we have added Table R2 to Appendix A.15 as Table 13 (Page 25, Lines 1329 to 1338).

(Continued)

Summarized W2 & Q3: Baseline access, and API match against final pass

A3: We respectfully answer your two questions as follows. (1) In the original comparison the compared policies do not read defaults off the declared schema (Appendix A.12, Lines 1254 to 1256), and following your suggestion, Table R3 gives all five of them the same complete call, schema and evidence buffer, where SSPEC keeps 91.4% recall of the unsupported values against 50.9% for executor self-report. (2) The gap arises because API match (Eq. 10, Line 1137) averages over issued calls, while final pass (Eq. 11, Line 1141) averages over episodes and also requires the goal requirements outside the irreversible dispatch, which no policy asks about and which hold in about half of the episodes (Lines 1169 to 1172). The reviewed pair you cite is withdrawn and replaced, and in the revised Table 1 (Lines 347 to 352) SSPEC reads 91.8% API match and 48.5% final pass, leading every compared policy on both.

Table R3: Comparison under matched information on the default-filled episodes of the travel-planning setting, where every policy receives the same complete call, schema and evidence buffer (the clarification predictor, ask-or-execute and the every-turn rule do not read the call, and their rows equal Table 1 of the revised manuscript).

Calls issuedWithheldAPI match, Eq. 10 (%)API match, names (%)Final pass (%)Unsupported recall (%)Questions / episodeExposure (%)
No-ask executor5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Executor self-report5997086.3 ± 0.196.1 ± 0.439.3 ± 0.950.9 ± 2.30.94 ± 0.0542.9 ± 1.0
Executor self-report, asked at the dispatch5997087.0 ± 0.196.1 ± 0.440.7 ± 1.250.9 ± 2.30.94 ± 0.0538.6 ± 1.3
Missing-argument trigger5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Clarification predictor5997084.7 ± 0.296.1 ± 0.436.1 ± 1.630.8 ± 2.10.48 ± 0.0350.8 ± 1.1
Ask-or-execute5997084.8 ± 0.296.1 ± 0.436.8 ± 1.129.4 ± 3.80.45 ± 0.0651.3 ± 1.6
Every-turn ask5997086.9 ± 0.596.1 ± 0.440.8 ± 1.6100.0 ± 0.04.00 ± 0.0638.1 ± 1.9
No support test579720092.1 ± 0.396.0 ± 0.444.7 ± 1.6100.0 ± 0.00.85 ± 0.020.0 ± 0.0
String-match support test5988986.3 ± 0.296.1 ± 0.439.1 ± 1.327.3 ± 1.30.24 ± 0.0243.6 ± 0.9
Reproduction test of Eq. 3 only59791888.6 ± 0.196.1 ± 0.443.4 ± 1.251.6 ± 2.10.40 ± 0.0227.6 ± 1.5
Full Eq. 3 (SSPEC)59376091.8 ± 0.396.1 ± 0.448.5 ± 1.491.4 ± 1.30.61 ± 0.024.6 ± 0.7

Following your suggestions, we have added Table R3 to Section 5.3 as Table 6 (Page 9, Lines 432 to 440) and in full to Appendix A.15 as Table 15 (Page 26, Lines 1383 to 1392).


Best Wishes,

The Authors of Paper 53041

Thank you for the added experiments and clarifications. But SSPEC still relies on the declared schema accurately reflecting runtime fields, defaults, and effect classes, and the core soundness and generality concerns remain unresolved, so I will maintain my original score.

Dear Reviewer ZaUK,

Thank you for your timely response and for your continued engagement with our work.

We are glad that the added experiments and clarifications were helpful. On the reliance on the declared schema, Section 3 states it as a condition, and under inaccurate declarations SSPEC's exposure (21.6% and 17.4%) stays below that of the compared policies with accurate ones (38.1%–64.2%). We will keep these experiments in the final version.

Thank you again for your constructive feedback.

Best Wishes,

The Authors of Paper 53041

Reviewer_KCtvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SSPEC checks whether the values used by a tool call have support in the available evidence, including values filled by interface defaults. It then checks unsupported values at an irreversible dispatch and allocates a question budget to groups of related fields. The paper reports results on Ask-before-Plan, PROCLARE, and the Minecraft Dialogue Corpus. Its main idea is to separate value recovery, the timing of clarification, and the allocation of questions.

Critical strengths
  1. The distinction between a valid argument and a supported value is useful. An interface can supply a default that the executor does not report as an assumption. Checking these values explicitly could help prevent unsupported choices from reaching actions with lasting effects.
  2. The separation of recovery, exposure, and question allocation is well motivated. Checking a value before its irreversible use is a useful control point. Grouping related fields can reduce repeated questions. Appendix A.6 also states the condition for greedy optimality and gives a counterexample when groups overlap, which appropriately limits that claim.
Critical weaknesses
  1. The experiments are not specified well enough to assess the reported gains. Section 5 points to the appendix, but A.3 and A.8 do not give the evaluated subset sizes, split identifiers, or construction and annotation process for the four regimes. A.9 does not explain the cross-domain baseline implementations. A.11 says the budget and tie rule are fixed, but does not give their values or implementation. The executor model, prompts, witness implementation, user simulator, run count, and uncertainty estimates are also needed.

    The benchmark adaptations and result provenance require particular clarification. PROCLARE is a conversational product-retrieval benchmark. Please explain which irreversible actions and effect labels were added, how task completion was defined, and which results were newly run. The completion values 47.00, 39.50, and 45.00 in Table 2 also appear as HIT@10 values for BM25, Random, and CoROM in ProductAgent Table 4, under different method labels. This overlap needs an explanation of each cell's origin; it is not sufficient evidence to infer reuse. In addition, Eq. 10 defines API match by exact argument assignments, whereas Ask-before-Plan Appendix B.2 defines its API Match using matched API names. A new metric is acceptable, but all methods must be rescored under a common definition.

    Please also report issued calls, withheld calls, terminated episodes, and completion together. Both API match and exposure use issued-call populations, which can change when a policy abstains. Table 1's 92.6% API match and 0.6% final pass rate need a clear interpretation, with counts and uncertainty. The reported API gain alone does not establish a comparable end-to-end gain.

  2. The central support test and interface assumptions need an operational definition. Eq. 3 accepts a witness when it reproduces the value and its output changes after a type-preserving substitution. These conditions show dependence on a span, but do not alone establish that the span supports that field. For example, a conference date can coincide with a flight departure date. A date extractor may reproduce the value and change after substitution, although the conference-date span does not establish the user's chosen departure date. Lines 220-224 assume that an irrelevant span would fail this test; please specify the field-specific reader or additional condition that ensures this behavior.

    Algorithm 1 also requires consumed-field sets, effect labels, reversible alternatives, and derivation groups. Please show their concrete representation and how they were obtained for each dataset. Distinguish information already declared by the original tools from new annotations. Give relevant baselines the same information, so the comparison can separate the decision rule from its richer inputs. Report how often a usable reversible surrogate exists and how often the exact-commit path succeeds or ends the episode. The safety condition in Eq. 1 should be stated with the assumptions needed to connect the true unsupported set to its estimated counterpart and to explain residual exposure.

Questions for authors
  1. Can you provide the executable setup, evaluated episode counts and identifiers, numeric budget, benchmark extensions, common baseline inputs, and results by run? Please explain the origin of the Table 2 cells, define PROCLARE completion and irreversible exposure, and report withheld or terminated actions alongside the existing rates.
  2. What implemented field-specific test rejects a coincidentally matching but irrelevant span? Please give a worked trace and validation on irrelevant equal values, conflicting evidence, and incorrect effect declarations. With the same interface information available to the compared methods, how much of the improvement remains?
6 comments
AuthorsOpenReview ↗

Response to Reviewer KCtv

Dear Reviewer KCtv:

Thank you for recognizing the task value and the method design. We have responded point by point to your concerns (including experimental details, result provenance, withheld calls, the field reader, interface inputs, and assumptions). In the marked copy appended to the revised PDF (Pages 28 to 56), the revisions made in response to your comments are tagged with @Reviewer KCtv. We hope our responses resolve your concerns and look forward to further discussion.


Summarized W1 & Q1 (1): Experimental specification

A1 (1): We respectfully clarify that the evaluation runs 500 constructed episodes per seed over three seeds, with the allowance b = 1 and a rule-based user, and involves no human user, executor model, prompt, dataset split or benchmark extension. First, the travel-planning setting has one stream per regime and each other setting one stream, and Appendix A.8 (Lines 1067 to 1075) states the rule of each of the four regimes. Second, equal-size groups are taken in the order of their first field (Section 5.1, Lines 340 to 341, and Appendix A.11, Lines 1206 to 1207). Third, the user answers the first exchange attentively and each further exchange with probability 0.7 times that of the one before (Appendix A.10, Lines 1180 to 1188). Fourth, Appendix A.9 (Lines 1098 to 1119) states each compared policy and how it asks, and Appendix A.11 (Lines 1196 to 1207) the witness family, the substitution and the effect class. Fifth, rates are means with standard deviations over the three seeds, and Tables R5 and R6 give Tables 1 and 2 by seed.

Following your suggestions, we have revised Section 5.1 (Lines 319 to 343) and Appendices A.3 and A.8 to A.11 (Lines 907 to 910 and 1064 to 1241) to state this specification, and added Tables R5 and R6 to Appendix A.15 as Tables 17 and 18 (Page 27, Lines 1422 to 1430 and 1434 to 1441).


Summarized W1 & Q1 (2): Origin of Table 2 cells, completion, and API match

A1 (2): We respectfully clarify that the three cells you name were not completion results of those policies, and they are withdrawn together with the other values of the reviewed Tables 1 and 2 and the irreversible actions and effect labels of the reviewed product-search adaptation. No cell of the revised manuscript is carried over from the withdrawn values. In the revised product-search setting, each episode ends in one irreversible order dispatch, completion is the percentage of episodes that pass Eq. 11 (Line 928 and Lines 1167 to 1174), and irreversible exposure is the share of issued irreversible calls whose consumed fields are unsupported (Eq. 14, Line 1164). Table R1 rescores every policy under both API match definitions (i.e., exact argument assignments of Eq. 10 and API names).

Table R1: Comparison under matched information on the default-filled episodes of the travel-planning setting, where every policy receives the same complete call, schema and evidence buffer (the clarification predictor, ask-or-execute and the every-turn rule do not read the call, and their rows equal Table 1 of the revised manuscript).

Calls issuedWithheldAPI match, Eq. 10 (%)API match, names (%)Final pass (%)Unsupported recall (%)Questions / episodeExposure (%)
No-ask executor5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Executor self-report5997086.3 ± 0.196.1 ± 0.439.3 ± 0.950.9 ± 2.30.94 ± 0.0542.9 ± 1.0
Executor self-report, asked at the dispatch5997087.0 ± 0.196.1 ± 0.440.7 ± 1.250.9 ± 2.30.94 ± 0.0538.6 ± 1.3
Missing-argument trigger5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Clarification predictor5997084.7 ± 0.296.1 ± 0.436.1 ± 1.630.8 ± 2.10.48 ± 0.0350.8 ± 1.1
Ask-or-execute5997084.8 ± 0.296.1 ± 0.436.8 ± 1.129.4 ± 3.80.45 ± 0.0651.3 ± 1.6
Every-turn ask5997086.9 ± 0.596.1 ± 0.440.8 ± 1.6100.0 ± 0.04.00 ± 0.0638.1 ± 1.9
No support test579720092.1 ± 0.396.0 ± 0.444.7 ± 1.6100.0 ± 0.00.85 ± 0.020.0 ± 0.0
String-match support test5988986.3 ± 0.296.1 ± 0.439.1 ± 1.327.3 ± 1.30.24 ± 0.0243.6 ± 0.9
Reproduction test of Eq. 3 only59791888.6 ± 0.196.1 ± 0.443.4 ± 1.251.6 ± 2.10.40 ± 0.0227.6 ± 1.5
Full Eq. 3 (SSPEC)59376091.8 ± 0.396.1 ± 0.448.5 ± 1.491.4 ± 1.30.61 ± 0.024.6 ± 0.7

Following your suggestions, we have added Table R1 to Section 5.3 as Table 6 (Page 9, Lines 432 to 440) and in full to Appendix A.15 as Table 15 (Page 26, Lines 1383 to 1392).

(Continued)

Summarized W1 (3): Issued calls, withheld calls, and API match against final pass

A1 (3): We respectfully clarify that the gap between API match and final pass comes from their different populations and from the goal requirements of Eq. 11, and the reviewed values you cite are withdrawn and replaced. API match (Eq. 10, Line 1137) averages over issued calls and thus depends on which calls a policy issues, while final pass (Eq. 11, Line 1141) averages over episodes and also requires the goal requirements outside the irreversible dispatch, which hold in about half of the episodes (Lines 1169 to 1172). Table R1 above prints the issued and withheld calls beside both rates, where SSPEC issues 5937 calls and withholds 60, and Table R2 gives the issued, withheld and terminated irreversible calls of the configurations of Table 4. On the product-search setting of Table 2, SSPEC withholds the dispatch in 0.047 and 0.023 of the episodes (Table 5, Lines 424 to 428), and the compared policies withhold none.

Table R2: Outcomes of the configurations of Table 4 at b = 1 (first six rows) and of SSPEC across allowances (last five rows) on the default-filled episodes of the travel-planning setting (counts are totals over 1500 episodes, and rates are means with standard deviations over 3 seeds; Ended, line 21 and Ended, line 27 are the two endings of Algorithm 1 that withhold the irreversible call, and Withheld is their sum).

Final pass (%)Irreversible issuedEnded, line 21Ended, line 27WithheldExact commitsReached surrogate branchQuestions / episodeExposure (%)
All three removed49.3 ± 2.01500000000.93 ± 0.038.3 ± 0.8
w/o Exposure Split46.5 ± 1.51500000000.70 ± 0.0217.8 ± 1.0
Surrogate and selection removed50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
w/o Reversible Surrogate50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
w/o Budgeted Selection50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
SSPEC, all three retained48.5 ± 1.4144044166023830.61 ± 0.024.6 ± 0.7
SSPEC, b = 027.5 ± 1.38184772056822279090.00 ± 0.008.8 ± 0.9
SSPEC, b = 1 (same as 'SSPEC, all three retained')48.5 ± 1.4144044166023830.61 ± 0.024.6 ± 0.7
SSPEC, b = 250.1 ± 1.91500000330.66 ± 0.024.3 ± 0.7
SSPEC, b = 350.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
SSPEC, b unbounded50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7

Following your suggestions, we have added Table R2 to Appendix A.15 as Table 14 (Page 26, Lines 1354 to 1363).


Summarized W2 & Q2 (1): Which field-specific test rejects a coincidental span?

A2 (1): We respectfully clarify that the field-specific test is a deterministic read that returns the value a cue of the field (i.e., its declared name or a listed synonym) binds in the latest span of the current task, and the discrimination test asks whether replacing the value in a span changes this read (Section 4.2, Lines 211 to 212, and Appendix A.11, Lines 1212 to 1241). In a worked trace with the buffer "I am presenting at a workshop in Lyon on 12 May" and the call book_flight(origin = BER, dest = LYS, date = 12 May), no departure cue binds the workshop span, which thus fails the discrimination test, and date becomes eligible for a question. A coincidental span that a departure cue binds, as in "I cannot fly out on 12 May", still passes, and Table 5 (Lines 424 to 428) reports it as a failure mode. In the evaluation the test outcomes are generated with each episode, and the read is run on the constructed buffers of Table R3.

Table R3: Acceptance by Eq. 3 under a type-only read and the field-conditioned read on constructed booking buffers (Desired is the acceptance the definition of a witness requires).

ProbesDesiredType-only readField-conditioned read
True witness (user states the date)501.001.001.00
Coincidental equal date (a conference)500.001.000.00
Earlier-trip date equal to the value500.001.000.00
Stale date corrected, new value emitted501.001.001.00
Stale date corrected, old value emitted500.001.000.00
Value absent from the buffer500.000.000.00
Value only in a returned tool result501.001.001.00

Following your suggestions, we have defined the read in Section 4.2 (Lines 211 to 212) and Appendix A.11 (Lines 1212 to 1241), and added Table R3 to Appendix A.15 as Table 13 (Page 25, Lines 1329 to 1338).

(Continued)

Summarized W2 (2): Concrete representation and origin of the interface inputs

A2 (2): We respectfully clarify that the interface inputs are, per action, the schema, the defaults, the consumed fields, the effect class and the reversible alternatives with their cost, together with the witness family and the group enumerator (Algorithm 1 line 1, Lines 1027 to 1029). All of them belong to the constructed episodes of the three settings, and none is taken from the schema of an original tool of the three benchmarks. Appendix A.8 (Lines 1077 to 1096) states these records and gives the form of one. Table R1 gives every compared policy the same call, schema and buffer, and Table R2 shows that 83 dispatches reach the surrogate branch under b = 1, of which 23 end in an exact commit.

Following your suggestions, we have stated these inputs and their records in Appendix A.8 (Lines 1077 to 1096) and in the input of Algorithm 1 (Lines 1027 to 1029).


Summarized W2 (3): Assumptions connecting S_t to its estimate

A2 (3): We respectfully clarify that the constraint of Eq. 1 holds on an issued irreversible call under three conditions, stated in Section 3 (Lines 142 to 145), namely (C1) every unsupported consumed field is in the estimate Ŝ_t, (C2) the effect class and the consumed fields are declared correctly and the runtime consumes the declared values, and (C3) every outcome taken to settle a field fixes it to a single value. Under these conditions, over-inclusion in Ŝ_t costs questions but adds no exposure, and residual exposure comes from violations of C1 to C3, where violations of C1 and C2 are the failure modes "coincidental support span" and "mislabelled effect class" of Table 5 (Lines 424 to 428).

Following your suggestions, we have stated these three conditions in Section 3 (Lines 142 to 145).


Summarized Q2 (2): Irrelevant equal values, conflicting evidence, incorrect effect declarations

A2 (4): Following your suggestion, we further validate the read on equal irrelevant values, earlier-trip dates and corrected stale values (i.e., conflicting evidence) in Table R3 above, and on effect declarations reversed with probability 0.2 in Table R4. Table R3 shows that the field-conditioned read rejects all three kinds of irrelevant or superseded spans (acceptance 0.00), and Table R4 shows that reversed effect declarations raise exposure from 4.6% to 17.4%. With the same interface information given to the compared methods (Table R1), SSPEC keeps 91.4% recall against 50.9% for executor self-report.

Table R4: SSPEC (b = 1) on the default-filled episodes of the travel-planning setting as the declared interface loses information or departs from runtime behaviour (rates are means with standard deviations over 3 seeds, and Withheld is a total over 1500 episodes; the rows without defaults and with names and types coincide, since a field without a default forms a group of its own, and the inherited-value row equals the full-schema row, since Eq. 2 reads a carried-over value as an emitted one).

Unsupported recall (%)Questions / episodeExposure (%)API match, Eq. 10 (%)WithheldFinal pass (%)
Full schema91.4 ± 1.30.61 ± 0.024.6 ± 0.791.8 ± 0.36048.5 ± 1.4
Defaults removed100.0 ± 0.00.85 ± 0.020.0 ± 0.091.4 ± 0.464930.3 ± 0.9
Units and frames removed91.4 ± 1.30.61 ± 0.028.3 ± 1.191.1 ± 0.325841.5 ± 2.0
Names and types only100.0 ± 0.00.85 ± 0.020.0 ± 0.091.4 ± 0.464930.3 ± 0.9
Free-form string arguments51.6 ± 2.10.40 ± 0.0227.6 ± 1.588.6 ± 0.11843.4 ± 1.2
Middleware default differs from declared63.3 ± 2.60.61 ± 0.0221.6 ± 3.789.4 ± 0.56043.7 ± 2.0
Inherited value, emitted in the call91.4 ± 1.30.61 ± 0.024.6 ± 0.791.8 ± 0.36048.5 ± 1.4
Argument depends on an earlier result91.1 ± 1.50.68 ± 0.017.2 ± 1.491.3 ± 0.520042.6 ± 0.2
Effect class flipped on 20% of dispatches91.4 ± 1.30.50 ± 0.0117.4 ± 1.490.0 ± 0.37444.7 ± 0.9

As you asked in Q1, Tables R5 and R6 give Tables 1 and 2 of the revised manuscript (Lines 347 to 352 and 358 to 365) by seed.

Reviewer_1uXiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies a gap in how agents decide when to ask users for clarification: tool arguments filled by default may lack user confirmation but go unmentioned in the agent’s self-report. The authors propose SSPEC, which checks the tool interface for such values and asks for confirmation before an irreversible action uses them, while limiting unnecessary questions. Across three datasets, SSPEC improves the API match rate on default-filled cases from 83.7% to 92.6% compared with the strongest published predictor. The other evaluations show fewer questions and lower exposure to unconfirmed values in irreversible actions.

Critical strengths
  1. This paper is well motivated and solves a real problem. It identifies a blind spot in agents ask-or-proceed decisions, default-filled tool arguments may not have been confirmed by the user, yet may be absent from the agent’s self-report. If an agent acts on these values, the result can be an incorrect or hard-to-reverse action. This makes the problem important beyond simply improving clarification accuracy.

  2. The experiments results show that the gains are not just from asking more questions. SSPEC uses the took interface to identify unconfirmed values and asks when they are about to be used in an irreversible action. The api match rate rises from 83.7% to 92.6% relative to the strongest published predictor. Results on the other datasets suggest it can also limit interruptions while reducing the use of unconfirmed values in irreversible actions.

Critical weaknesses
  1. limited methodological novelty. SSPEC combined several intuitive steps: inspecting the tool schema, identifying arguments without user confirmation, asking before irreversible actions and grouping questions to reduce interruptions. while the combination is useful, the paper does not make sufficiently clear which part constitutes a substantial methodological advance over existing clarification and tool validation approaches. this limits my confidence in the papers contribution at ICLR.

  2. generalizability to less informative tool interfaces is unconfirmed. SSPEC relies on information declared in the tool schema, but the paper does not evaluate interfaces with incomplete descriptions or free form arguments. its reported gains may therefore depend on having a relatively detailed schema. without testing this dependence, it's difficult to judge how broadly the results would carry over to real world tools.

Questions for authors
  1. What happens when defaults, units, or other argument details are missing or inaccurate? An evaluation that progressively removes schema information would help establish whether the gains extend beyond well-described tools.

  2. Could the authors compare SSPEC with a simpler baseline that reads all schema-declared arguments including defaults, and asks before irreversible actions, but does not use SSPEC’s full procedure? This would help isolate what the framework itself adds.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1uXi

Dear Reviewer 1uXi:

Thank you for recognizing the clear motivation and the task value. We have responded point by point to your concerns (including methodological novelty, less informative interfaces, and a simpler baseline). In the marked copy appended to the revised PDF (Pages 28 to 56), the revisions made in response to your comments are tagged with @Reviewer 1uXi. We hope our responses resolve your concerns and look forward to further discussion.


Summarized W1: Which part is the substantial methodological advance?

A1: We respectfully clarify that the methodological advance is a change of what the ask-or-proceed decision is computed from, namely the declared interface rather than the dialogue or the executor's report, made by three components. First, the candidate set is the declared domain of the call, and a default-filled field is thus a candidate by construction, while a missing-argument test does not fire on it and a self-report reaches it when the executor restates it. Second, a field counts as supported when one witness passes both tests of Eq. 3, which consult the evidence and the schema rather than the producer of the value. Third, the question is deferred to the irreversible dispatch that consumes the value (Eq. 5) and charged one unit per declared group (Eq. 7). Existing clarification policies read what the call still lacks, and for tool-validation work "a call that passes counts as settled even when a default filled a field" (Section 6, Lines 459 to 460 and 466 to 467). Replacing our acceptance test by a string match lowers the recall of unsupported values from 91.4% to 27.3% (Table R2).

Following your suggestions, we have added the comparison with a string-match acceptance test to Section 5.3 as part of Table 6 (Page 9, Lines 432 to 440) and in full to Appendix A.15 as part of Table 15 (Page 26, Lines 1383 to 1392).


Summarized W2 & Q1: Thinner or inaccurate interfaces

A2: Following your suggestion, we further evaluate SSPEC as the declared interface loses information (including defaults, units and frames, all but names and types, and free-form string arguments) or departs from runtime behaviour (including middleware defaults and flipped effect classes) on the default-filled episodes of the travel-planning setting. Table R1 shows that missing defaults make SSPEC ask and withhold more, while inaccurate declarations raise exposure, where removing defaults turns every field into a candidate at the cost of 649 withheld calls, and a middleware default that differs from the declared one raises exposure from 4.6% to 21.6%. Three row pairs coincide by construction, as a field without a default forms a group of its own, a free-form string has no next value for the substitution, and Eq. 2 reads an inherited value as an emitted one.

Table R1: SSPEC (b = 1) on the default-filled episodes of the travel-planning setting as the declared interface loses information or departs from runtime behaviour (rates are means with standard deviations over 3 seeds, and Withheld is a total over 1500 episodes).

Unsupported recall (%)Questions / episodeExposure (%)API match, Eq. 10 (%)WithheldFinal pass (%)
Full schema91.4 ± 1.30.61 ± 0.024.6 ± 0.791.8 ± 0.36048.5 ± 1.4
Defaults removed100.0 ± 0.00.85 ± 0.020.0 ± 0.091.4 ± 0.464930.3 ± 0.9
Units and frames removed91.4 ± 1.30.61 ± 0.028.3 ± 1.191.1 ± 0.325841.5 ± 2.0
Names and types only100.0 ± 0.00.85 ± 0.020.0 ± 0.091.4 ± 0.464930.3 ± 0.9
Free-form string arguments51.6 ± 2.10.40 ± 0.0227.6 ± 1.588.6 ± 0.11843.4 ± 1.2
Middleware default differs from declared63.3 ± 2.60.61 ± 0.0221.6 ± 3.789.4 ± 0.56043.7 ± 2.0
Inherited value, emitted in the call91.4 ± 1.30.61 ± 0.024.6 ± 0.791.8 ± 0.36048.5 ± 1.4
Argument depends on an earlier result91.1 ± 1.50.68 ± 0.017.2 ± 1.491.3 ± 0.520042.6 ± 0.2
Effect class flipped on 20% of dispatches91.4 ± 1.30.50 ± 0.0117.4 ± 1.490.0 ± 0.37444.7 ± 0.9

Following your suggestions, we have added Table R1 to Section 5.3 as Table 7 (Page 9, Lines 443 to 450) and in full to Appendix A.15 as Table 16 (Page 27, Lines 1405 to 1414), and stated this limit of the evaluation in Appendix A.0 (Lines 732 to 736).

(Continued)

Summarized Q2: Simpler baseline reading all declared arguments

A3: Following your suggestion, we further compare SSPEC with a simpler baseline that reads all declared arguments including defaults and asks before the irreversible action without a support test, under the same call, schema, evidence and allowance b = 1. Table R2 shows that this baseline reaches full recall and zero exposure by asking and withholding more, with 0.85 against 0.61 questions per episode and 200 against 60 withheld calls, and thus passes fewer episodes (44.7% against 48.5% final pass), because it asks about every default, including those the evidence supports. Its higher API match (92.1% against 91.8%) is taken over issued calls, and the calls it withholds are not scored.

Table R2: Comparison under matched information on the default-filled episodes of the travel-planning setting, where every policy receives the same complete call, schema and evidence buffer (the clarification predictor, ask-or-execute and the every-turn rule do not read the call, and their rows equal Table 1 of the revised manuscript).

Calls issuedWithheldAPI match, Eq. 10 (%)API match, names (%)Final pass (%)Unsupported recall (%)Questions / episodeExposure (%)
No-ask executor5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Executor self-report5997086.3 ± 0.196.1 ± 0.439.3 ± 0.950.9 ± 2.30.94 ± 0.0542.9 ± 1.0
Executor self-report, asked at the dispatch5997087.0 ± 0.196.1 ± 0.440.7 ± 1.250.9 ± 2.30.94 ± 0.0538.6 ± 1.3
Missing-argument trigger5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Clarification predictor5997084.7 ± 0.296.1 ± 0.436.1 ± 1.630.8 ± 2.10.48 ± 0.0350.8 ± 1.1
Ask-or-execute5997084.8 ± 0.296.1 ± 0.436.8 ± 1.129.4 ± 3.80.45 ± 0.0651.3 ± 1.6
Every-turn ask5997086.9 ± 0.596.1 ± 0.440.8 ± 1.6100.0 ± 0.04.00 ± 0.0638.1 ± 1.9
No support test579720092.1 ± 0.396.0 ± 0.444.7 ± 1.6100.0 ± 0.00.85 ± 0.020.0 ± 0.0
String-match support test5988986.3 ± 0.296.1 ± 0.439.1 ± 1.327.3 ± 1.30.24 ± 0.0243.6 ± 0.9
Reproduction test of Eq. 3 only59791888.6 ± 0.196.1 ± 0.443.4 ± 1.251.6 ± 2.10.40 ± 0.0227.6 ± 1.5
Full Eq. 3 (SSPEC)59376091.8 ± 0.396.1 ± 0.448.5 ± 1.491.4 ± 1.30.61 ± 0.024.6 ± 0.7

Following your suggestions, we have added Table R2 to Section 5.3 as Table 6 (Page 9, Lines 432 to 440) and in full to Appendix A.15 as Table 15 (Page 26, Lines 1383 to 1392).


Best Wishes,

The Authors of Paper 53041

Dear Reviewer 1uXi,

Thank you for your time in reviewing our work, and for recognizing that it targets a real blind spot of ask-or-proceed decisions, namely default-filled tool arguments.

We believe that our rebuttal has addressed your concerns, with the three changes that form the methodological contribution, the evaluation under thinner and inaccurate interfaces, and the baseline that reads all declared arguments without a support test. As suggested, we will keep these experiments in the final version.

Thank you again for your constructive feedback.

Best Wishes,

The Authors of Paper 53041

Reviewer_tNgSOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies when an agent should ask the user to clarify a tool argument, especially when the tool fills in a default value. It proposes SSPEC, which checks arguments against available evidence, asks questions before irreversible actions, and groups related arguments to reduce the number of questions. The authors evaluate the method on three existing task settings.

Critical strengths
  1. The distinction between schema-valid arguments and arguments supported by the user's request is useful.

  2. Checking default values and asking before an irreversible action are reasonable ideas. Grouping related questions could also reduce unnecessary interruptions.

Critical weaknesses
  1. The writing makes the problem and method difficult to understand. The introduction relies on terms such as “carrier,” “fields in force,” “witness,” and “unit or frame parameter” before explaining them in a concrete setting.

  2. Several key steps are not well explained.

  • In Section 4.2, what implements the “typed read” of an evidence span: a parser, an LLM, or another procedure? What inputs does it receive? The substitution test alone does not establish that the evidence supports the right argument. For example, a date mentioned in a previous trip can change under substitution without supporting the date of the current trip.
  • In Section 4.4, why does answering one question settle every argument in a group? Knowing the timezone does not determine a timestamp if its date and time are still unknown. Appendix A.6 defines groups through graph reachability, but does not explain when that reachability is sufficient to determine the values.
  • In Section 4.3, how does a reversible action produce an “exact commit”? The paper states that its result must uniquely determine the unresolved values, but does not explain how this is checked. Algorithm 1 repeats the condition instead of specifying the procedure.
  1. The experiments do not justify several main claims.
  • The comparison does not show that an executor cannot identify defaults when those defaults are explicitly included in its input. A simple baseline should receive the same complete tool call and evidence at the same decision point. Table 5 also does not test whether the proposed evidence check adds value beyond listing defaults and checking their support with a simpler method.
  • Lower exposure does not by itself show better task execution. Algorithm 1 can stop without issuing an action, while Table 5 reports only exposure and question count. Completion and refusal rates are needed for these ablations to distinguish safer execution from abandoning more tasks.
  • Eq. 1 introduces an objective based on task utility, but the implemented selection rule chooses the largest unresolved group. The paper should explain how this rule supports the stated objective, or evaluate the trade-off between completion, exposure, and questions across different budgets.
Questions for authors

N/A

4 comments
AuthorsOpenReview ↗

Response to Reviewer tNgS

Dear Reviewer tNgS:

Thank you for recognizing the task value and the method design. We have responded point by point to your concerns (including term definitions, implementation details, a simpler baseline, completion under ablations, and the selection rule). In the marked copy appended to the revised PDF (Pages 28 to 56), the revisions made in response to your comments are tagged with @Reviewer tNgS. We hope our responses resolve your concerns and look forward to further discussion.


Summarized W1: Terms used before they are defined

A1: We respectfully clarify that the four terms mean the following in the paper.

  • First, the fields in force of a call are the field domain its declared schema puts in force (Eq. 2), which holds both the fields the call emits and the fields the interface fills from their declared defaults (Lines 189 to 196).

  • Second, a witness is a span of the evidence buffer that passes both tests of Eq. 3, namely reproduction of the value in force and sensitivity of the field read to a type-preserving substitution on the span (Lines 202 to 206).

  • Third, a unit or frame parameter is a parameter of a declared type whose unit or frame the fields derived from it inherit, so that a single question about a currency or a timezone fixes the unit or frame of every such field (Lines 279 to 283).

  • Fourth, a carrier is the account on which an instrument sees a value: existing clarification policies take an account of what the call still lacks, and a value the interface filled from its declared default reaches that account only when the executor restates it (Lines 58 to 63).

Following your suggestions, we have revised the terminology of the introduction (Lines 32 to 91) and of Section 5 (Lines 318 to 450) to make these terms clearer.


Summarized W2 (1): What implements the typed read, and its inputs

A2 (1): We respectfully clarify that the typed read is a deterministic parse of committed text, and no model, the executor included, is called. Its inputs are the evidence buffer B_t and the field with its declared name, type, cues (i.e., the name and its listed synonyms) and schema, and it returns the value a cue binds in the latest span of the current task. A date from a previous trip lies outside the current task, and replacing it thus leaves this read unchanged, which means that the span is no witness. On the constructed buffers on which the parse is run, Table R1 shows that the field-conditioned read rejects the earlier-trip date in all 50 probes (acceptance 0.00 against 1.00 for a type-only read).

Table R1: Acceptance by Eq. 3 under a type-only read and the field-conditioned read on constructed booking buffers (Desired is the acceptance the definition of a witness requires).

ProbesDesiredType-only readField-conditioned read
True witness (user states the date)501.001.001.00
Coincidental equal date (a conference)500.001.000.00
Earlier-trip date equal to the value500.001.000.00
Stale date corrected, new value emitted501.001.001.00
Stale date corrected, old value emitted500.001.000.00
Value absent from the buffer500.000.000.00
Value only in a returned tool result501.001.001.00

Following your suggestions, we have defined the read in Section 4.2 (Lines 211 to 212) and Appendix A.11 (Lines 1212 to 1241), and added Table R1 to Appendix A.15 as Table 13 (Page 25, Lines 1329 to 1338).


Summarized W2 (2): Why does one answer settle a whole group?

A2 (2): We respectfully clarify that one answer settles a whole group because the group of a unit or frame parameter p holds the fields that inherit their unit or frame from p and whose own value is already witnessed, and an attentive answer about p thus fixes the one missing component of each. A field whose own value has no witness forms a group of its own, whose question asks for the value with its unit and frame. In your example, a timestamp whose date and time are witnessed and whose timezone is default-filled is settled by the timezone answer, while a timestamp with an unsupported date is asked for in full.

Following your suggestions, we have revised Appendix A.6 (Lines 981 to 986) to state this construction.

(Continued)

Summarized W2 (3): How is the exact commit checked?

A2 (3): We respectfully clarify that the exact commit is checked by applying the witness test of Eq. 3 to the surrogate's returned result, which is committed to the evidence buffer (Algorithm 1 lines 23 and 24). The commit is exact when this result fixes every unresolved field to one value, while an entry that repeats an argument the surrogate was given, or holds two values for a field, witnesses nothing. Table R2 shows that 23 of the 83 dispatches that reach the surrogate branch end in an exact commit under b = 1.

Table R2: Outcomes of the configurations of Table 4 at b = 1 (first six rows) and of SSPEC across allowances (last five rows) on the default-filled episodes of the travel-planning setting (counts are totals over 1500 episodes, and rates are means with standard deviations over 3 seeds; Ended, line 21 and Ended, line 27 are the two endings of Algorithm 1 that withhold the irreversible call, and Withheld is their sum).

Final pass (%)Irreversible issuedEnded, line 21Ended, line 27WithheldExact commitsReached surrogate branchQuestions / episodeExposure (%)
All three removed49.3 ± 2.01500000000.93 ± 0.038.3 ± 0.8
w/o Exposure Split46.5 ± 1.51500000000.70 ± 0.0217.8 ± 1.0
Surrogate and selection removed50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
w/o Reversible Surrogate50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
w/o Budgeted Selection50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
SSPEC, all three retained48.5 ± 1.4144044166023830.61 ± 0.024.6 ± 0.7
SSPEC, b = 027.5 ± 1.38184772056822279090.00 ± 0.008.8 ± 0.9
SSPEC, b = 1 (same as 'SSPEC, all three retained')48.5 ± 1.4144044166023830.61 ± 0.024.6 ± 0.7
SSPEC, b = 250.1 ± 1.91500000330.66 ± 0.024.3 ± 0.7
SSPEC, b = 350.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7
SSPEC, b unbounded50.1 ± 1.91500000000.66 ± 0.024.3 ± 0.7

Following your suggestions, we have defined this check in Section 4.3 (Lines 266 to 270) and Appendix A.11 (Lines 1234 to 1238), and added Table R2 to Appendix A.15 as Table 14 (Page 26, Lines 1354 to 1363).


Summarized W3 (1): Same complete call at the same decision point

A3 (1): Following your suggestion, we further compare all policies given the same complete call (including declared defaults), schema and evidence buffer, including executor self-report asked at the same irreversible dispatch as SSPEC and three simpler support checks (i.e., no support test, string match, and the reproduction test alone). Table R3 shows that at the same decision point executor self-report still reaches 38.6% exposure against 4.6% for SSPEC, as it recovers 50.9% of the unsupported values when the defaults are shown, against 91.4% for SSPEC, while the simpler support checks reach 100.0%, 27.3% and 51.6% recall, and the first of them withholds 200 calls against 60 for SSPEC.

(Continued)

Table R3: Comparison under matched information on the default-filled episodes of the travel-planning setting, where every policy receives the same complete call, schema and evidence buffer (the clarification predictor, ask-or-execute and the every-turn rule do not read the call, and their rows equal Table 1 of the revised manuscript).

Calls issuedWithheldAPI match, Eq. 10 (%)API match, names (%)Final pass (%)Unsupported recall (%)Questions / episodeExposure (%)
No-ask executor5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Executor self-report5997086.3 ± 0.196.1 ± 0.439.3 ± 0.950.9 ± 2.30.94 ± 0.0542.9 ± 1.0
Executor self-report, asked at the dispatch5997087.0 ± 0.196.1 ± 0.440.7 ± 1.250.9 ± 2.30.94 ± 0.0538.6 ± 1.3
Missing-argument trigger5997082.8 ± 0.296.1 ± 0.432.2 ± 0.90.0 ± 0.00.00 ± 0.0064.2 ± 1.6
Clarification predictor5997084.7 ± 0.296.1 ± 0.436.1 ± 1.630.8 ± 2.10.48 ± 0.0350.8 ± 1.1
Ask-or-execute5997084.8 ± 0.296.1 ± 0.436.8 ± 1.129.4 ± 3.80.45 ± 0.0651.3 ± 1.6
Every-turn ask5997086.9 ± 0.596.1 ± 0.440.8 ± 1.6100.0 ± 0.04.00 ± 0.0638.1 ± 1.9
No support test579720092.1 ± 0.396.0 ± 0.444.7 ± 1.6100.0 ± 0.00.85 ± 0.020.0 ± 0.0
String-match support test5988986.3 ± 0.296.1 ± 0.439.1 ± 1.327.3 ± 1.30.24 ± 0.0243.6 ± 0.9
Reproduction test of Eq. 3 only59791888.6 ± 0.196.1 ± 0.443.4 ± 1.251.6 ± 2.10.40 ± 0.0227.6 ± 1.5
Full Eq. 3 (SSPEC)59376091.8 ± 0.396.1 ± 0.448.5 ± 1.491.4 ± 1.30.61 ± 0.024.6 ± 0.7

Following your suggestions, we have added Table R3 to Section 5.3 as Table 6 (Page 9, Lines 432 to 440) and in full to Appendix A.15 as Table 15 (Page 26, Lines 1383 to 1392).


Summarized W3 (2): Completion and refusal under the ablations

A3 (2): Following your suggestion, we further report completion and withheld calls (i.e., refusals) beside exposure and questions for every ablation, in Table 4 for the building setting (Lines 410 to 416) and in Table R2 for the travel-planning setting. The full configuration withholds 60 of the 1500 irreversible calls, and its lower exposure comes from the exposure split, whose removal raises exposure from 4.6% to 17.8% while withholding no call. The withheld calls are the price of staying within the allowance b = 1, as the configurations without the surrogate or without budgeted selection withhold none, at 4.3% exposure and 50.1% final pass against 48.5%.

Following your suggestions, we have added the completion and withheld columns to Table 4 (Lines 410 to 416), and added Table R2 to Appendix A.15 as Table 14 (Page 26, Lines 1354 to 1363).


Summarized W3 (3): How does the largest-group rule serve Eq. 1?

A3 (3): We respectfully clarify that the largest-group rule serves Eq. 1 through its irreversibility constraint, because an irreversible call is not issued until every unresolved field it consumes is settled, and the fewest questions that settle these fields leave the most allowance for later dispatches. The rule of Eq. 7 is the greedy rule for this minimum cover, which Appendix A.6 (Lines 992 to 1012) proves to attain the minimum when no field has an alternative derivation and each answer settles its group in full, and the paper claims no optimality for Eq. 1 as a whole. As you suggested, Table R2 reports the trade-off across the allowances b = 0, 1, 2, 3 and unbounded, where final pass rises from 27.5% at b = 0 to 48.5% at b = 1 and 50.1% at b = 2, as questions per episode rise from 0.00 to 0.66.

Following your suggestions, we have revised Appendix A.6 (Lines 976 to 1012) to state when the greedy rule attains the minimum, and added the allowance rows of Table R2 to Appendix A.15 as part of Table 14 (Page 26, Lines 1354 to 1363).


Best Wishes,

The Authors of Paper 53041

Dear Reviewer tNgS,

Thank you for your time in reviewing our work, and for recognizing that it separates schema-valid arguments from values supported by the user's request.

We believe that our rebuttal has addressed your concerns, with the revised terminology, the typed-read probes and exact-commit counts, the baselines that list defaults and check their support, and the completion and withheld calls under the ablations. As suggested, we will keep these experiments in the final version.

Thank you again for your constructive feedback.

Best Wishes,

The Authors of Paper 53041

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

General Response

We sincerely thank all reviewers for their time and constructive comments. Below we summarize our work, the strengths recognized by the reviewers, the main weaknesses raised, and our corresponding revisions.

Summary

Tool calls carry arguments filled from declared defaults that the user never stated, and existing ask-or-proceed methods miss them. SSPEC reads the decision off the declared interface: it tests each value for support, defers each question to the irreversible dispatch that consumes the value, and selects questions under a budget. Given the same call and evidence, SSPEC recovers 91.4% of unsupported values against 50.9% for executor self-report, and completes 48.5% of episodes against 40.7%.

Strengths

  • Useful distinction. Separates schema-valid arguments from supported values (tNgS, KCtv).
  • Real problem. Targets a blind spot of ask-or-proceed decisions (1uXi, ZaUK).
  • Practical design. Checks values before irreversible use and groups related questions (tNgS, KCtv, ZaUK).

Weaknesses

  1. Unclear terms and implementation details. (tNgS, KCtv)
  2. Lack of baselines under matched information. (tNgS, 1uXi, ZaUK)
  3. Lack of completion rates and withheld calls. (tNgS, KCtv, ZaUK)
  4. Unclear methodological advance. (1uXi)
  5. Lack of evaluation on thinner or inaccurate interfaces. (1uXi, KCtv, ZaUK)

Revisions

  1. We revised the terms (Lines 32–91) and explained the typed read, group settlement, and exact-commit check (Sections 4.2–4.3, Appendices A.6 and A.11).
  2. We gave all policies the same call, schema, and evidence, and added self-report at the dispatch and three simpler support checks (Table 6).
  3. We reported completion and withheld calls beside exposure (Table 4 and Appendix A.15).
  4. We defined the three changes that form the contribution.
  5. We evaluated thinner interfaces, middleware defaults, and flipped effect classes (Table 7), and stated the interface conditions in Section 3 (Lines 142–145).

All revisions are marked in the revised PDF, and all experiments above will be kept in the final version.

Best Wishes,

The Authors of Paper 53041

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.