- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the migration of an already-issued tool call when an API or MCP interface changes before execution. Its main observation is that a rewritten call can satisfy the new schema and reuse existing values while assigning those values to different semantic roles. The proposed system, ContractPatch, constructs a migrated call and a certificate that identifies the target operation, field correspondences, permitted deletions, and justified new literals. An independent verifier reconstructs these relationships, checks target-schema validity and location-aware value conservation, and abstains when the supported correspondence is ambiguous. The paper reports 60/60 safe adaptations and 45/45 correct refusals on its main benchmark. On a frozen Notion release pair, it safely migrates 11/15 adaptable calls and refuses all six unsupported calls. Additional evaluation includes held-out providers, 20,000 fuzz trials, and 28 endpoint execution steps.
Critical strengths
-
The paper identifies a concrete limitation of ordinary call validation. Section 3.4 shows why copying an observed database_id into data_source_id can pass schema validation and introduce no new literal while lacking evidence that the identifier domains correspond. The distinction between incorrect migrations and executions that should have been refused is also explicitly captured by (U_M) and (U_A). This makes the evaluation more informative than measuring schema validity or binary migration decisions alone.
-
The evaluation examines both safety and the cost of conservative refusal. The frozen Notion experiment reports four missed adaptations rather than hiding them behind a perfect safety score, yielding 73.3% normalized coverage. Held-out GitHub cases and actual REST/MCP execution checks provide additional evidence beyond static schema matching. These results support the practical value of selective migration within a deliberately bounded transformation language.
Critical weaknesses
-
The semantic justification of the correspondence rules remains insufficiently established. Field alignment relies on reciprocal unique lexical matches, while deletion is permitted when no plausible replacement exists; the verifier recomputes the same rule language. Theorem 1 establishes compliance with these rules and structural invariants, but does not independently establish that a lexical correspondence preserves an argument’s meaning or that deleting a live argument preserves its constraints. The paper acknowledges its bounded guarantee, but should specify these acceptance rules more precisely and justify their semantic scope; this is a gap in support for the authorization interpretation, not a demonstrated unsafe execution.
-
The main comparison does not isolate the advantage over simpler conservative migration rules. Table 5 shows that 58/60 adaptable main-benchmark cases involve optional-field additions, field removals, or requiredness tightening; all type changes, shape replacements, and tool splits require abstention. Without a deterministic schema-diff baseline or component ablations, perfect main-benchmark performance does not establish how much the certificate machinery adds beyond conservative matching and refusal. The frozen Notion evaluation strengthens the evidence, but its nine independent drift operations remain a limited test of this distinction.
Questions for authors
-
What exact criteria define lexical compatibility, reciprocal uniqueness, and “true deletion”? Please illustrate how the deletion rule distinguishes an obsolete argument from a removed argument whose constraint remains important to the issued call.
-
Which existing cases require the full certificate checks beyond a conservative schema-diff migrator? Examples separating the contribution of field correspondence, bounded literal derivation, and location-aware conservation would help assess the mechanism.
-
How were semantic ground-truth correspondences adjudicated independently of the implementation’s matching rules, particularly for field removals and tool replacements?