- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces InEx, which combines the reconstruction of internal evidence from trajectories with external environmental evidence actively gathered by an evaluator to assess agent tasks that lack cheap online ground truth, and further uses the evaluation results as reward signals for downstream agent training.
Critical strengths
-
The paper focuses on an important and underexplored issue: how to reliably determine if an agent truly completes a task when the task has objective success conditions but the ground truth can't be directly accessed during evaluation. Dividing evidence into intrinsic and extrinsic types is a clear and practically meaningful perspective, especially for explaining failures where the 'answer seems reasonable but the trajectory doesn’t support it.'
-
I like how the paper organizes step-by-step trajectory checking, evidence chain aggregation, and active environment verification into a complete process. The same InEx-I checkpoint is directly evaluated on WebVoyager, WebArena, AndroidWorld, and OSWorld, and it outperforms WebJudge across the board, which shows that the learned trajectory checking capability has some cross-environment transfer value, not just specific engineering optimizations for one site.
-
The experiments are fairly comprehensive. Besides 4,000 trajectories across the four environments, the paper also provides component ablations, handoff ablations, 3B model experiments, reward error type analysis, and downstream PAE training. The analysis of success precision and failure recall, in particular, is more relevant to the real risk when the evaluator is used as a reward model than simply reporting accuracy.
Critical weaknesses
-
The core training process is underdisclosed, making current results difficult to reproduce and interpret. The appendix does not report SFT/RL data scale, task composition, ratio of real vs. synthetic samples, specific RL algorithms, and key hyperparameters; The Value function and task distribution factor (rho_i) in formula (12) are also not clearly defined. Generating SFT traces, providing RL rewards, and implementing the "frontier model" for InEx-E do not provide specific models or versions. Given that SFT and RL bring about 13.6 and 4.9 percentage point improvements respectively, these deficiencies directly affect primary performance sources rather than secondary implementation details.
-
There is a clear capability and computational budget asymmetry between full InEx and the main table baseline. InEx uses task-specific training with 7B evaluator, frontier model, and up to 20 additional steps of environmental interaction, while Claude, GPT-5, Qwen3-VL, and WebJudge in Table 1 are mainly passive trajectory evaluators. The paper discusses interactive evaluation methods such as Agent-as-a-Judge and ProRe in related work, but does not compare them under the same environmental interface and interaction budget. Table 4's "Frontier I w/o scaffold" is a helpful control experiment but only reported on WebVoyager, still insufficient to support strong SOTA conclusions for the full method in four environments.
Questions for authors
See the weaknesses above.
6 comments
Response to Reviewer dDPi (2/3)
W2a. Does the Table 1 comparison account for full InEx's extra capability and compute budget?
Giving GPT-5.0 the environment actions and step budget of InEx-E does not close the gap. This frontier I+E baseline has the same frontier model, actions and budget as InEx-E but no trained scaffold. It reaches 86.6 on the macro average, 5.2 points below InEx, although it uses 184.2K API tokens per WebVoyager trajectory and InEx 32.2K. The table lists what each evaluator uses, its accuracy and its cost per trajectory on WebVoyager:
| Evaluator | Local model | Frontier model | Env. actions | WebVoyager acc. (%) | Macro acc. (%) | Latency (s) | Total tokens (K) | API tokens (K) |
|---|---|---|---|---|---|---|---|---|
| GPT-5.0 judge (Table 1) | – | GPT-5.0 | No | 81.3±0.7 | 81.5 | 11.5 | 13.8 | 13.8 |
| WebJudge (Table 1) | Released 7B | GPT-5.0 | No | 86.7±0.6 | 86.3 | 15.3 | 63.4 | 18.3 |
| Frontier I+E w/o scaffold (Table 4) | – | GPT-5.0 | Same as InEx-E | 86.8±1.4 | 86.6 | 41.3 | 184.2 | 184.2 |
| InEx-I (Table 1) | Trained InEx-I (7B) | – | No | 90.1±0.3 | 89.5 | 18.6 | 71.3 | 0 |
| InEx (Table 1) | Trained InEx-I (7B) | GPT-5.0 | Yes | 91.4±0.3 | 91.8 | 35.4 | 103.5 | 32.2 |
This baseline is the most expensive evaluator we measured. On the macro average it is also below InEx-I alone, which uses neither a frontier model nor the environment at inference. W2b and W2c below report ProRe under the same interface and this baseline in all four environments. Costs are in Appendix F.4 (Table 9); the baseline's cost and four-environment results are in Appendix F.12 (Table 17), marked in blue.
W2b. How does full InEx compare with interactive evaluators such as Agent-as-a-Judge and ProRe under the same environment interface and interaction budget?
We ran ProRe (Dai et al., 2026), which probes the environment state, with its released code and default settings on AndroidWorld, the only environment its released code targets. As shown in the table below, under the same environment interface, the same frontier model and step caps of the same order, InEx is 8.5 points more accurate than ProRe with its default probing agent, and InEx-I alone, without any environment action, is above both ProRe runs. Agent-as-a-Judge cannot be run as published. It judges code-generation agents against a hand-annotated requirement hierarchy per task, with tools that read the agent's workspace. Our tasks give one instruction and no requirement list, with evidence in a browser, emulator or desktop. W2c reports a generic agentic judge in its place, the frontier I+E baseline. The interactive GUI evaluators of Cui et al. (2026) and Shi et al. (2026), cited in Section 2, are concurrent work; we will add a comparison with them.
Specifically, GPT-5.0, the model of InEx-E, replaces ProRe's retired default reasoner, Gemini-2.5-Pro. Both judge the same 1,000 trajectories from the same restored start states through the official actions. Accuracy in %, mean ± std over three runs:
| Evaluator | Reasoner / final verdict | Probing agent | Env. step budget | AndroidWorld |
|---|---|---|---|---|
| GPT-5.0 judge (Table 1) | GPT-5.0 | – | 0 | 83.2±0.7 |
| ProRe | GPT-5.0 | V-Droid-8B (default) | ≤ trajectory length (16.6 avg.) | 86.0±0.7 |
| ProRe | GPT-5.0 | M3A (GPT-5.0) | ≤ trajectory length (16.6 avg.) | 83.5±1.4 |
| InEx-I (Table 1) | InEx-I | – | 0 | 92.1±0.3 |
| InEx (Table 1) | GPT-5.0 | InEx-E (GPT-5.0) | 15 (4.3 avg. when acting) | 94.5±0.3 |
ProRe with its default probing agent reaches 86.0%, in line with the 86.2% its paper reports with a GPT-5 reasoner. InEx-I alone reaches 92.1%, so 6.1 of the 8.5 points do not depend on environment interaction. Details are in the new Appendix F.10 (Table 15), and those of the generic agentic judge in Appendix F.12, marked in blue.
Response to Reviewer dDPi (3/3)
W2c. Can Table 4's Frontier I+E w/o scaffold control be reported beyond WebVoyager?
Yes. The control now covers all four environments and stays below InEx in each, by 5.2 points on the macro average. In this control, GPT-5.0 inspects the trajectory, gathers environment evidence and gives the verdict without the InEx-I scaffold. It has the same official actions and interaction budget as InEx-E (up to 20 steps, 15 on AndroidWorld). Unlike standalone InEx-E (Table 8), it first carries out the intrinsic stage itself, prompted to follow InEx-I's workflow. Accuracy in %, mean ± std over three runs on the same 1,000 trajectories:
| Evaluator | Env. actions | WebVoyager | WebArena | AndroidWorld | OSWorld | Macro |
|---|---|---|---|---|---|---|
| GPT-5.0 judge (Table 1) | No | 81.3±0.7 | 84.5±0.9 | 83.2±0.7 | 77.1±1.8 | 81.5 |
| Frontier I+E w/o scaffold | Yes | 86.8±1.4 | 88.6±2.1 | 87.1±0.8 | 83.9±2.1 | 86.6 |
| InEx-I (Table 1) | No | 90.1±0.3 | 91.1±0.7 | 92.1±0.3 | 84.6±1.3 | 89.5 |
| InEx (Table 1) | Yes | 91.4±0.3 | 93.0±0.5 | 94.5±0.3 | 88.2±1.0 | 91.8 |
With environment access, GPT-5.0 beats its direct judge in every environment, but it reaches InEx-I's level only on OSWorld (83.9% against 84.6%). The gain of InEx appears when environment evidence is added on top of the trained scaffold. Details are in the new Appendix F.12 (Table 17), marked in blue.
Response to Reviewer dDPi (1/3)
We sincerely thank the reviewer for recognizing the importance of this underexplored problem and the clarity of the intrinsic/extrinsic perspective, the transfer of a single InEx-I checkpoint across four environments, and the comprehensiveness of our experiments, including the success-precision and failure-recall analysis. We address the questions below.
W1a. What SFT/RL data, RL algorithm and key hyperparameters were used to train InEx-I, and how are Value(·) and ρ_i defined?
The revised Appendix B specifies InEx-I's training (SFT and RL data, RL algorithm, hyperparameters, Value and ρ), summarized in the table. W1b below names the model in each role. We will release the training code and the trained InEx-I weights (Appendix B.5), and the predictions, prompts, aggregation code and train/evaluation exclusion list as specified in Appendices E.5 and F.4.
Value and ρ (Eq. 12 and B.4), for rollout j of trajectory i on website k_i: Value(R_ij, c_ij, ρ_i) = (1 + λR_ij) · c_ij · ρ_i, ρ_i = clip(p_target(k_i) / p_sample(k_i), 0.5, 2), p_target(k) ∝ 1 / a_k, where a_k is the SFT checkpoint's accuracy on website k, measured on SFT data only, and p_sample(k) is its share of the accumulated batch. Weights are divided by their batch mean and held constant in the gradient. The confidence c is 1 for checker labels and the judge's confidence otherwise. The factor ρ favors websites where the SFT checkpoint is weakest, within fixed bounds.
The verified traces teach the workflow, and the synthetic negatives show plausible but unsupported trajectories. Uniform-weight RL raises WebVoyager accuracy from 85.2 after SFT to 88.5. The Value weighting brings it to 90.1. The weighted model is ahead in every environment, although on OSWorld the difference lies within run-to-run variation (Appendix F.8, Table 13). See Appendix B (B.1–B.5), marked in blue.
W1b. Which specific models generate the SFT traces, provide the RL rewards, and serve as the InEx-E frontier model?
The table names the model in each role.
Appendix E.4 names these agents. InEx uses frontier models in two places only. The first is the training of InEx-I, where GPT-4o writes the SFT traces and drives the agent whose trajectories are judged during RL, and GPT-5.0 judges the rewards of 187 seed tasks. The second is InEx-E, which gives the final verdict. At inference, InEx-I runs on open weights. The GPT-5.0 judge and InEx-E use the same model. See the new Appendix F.13 (Table 18), marked in blue.