- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes AgenticDoc, an agent-based framework that enhances document images before passing them to a frozen document parser. The agent selects enhancement tools and parameters, observes intermediate results, and can revisit historical image states before deciding to stop. Training combines online image degradation, parsing-based feedback, and Action-to-Token On-Policy Distillation (A2T-OPD). A2T-OPD converts a search-derived distribution over executable actions into conditional token targets through a semantic action trie and is jointly optimized with a critic-free PPO objective.
The method is evaluated on Wild-OmniDocBench, OmniDocBench v1.6, and PureDocBench. On Wild-OmniDocBench, the authors report an average improvement of 7.15 points across 13 parsers, including 7.44 points across nine parsers not used for training feedback, while largely maintaining average performance on clean OmniDocBench inputs.
Critical strengths
-
Consistent transfer across downstream parsers. The strongest empirical result is that AgenticDoc improves all nine parsers not used for training feedback on Wild-OmniDocBench. These include different parsing architectures and general-purpose VLMs. This breadth provides meaningful evidence that the learned enhancement policy transfers beyond the specific models supplying its training signal.
-
A practically useful formulation of document enhancement. Optimizing an input-side policy against downstream parsing quality allows existing parsers to benefit without updating their parameters. Evaluating both degraded and clean documents is also valuable, since unnecessary enhancement can damage otherwise readable inputs. Together, these aspects make the framework relevant to document-processing pipelines facing variable capture conditions.
Critical weaknesses
-
The novelty and contribution of the proposed components are not sufficiently established. The related-work discussion omits closely relevant approaches, including RL-Restore (CVPR 2018), which learns sequential restoration-tool selection; AgenticIR (ICLR 2025), which incorporates reflection, rollback, and replanning; and AdaptiveISP (NeurIPS 2024), which optimizes adaptive image-processing pipelines using downstream task performance. These precedents do not negate the paper’s document-specific contributions, but they make a clearer positioning and a relevant adapted baseline important.
The ablations also leave the main contributions insufficiently isolated. Table 3 does not directly evaluate intermediate visual feedback, historical-state revisiting, or adaptive degradation sampling. The action-order analysis in Appendix B does not establish the necessity of online replanning or rollback. In Table 4, adding A2T-OPD introduces search-derived supervision as well as a new distillation objective, while Search-SFT differs in its supervision targets and training-state distribution. Consequently, these comparisons do not isolate the value of the action-to-token formulation. A controlled comparison using identical student-visited states, candidate actions, and search-derived teacher distributions would be more informative. Furthermore, the teacher probability-preservation result should be distinguished from any claim about equivalence between the local training loss and an action-level KL objective.
-
Incomplete methodological details and limited reporting weaken reproducibility and the interpretation of the results. Important details remain unspecified, including the training search procedure and evaluation budget, complete action-parameter discretization, the dewarping model, training-data sources and annotation provenance, and several RL hyperparameters. The critic-free PPO description also lacks an explicit return-computation algorithm, particularly for historical-state revisiting and discarded branches. Additionally, the main text describes filtering for images that both parsers recognize correctly, whereas Appendix D.1 uses agreement between their outputs; agreement alone does not establish correctness.
The reported gains lack uncertainty estimates or repeated-training results, which matters particularly for the 0.86-point improvement over Search-SFT and the 0.68-point improvement over A2T-OPD alone. Reporting primarily overall scores also obscures which parsing components and degradation conditions benefit. Although average enhancement latency is provided, the measurement conditions, end-to-end parsing overhead, and search-related training costs are insufficiently documented. Training on 1,000 source images demonstrates limited source-data requirements, but does not by itself establish computational efficiency relative to parser training on larger datasets.
Questions for authors
-
Search-teacher construction: Could you provide pseudocode specifying candidate actions and parameter values, search expansion and pruning, and the evaluation budget used to construct ? Does the beam width of 3 mentioned in Appendix B also apply to training? How many parser evaluations are required per student-visited state on average?
-
PPO update weights: Could you provide the exact process-reward definition and return-computation algorithm, including stopping, invalid calls, historical-state revisiting, and discarded branches? Is the policy-update weight simply the return divided by a running RMS, without a baseline? Please also specify any discounting and how the running RMS is maintained.
-
Attribution of the A2T-OPD gain: Do you have a comparison that fixes the student-visited states, search results, and soft teacher distributions while changing only the supervision method—for example, comparing trie-based distillation with teacher-probability-weighted sequence cross-entropy over complete actions? Such evidence would help distinguish the contribution of action-to-token distillation from that of search supervision.
-
Evidence for interactive correction: How does the full policy compare with a policy that plans from the initial image only, or one that observes intermediate images but cannot revisit earlier states? What fraction of evaluation trajectories actually revisit historical images, and what benefit does this provide? These results would directly inform my assessment of the paper’s central motivation.
4 comments
Rebuttal [2/2]
Q1 – Search-teacher construction
Thank you for requesting these details. We have added teacher-construction pseudocode to Appendix A.3, specifying the budget-conditioned returns and teacher probabilities.
During training, we query the cache for the current image and remaining budget, computing missing results online. Search uses depth 3 and beam width 3, evaluating up to six first-level candidates and retaining the best three nodes per level. Each retained node has at most five successors because consecutive identical actions are excluded.
A full-budget query therefore evaluates at most 6 + 3×5 + 3×5 = 36 candidate images. Across four feedback parsers, these require at most 144 parser–image evaluations before cache reuse. These are upper bounds, not measured averages.
The six calls are denoising, tone adjustment, dewarping, USM sharpening, 1.5× resizing plus USM, and Laplacian sharpening. Each acts on its search-node image; parameters are in the supplied code.
The teacher uses the best explored parsing-score improvement within budget, minus 0.05 per enhancement call. Any evaluated intermediate image can be an endpoint.
Q2 – PPO update weights
We are happy to clarify the reward and return assignment. A successful call receives the parsing-score difference between its output and selected input image, minus costs. The selected input may be historical. The PPO enhancement cost is 0.0001, invalid-call penalty 0.20, and tool-failure penalty 0.02, separate from the teacher-search cost of 0.05. Immediate parse reward is zero.
Returns are undiscounted, accumulated backward along the final image's dependency path. Discarded-branch actions retain only their own immediate negative feedback; nonnegative off-path rewards and unscorable transitions are excluded from PPO return-based updates.
The update weight is the assigned return divided by running RMS, without baseline or mean subtraction. Let v be the global mean squared nonzero assigned returns, aggregated across workers using squared-return sums and counts. The first nonempty batch initializes m=v; later batches update m ← 0.99m + 0.01v. The weight is R / √max(m, 10⁻⁸), shared by generated tokens in the assistant turn; environment observations are excluded.
Q3 – Attribution of the A2T-OPD gain
We appreciate this suggestion. The key challenge is mapping action-level preferences to the few tokens that determine tool and parameter choices. Full-sequence supervision also optimizes shared syntax and fixed fields that do not distinguish candidate actions, potentially diluting the effective gradient signal for these decisions. Teacher-probability-weighted sequence CE still distributes supervision across the complete call.
Before A2T-OPD, we explored complete-action distillation using forward, reverse, and mixed KL objectives. These preliminary implementations did not yield effective policy learning. This experience motivated the decision-token formulation in Section 3.3 and Figure 3.
A2T-OPD explicitly concentrates alignment where candidate actions diverge: teacher mass is conditioned at each trie branch, student probabilities are normalized over the same candidate tokens, and all candidate tokens at student-visited decision positions receive supervision. Shared and fixed tokens remain in context without direct distillation loss. The proposition establishes teacher-distribution factorization, without claiming that the visited-token loss equals action-level KL.
Table 5 reports a +0.86-point advantage over Search-SFT under the same SFT initialization and optimization-step budget, with consistent advantages across four seeds in W2. A2T-OPD addresses the underlying supervision bottleneck by directing the soft teacher’s guidance to the token decisions that determine the enhancement action.
Q4 – Evidence for interactive correction
Thank you for suggesting a direct recovery test. General Response R2 and Appendix E, Table 10 report the completed comparison. Revisiting occurs on 14/1,350 pages (1.04%), always returning to the original image after the first enhancement.
For each case, we retain the path before revisiting and exhaustively evaluate all legal continuations that process only the latest image within the same remaining budget, including immediate parse. Against the best such result, AgenticDoc achieves 12 wins, 2 ties, and a +2.12pp four-parser mean gain across these 14 cases.
This demonstrates recovery at observed revisiting points. It is a targeted comparison rather than a separately trained initial-image-only or no-revisiting policy ablation.
We hope these clarifications and additional results address your concerns and provide a clearer understanding of our contributions. If any questions or concerns remain, we would appreciate the opportunity to address them and would be happy to respond promptly.
Thank you for the additional experiments and clarifications. The multi-seed results strengthen confidence in the reported gains, and the training procedure is now clearer. However, my main methodological concerns remain.
The path-filtered, RMS-scaled returns retain negative feedback from discarded branches while excluding nonnegative feedback. This heuristic’s relationship to the full interaction objective remains unclear, and its benefits over simpler credit-assignment schemes are not established. My concern is not the absence of a critic itself, but the insufficient justification of these choices.
Likewise, qualitative reports of unsuccessful complete-action distillation do not replace controlled comparisons using the same teacher distributions, student-visited states, and training budget. The results support the combined system’s effectiveness but do not isolate A2T-OPD’s contribution. I therefore maintain my original rating.
Thank you for taking the time to read our earlier response and follow up. We appreciate your questions about credit assignment and distillation, which helped us identify where further evidence would be most useful. We have now completed two controlled comparisons on Wild-OmniDocBench to address these points.
1. PPO credit assignment for discarded branches
To address your question about credit assignment, we compare three schemes on Wild-OmniDocBench using the same SFT initialization, A2T-OPD setup, enhancement budget, and training budget.
- Chronological reward-to-go sums rewards from each step to the end of the interaction. Each reward is the change in the current image’s score minus applicable costs and penalties. If a later revisit makes the net interaction return positive, a harmful first enhancement can receive a positive return even when its output is discarded.
- Final-path returns w/o off-path feedback accumulates returns along the image lineage that produced the final output. Successful enhancements whose outputs are discarded receive zero PPO weight.
- Final-path returns w/ off-path feedback (ours) uses the same final-path rule and gives a discarded enhancement its immediate reward when that reward is negative.
| Credit assignment | PaddleOCR-VL-1.6 | MinerU2.5-Pro | Unlimited-OCR | OvisOCR2 | Avg. | Revisited pages | First-enhancement drops | Enhancements/page |
|---|---|---|---|---|---|---|---|---|
| Chronological reward-to-go | 92.38 | 91.94 | 87.75 | 91.29 | 90.84 | 65 | 108/1,331 (8.11%) | 2.12 |
| Final-path returns w/o off-path feedback | 92.61 | 92.00 | 88.25 | 91.51 | 91.09 | 27 | 52/1,280 (4.06%) | 1.77 |
| Final-path returns w/ off-path feedback (ours) | 92.70 | 92.08 | 88.27 | 91.67 | 91.18 | 14 | 39/1,280 (3.05%) | 1.67 |
We count a first-enhancement drop when the first successful enhancement lowers the average score relative to the original image. The denominator counts pages with a scorable first enhancement.
The chronological baseline assigns credit using the net return of the full interaction. A later revisit can therefore give positive credit to an earlier enhancement whose output was discarded. Assigning recovery gains along the final image lineage improves the mean parsing score from 90.84 to 91.09, reduces the first-enhancement drop rate from 8.11% to 4.06%, and lowers enhancement use from 2.12 to 1.77 calls per page. These results are consistent with crediting the actions that produced the recovered image.
The last two rows show what off-path feedback contributes. The extra feedback gives PPO a local signal to discourage a harmful discarded enhancement even when the interaction ends with a net gain. In this comparison, adding off-path feedback reduces first-enhancement drops from 52 to 39 (−25%), revisited pages from 27 to 14, and average enhancement calls by 0.10, with comparable final parsing scores (91.09 and 91.18). The main benefit we observe is fewer early degradations and fewer enhancement calls at similar final parsing quality.
2. Contribution of action-to-token distillation
To isolate the contribution of action-to-token distillation, we use the original A2T-OPD-only run from Table 5 as the reference. We train the teacher-weighted full-action CE baseline from the same SFT checkpoint by replaying the exact sequence of states visited during that run, including intermediate states. At each state, CE uses the recorded candidate actions, search results, and soft teacher probabilities to weight the cross-entropy over complete action sequences. Both runs use the same training budget, with PPO disabled.
| Supervision method | PaddleOCR-VL-1.6 | MinerU2.5-Pro | Unlimited-OCR | OvisOCR2 | Avg. |
|---|---|---|---|---|---|
| Teacher-weighted full-action CE | 91.82 | 91.42 | 87.22 | 90.62 | 90.27 |
| A2T-OPD | 92.07 | 91.82 | 87.40 | 90.70 | 90.50 |
On Wild-OmniDocBench, A2T-OPD raises the mean parsing score from 90.27 to 90.50 (+0.23 points), with gains on all four parsers (+0.08 to +0.40 points). With training states and teacher targets held fixed, these gains support the contribution of the action-to-token supervision rule.
Candidate calls often share syntax and fixed fields, so many tokens covered by full-action CE carry little information about which action the teacher prefers. At each trie branch, A2T-OPD converts the teacher probability mass of actions compatible with the current prefix into a soft target over valid next tokens. This gives the student a direct comparison between the choices that determine the tool or its parameters, even when the complete calls share many tokens or differ in length.
We hope these comparisons address the questions you raised. Thank you again for taking the time to consider our response. We would be grateful for any further feedback.
Rebuttal [1/2]
Thank you for your careful review and constructive suggestions. We respond in two parts: this part addresses the two weaknesses, and Part 2 answers the four specific questions.
W1 – Novelty and contribution
Thank you for highlighting the need for clearer positioning and component-level evidence. We have expanded Section 2.3 to discuss RL-Restore, AgenticIR, AdaptiveISP, and DocRes. The completed DocRes/AdaptiveISP comparisons, settings, and visual analysis are presented in General Response R3. AgenticDoc exceeds these baselines by 11.46 and 10.42 points, respectively, on the four-parser Wild-OmniDocBench average, while largely preserving the clean-document average.
We have also completed the adaptive-sampling and teacher-search controls:
These are Overall scores (%) on Wild-OmniDocBench. Removing adaptive sampling reduces the average by 0.31 points; replacing the multi-step teacher with a single-step teacher reduces it by 0.62 points. Both changes reduce scores on all four parsers. The sampling control retains online degradation generation, and the single-step teacher retains candidate-action evaluation. The latter therefore tests the added value of considering subsequent enhancement steps.
The A2T-OPD attribution question is discussed in Q3, and the completed revisiting comparison in Q4.
W2 – Reproducibility and reporting
We appreciate your detailed requests. Appendix A.3 now provides teacher-construction pseudocode. During training, cached search results are queried first; missing results are computed online. Multi-level caching reuses intermediate images and parsing results.
The training source collection contains 1,000 images after filtering: 300 each from DocLayNet, PubLayNet, and DocStruct4M, plus 100 web-document images. We use only their images, without the original annotations. Training pseudo-references are obtained from MinerU2.5-Pro and PaddleOCR-VL-1.6, retaining pages with NED < 0.02. Agreement is a filtering proxy; shared errors can remain. These details are clarified in Section 3.2 and Appendix D.1.
Our dewarping tool is based on DewarpNet, adapted and trained by our team. We will publicly release the complete training dataset and code, including the trained dewarping model.
To address the variability of the original +0.86 and +0.68 differences, we completed repeated training with four seeds under the same evaluation protocol:
Scores are four-parser averages on Wild-OmniDocBench. The joint method leads in every run: its advantage ranges from 0.71 to 1.11 points over Search-SFT and 0.23 to 1.23 over A2T-OPD only. These results show that the observed advantages persist across the evaluated training seeds.
Enhancement-action counts exclude
parse; AgenticDoc runtimes include policy inference, explicitparsegeneration when present, tool execution, and final-image output, but exclude downstream parsing.For inference cost, General Response R1 and Section 4.2.2 report enhancement-plus-parsing measurements: an average 0.54 s/image increase on clean OmniDocBench v1.6 and 12.10 s/image reduction on Wild, across the four parsers. Parser measurements use serial processing on a single A100-80GB GPU.
The 1,000-image figure describes training-data requirements. It does not establish lower training compute than large-scale parser training; teacher-search costs are discussed separately in Q1.