acceptodds
Under review as a conference paper at ICLR 2027

ACTS: Controlling Evidence Acquisition Before Writing Questions for Search Agents

Abstract

Training search agents for open-world problems requires high-quality long-horizon trajectories that remain scarce and costly. The bottleneck begins before teacher rollout: a synthetic question can appear semantically complex yet collapse to one lookup when its clues reside on a single page, while a larger evidence graph may add redundancy without requiring new retrieval. Existing synthesis methods control end-product properties such as hop count, graph size, or question-level difficulty, but not the acquisition process realized during task construction. This creates a search-demand gap between apparent complexity and realized acquisition. We introduce ACTS (Acquisition-Controlled Task Synthesis), which designs a task’s evidence-acquisition topology before writing its question. From construction-time retrieval, ACTS builds linked entity and document-navigation hypergraphs that record grounded evidence and the queries and documents that made it reachable. Complementary search policies expand a shared state, and a lexicographic max–min objective selects the branch subset that maximizes the weaker of its semantic and acquired-evidence scores, with mean quality and nonredundancy as later tie-breakers. ACTS separately chooses evidence for question writing and the next construction frontier, preserving exploration without exposing every observation to the writer. Generated questions are screened for answer leakage and no-tool shortcuts, then independently solved by a tool-using teacher. In matched construction, ACTS increases mean committed evidence sources from 4.50 to 9.69 while achieving a 70.17% teacher solve rate. Under anti-contamination evaluation, ACTS-trained models reach 67.61% on full BrowseComp and 52.13% on HLE-Text with SFT alone, demonstrating that acquisition-controlled synthesis produces effective supervision across diverse search settings.

Primary area:
datasets and benchmarks
Keywords:
Search agents, synthetic data generation, evidence acquisition, long-horizon search, hypergraphs

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2ozUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ACTS generates search-agent training tasks by controlling evidence acquisition before writing questions. It combines entity and document-navigation hypergraphs, branch selection, and separate decisions for retaining evidence and continuing exploration. Solver trajectories then train student models. Experiments evaluate downstream gains, construction ablations, and comparisons with QUEST.

Critical strengths
  • Consistent downstream gains. All three students improve across four benchmarks. For example, Qwen3.8-27B improves from 49.05% to 67.61% on BrowseComp (Table 1, p.8), supporting the practical value of the data recipe.
  • Concrete construction controls. The dual hypergraphs, max–min selection over semantic and navigation evidence, and separation of retention from exploration address distinct construction choices, with six ablations (§§3–4). Beyond existing browse-then-write methods such as QUEST, these controls are the main methodological contribution.
Critical weaknesses
  • Construction paths may not be necessary for solving the tasks. Sources counts graph URLs supplied to the writer, rather than sources visited by the solver. Node recall of 59.95% and sequence LCS recall of 41.52% show path overlap but do not rule out shorter retrieval routes (Appendix B.5, p.17). This limits the claim that construction controls determine the acquisition difficulty faced by solvers.
  • Budget differences and small samples limit attribution. ACTS and QUEST use similar trajectory counts, but total tool calls differ by about 1.79×, and matched token budgets for supervision are not established. On BC-130, the selector answers only four and three more questions correctly than random and entity-only selection (§4.4; Table 2). Data-efficiency and selector gains therefore remain uncertain.
Questions for authors
  • Acquisition difficulty: How do sources visited in existing solver trajectories align with construction paths? If needed, a small test with a short retrieval budget and the same solver could assess path necessity; alternatively, please narrow the difficulty-control claim.
  • Budgets and uncertainty: Please report supervised tokens, update counts, and construction budgets for ACTS and QUEST. If they differ materially, a small comparison with equal supervision would help. For the selector, please report paired uncertainty using existing BC-130 outcomes.
5 comments

Response to Q1

Acquisition difficulty: How do sources visited in existing solver trajectories align with construction paths? If needed, a small test with a short retrieval budget and the same solver could assess path necessity; alternatively, please narrow the difficulty-control claim.

Thank you. We would like to clarify how the sources visited by the teacher align with the construction paths.

For each question, we compare the URLs opened in the teacher's trajectory with the source URLs recorded during that question's construction, excluding the answer's sources. Among the 6,741 trajectories with recorded non-answer sources, 66.37% open at least one of these URLs. Pairing the same trajectories with other questions' construction records reduces this rate to 0.24% and 0.31%. These results show that independent teacher search revisits some of the sources used to construct the same question, and that this overlap is specific to that question.

As suggested, we randomly sampled 100 retained ACTS questions that had passed GLM-5.1's no-tool screening with thinking disabled. We then asked the original teacher, Kimi-K3 (reasoning effort: max), to solve them again under different retrieval budgets. We generated one new trajectory per question per budget and graded the final answer to assess whether the teacher could answer with fewer retrieval calls.

Retrieval calls count searches and page openings (search + open). Find and local Python remain available in every condition and are also included in total tool calls. The normal budget allows up to 500 total tool calls.

Allowed retrieval calls (search + open)0135Normal budget
Correct / 1005860797083
Mean retrieval calls (search + open)0.000.962.854.5346.40
Mean total tool calls (all four tools)0.062.176.187.1053.38

With reasoning enabled, Kimi-K3 answers 58/100 questions with zero retrieval and 83/100 under the normal budget. The normal budget yields 13 more correct answers than the setting limited to five retrieval calls.

These results show that a stronger reasoning-enabled model can solve many retained questions without retrieval or with only a few retrieval calls. It may already know information that another model needs to retrieve, or find a shorter route to the answer. The amount of retrieval needed and the solution path therefore depend on the solver. We will make this dependence explicit, limit our acquisition-difficulty claims to the tested models and settings, and interpret construction-path recovery as overlap with the tested teacher's search, rather than a shortest or required solution route.


Response to Q2

Budgets and uncertainty: Please report supervised tokens, update counts, and construction budgets for ACTS and QUEST. If they differ materially, a small comparison with equal supervision would help. For the selector, please report paired uncertainty using existing BC-130 outcomes.

We appreciate your suggestions on reporting training and construction budgets and quantifying uncertainty in the selector comparison.

The original comparison already uses comparable supervised-token budgets. Although ACTS has more tool calls, it averages fewer supervised tokens per tool call: approximately 148, compared with 238 for QUEST. On BC-130, ACTS scores 89/130 (68.46%), versus QUEST's 78/130 (60.00%), a gain of 8.46 percentage points.

We additionally compare the two methods under comparable total training-token budgets: 0.990B for ACTS and 1.003B for QUEST. We retain all 4,670 complete ACTS trajectories and shorten training from 198 to 120 optimizer updates, matching QUEST's 120 updates. ACTS scores 84/130 (64.62%), compared with 78/130 (60.00%) for QUEST, retaining a 4.62-percentage-point advantage.

QuantityACTS (Table 3)ACTS (120 updates)QUEST
Training trajectories4,6704,6704,516
Training tokens1.654B0.990B1.003B
Supervised tokens220.88M132.34M200.27M
Optimizer updates198120120
SFT GPU hours (H200)478.00262.46331.80
BC-13089/130 (68.46%)84/130 (64.62%)78/130 (60.00%)

These results show that ACTS outperforms QUEST on BC-130 with both a slightly larger but comparable supervised-token budget and a smaller one. Its advantage also persists under comparable total training-token budgets and the same number of optimizer updates.

All three models start from the same Qwen3.8-27B base and use the same training templates. B denotes billion and M million.

ACTS constructs questions from a fixed, offline KILT database. Its document retrieval and reading are performed locally, with no live-web search or external search API calls. ACTS allows at most 27 expansion steps: three policies, three steps per policy per round, and three rounds. During expansion, ACTS uses Kimi-K3 to extract facts for the evidence graphs. Its construction costs also include model inference for question writing and editing (Kimi-K3). QUEST uses Claude Sonnet 4.5 to collect web evidence and write questions. Its construction costs include model inference for evidence processing and question generation, Serper searches, Jina page retrieval, GPT-5-mini page summarization, and GPT-5 evaluation-script generation. Both trajectory sets use the same Kimi-K3 teacher (reasoning effort: max), tools, and maximum of 500 tool calls.

For the selector, we computed paired 95% confidence intervals from the existing Table 2 outcomes on the same 130 questions. Full ACTS answers 59 correctly, random selection 55, and entity-only selection 56.

ComparisonAccuracy differencePaired 95% confidence interval
Full ACTS vs. random+3.08 points[-6.15, 12.31] points
Full ACTS vs. entity-only+2.31 points[-6.92, 11.54] points

We resample questions while keeping each question's results from the two models together. Both intervals include zero. We will report them with Table 2 so that the four- and three-question differences are presented with their uncertainty.


We hope these responses and the new experiments address the reviewer's concerns, and we would be grateful if the reviewer would consider further strengthening the support for our paper. We are happy to answer any further questions.

Thank the authors for the additional analyses. I maintain my rating of 3 (weak acceptance), with stronger support for the complete data recipe. The remaining uncertainty concerns component-level attribution and the strength of the difficulty-control claim.

Reviewer_H1UvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ACTS synthesizes search-agent training tasks by organizing evidence acquisition before question writing. It combines linked entity and document-navigation hypergraphs, multi-policy evidence selection, and independent continuation-frontier selection. Generated questions are filtered and solved by a tool-using teacher to produce SFT trajectories. Experiments across three student backbones and four benchmarks report substantial improvements, supported by construction ablations and a comparison with QUEST.

Critical strengths
  1. The method addresses a meaningful training-data problem. ACTS explicitly records both grounded facts and their acquisition history, and separates evidence retention from continued exploration. These design choices provide a concrete approach to constructing search-oriented supervision beyond static measures such as graph size or hop count.
  2. The experiments demonstrate useful downstream transfer within the paper evaluation setting. Table 1 reports improvements over the corresponding ReAct baselines across three backbones and four benchmarks.
Critical weaknesses
  1. The strength of the starting baseline needs further clarification. The Qwen3.5-35B-A3B model card reports 61.0% on BrowseComp [1], whereas the paper reports 40.92% for its ReAct baseline and 61.45% for ACTS under an anti-contamination protocol. These results are not directly comparable, and the discrepancy does not invalidate the within-protocol improvement. However, the paper does not establish whether similarly large gains persist with a stronger inference configuration. Clarifying the effects of context management, inference settings, and contamination controls, and evaluating both the baseline and ACTS under the same stronger configuration and tool budget, would better establish the scope of the gains. 2.The incremental benefit over existing synthesis recipes remains insufficiently isolated. Prior work already controls graph structure and evidence dispersion (REDSearcher), addresses search shortcuts (FORT-Searcher), or filters for longer trajectories (OpenSeeker-v2). ACTS introduces distinct acquisition-control mechanisms, and its ablations and QUEST comparison provide useful evidence. However, the approximately trajectory-count-matched QUEST comparison contains 1.79× as many total training tool calls for ACTS. A comparison with a simpler graph-based synthesis and trajectory-filtering baseline under comparable supervision budgets would better establish the added value of these mechanisms. 3.Evidence for necessary search is weaker than evidence for elicited search behavior. Construction-path recovery shows that teacher search revisits relevant entities, but does not establish that those retrieval steps are necessary. Moreover, “Sources” measures the writer’s committed graph, and the no-tool solver runs with thinking disabled (Appendices B.1 and C.2). These measurements support the behavioral findings, but leave open whether stronger reasoning or shorter retrieval routes could solve the same tasks.

[1] https://huggingface.co/Qwen/Qwen3.5-35B-A3B

Questions for authors
  1. What explains the BrowseComp gap between the paper’s Qwen3.5 baseline and the official result? Do ACTS gains persist with a stronger inference configuration under the same contamination controls?
  2. Does ACTS outperform a simpler synthesis-and-filtering baseline when training tokens or other relevant supervision budgets are comparable? Please distinguish tool-call counts from actual training compute.
  3. How robust are the search-demand conclusions to reasoning-enabled no-tool solvers or limited-search solvers? Also, at which filtering stage is Table 2’s tool-required rate measured?
3 comments

Response to Q1

What explains the BrowseComp gap between the paper’s Qwen3.5 baseline and the official result? Do ACTS gains persist with a stronger inference configuration under the same contamination controls?

Thank you for this question. The Qwen model card reports 61.0% and describes a context-folding setup for Search Agent evaluation [1], whereas our original ReAct baseline and ACTS-trained model were both evaluated without context management; the scores therefore come from different inference setups.

To test whether ACTS’s gains persist under a stronger inference configuration, we followed the 64k discard-all context-management policy described in the GLM technical report [2] and increased the tool-call limit to 1,200 for both ReAct and ACTS. We kept the contamination controls identical across the two models.

Evaluation settingQwen3.5-35B-A3BQwen3.5-35B-A3B trained on ACTSImprovement
Without context management40.92%61.45%+20.53 points
With context management52.69%74.88%+22.19 points

The stronger inference configuration further improves the performance of the Qwen3.5-35B-A3B model trained on ACTS and widens its advantage over Qwen3.5-35B-A3B from 20.53 to 22.19 percentage points.

[1] https://huggingface.co/Qwen/Qwen3.5-35B-A3B

[2] https://arxiv.org/abs/2602.15763


Response to Q2

Does ACTS outperform a simpler synthesis-and-filtering baseline when training tokens or other relevant supervision budgets are comparable? Please distinguish tool-call counts from actual training compute.

We appreciate your question about supervision budgets and the simpler synthesis baseline. We report the additional training-budget comparison and distinguish tool calls from training tokens.

As suggested, we added an SFT comparison under comparable training budgets. Starting from the same Qwen3.8-27B backbone, both methods use 120 optimizer updates and comparable cumulative training-token budgets. ACTS achieves 84/130 (64.62%) on BC-130, compared with 78/130 (60.00%) for QUEST.

QuantityACTS, 120 updatesQUEST
Training trajectories4,6704,516
Training tokens0.990B1.003B
Optimizer updates120120
BC-13084/130 (64.62%)78/130 (60.00%)

With comparable cumulative training-token budgets, ACTS still outperforms QUEST by 4.62 percentage points on BC-130.

Tool-call counts record how often the teacher uses tools during trajectory generation. During SFT, the student processes the full trajectories, including questions, reasoning, tool calls, tool responses, and final answers. A tool response may contain a short snippet or a long page, so each call does not correspond to a fixed number of training tokens. We therefore compare training budgets using the cumulative training tokens processed across all SFT updates. B denotes billion.

Table 2 in the main paper already includes a simple graph-based synthesis and filtering baseline: an ordinary graph and a single SA-RW policy, followed by the same QA filtering, selection of correct teacher trajectories, and SFT procedure. We also report their tool-call counts and cumulative training tokens below. Both groups use six epochs and 42 optimizer updates.

QuantityFull ACTSOrdinary-graph SA-RW
Training trajectories2,0002,000
Total tool calls41,29638,926
Training tokens0.325B0.312B
Training epochs66
Optimizer updates4242
BC-13059/130 (45.38%)47/130 (36.15%)

With comparable tool-call counts and cumulative training token budgets, Full ACTS outperforms this simple baseline by 9.23 percentage points on BC-130.


Response to Q3

How robust are the search-demand conclusions to reasoning-enabled no-tool solvers or limited-search solvers? Also, at which filtering stage is Table 2’s tool-required rate measured?

Thank you for suggesting these tests and asking us to clarify the filtering stage.

We randomly sampled 100 retained ACTS questions that had passed GLM-5.1's no-tool screening with thinking disabled. We then asked the original teacher, Kimi-K3 (reasoning effort: max), to solve them again under different retrieval budgets. We generated one new trajectory per question per budget and graded the final answer to assess whether the teacher could answer with fewer retrieval calls.

Retrieval calls count search and page opening. Total tool calls also include find and local Python, which remained available in every condition.

Allowed retrieval calls (search + open)0135Normal budget
Correct / 1005860797083
Mean retrieval calls0.000.962.854.5346.40
Mean total tool calls0.062.176.187.1053.38

Retrieval raises accuracy from 58% to 83%, a 25-percentage-point gain, even with reasoning enabled. The stronger Kimi-K3 can use a smaller retrieval budget to solve questions that GLM-5.1 could not answer directly. Our search-demand conclusions therefore concern the tested solver and settings. Construction-path recovery shows overlap with teacher search, rather than establishing that the construction path is the shortest or a necessary route to the answer.

Across the 11,651 retained trajectories, Kimi-K3 averages 48.44 tool calls. Importantly, these questions have already passed GLM-5.1's no-tool screening, yet they continue to elicit substantial evidence acquisition when independently solved by a stronger teacher without access to the construction graph or path. This provides direct empirical evidence supporting the robustness of our search-demand conclusions across these two stages: the questions resist direct answering by the screening model and still induce extensive search behavior in the stronger teacher.

During construction, GLM-5.1 (thinking disabled, no tools) screens out questions it can answer directly. Table 2's Tool-required rate is measured at this pre-rollout screening stage, before final trajectory selection, as

Tool-required Rate=NunsolvedNsolved+Nunsolved.\text{Tool-required Rate}=\frac{N_{\text{unsolved}}}{N_{\text{solved}}+N_{\text{unsolved}}}.

Correctly answered questions are included in the denominator before being filtered out. This metric measures shortcut resistance relative to the specified screening model, not universal search necessity.


We hope these responses and the new experiments address the reviewer's concerns, and we would be grateful if the reviewer would consider raising the score. We are happy to answer any further questions.

AuthorsOpenReview ↗

Supplementary Evidence for Q3: Retrieval-Budget Dependence

Response to Q3 (Supplement)

Building on our earlier zero-retrieval analysis, we conducted an additional controlled comparison to test whether ACTS-generated questions benefit more from additional search than ordinary-graph SA-RW questions.

We evaluated 200 questions per method using the same reasoning-enabled Kimi-K3 solver (reasoning effort: max), with 179 matched starting document–entity strata and equal question counts per stratum. Questions were sampled after the same no-tool screening but before teacher-correctness filtering. All evaluation settings were held fixed. We compared a limit of five search/open calls with standard inference (up to 500 total tool calls); find and offline Python remained available.

Construction methodFive search/open callsStandard inferenceGain from additional retrieval
Full ACTS123/200 (61.50%)144/200 (72.00%)+10.50 pp
Ordinary-graph SA-RW156/200 (78.00%)163/200 (81.50%)+3.50 pp

ACTS gains 7.00 percentage points more from additional retrieval than ordinary-graph SA-RW (95% interval: 0.49–13.78 pp). This directly demonstrates stronger retrieval-budget dependence for ACTS-generated questions under the same solver and matched starting strata, further supporting our search-demand claim.

We will incorporate this result into the revised manuscript. We hope this additional evidence addresses your concern and would be grateful if you would reconsider your rating.

Thank you again for your time and consideration.

Reviewer_dJFfOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to build a better SFT dataset for search agents. Some questions look complex, but one simple search can answer them. ACTS starts from entities and documents. It uses breadth-first search, depth-first search, and walks without revisiting nodes. These collect facts and the paths used to find them. It checks all seven combinations of the three search policies. It first improves the weaker of the semantic and evidence scores. Then it considers the average score and repeated information. It chooses writing evidence and the next search frontier separately. The questions pass checks for answer leakage, repetition, and no-tool shortcuts. A teacher then solves them by its own search without seeing the construction graph. Traces with answers judged correct and valid tool use are used for SFT training.

Critical strengths

This paper shows that focusing on how the evidence is found, rather than just counting how many facts a question contains, which is useful for SFT data construction. The teacher also solves independently, providing another check after writing the question. The full system has strong results across models and tasks. The matched construction comparison and ablations are also helpful.

Critical weaknesses

(1) Table 2 has six ablations. But I did not find one where the next search can only start from branches kept for question writing. The 36.75% number shows how often other branches are used, not how much they help. So it is still unclear how much this RCD design improves question quality or downstream performance.

(2) Table 3 uses similar numbers of training traces: 4,670 for ACTS and 4,516 for QUEST. But these traces contain 243,613 versus 135,898 tool calls, about 1.79 times as many for ACTS. Both use six training epochs, but this does not ensure the same total number of training tokens. Richer traces may be a benefit of ACTS. So under the same token budget, we cannot tell whether ACTS performs better.

Questions for authors

(1) Can you compare full RCD with a version that only continues from branches kept for question writing? Please keep the evidence-selection rule, starting documents and entities, and maximum search budget the same. How do question quality and downstream results change?

(2) Can you compare ACTS and QUEST with the same total number of tokens processed during SFT, counting all epochs? Please also report GPU hours on the same hardware. Does ACTS still perform better?

1 comment

Response to Q1

Can you compare full RCD with a version that only continues from branches kept for question writing? Please keep the evidence-selection rule, starting documents and entities, and maximum search budget the same. How do question quality and downstream results change?

We appreciate this specific suggestion for testing RCD. We have added the requested ablation and report both question quality and downstream performance.

Full ACTS performs slightly better on BC-130 than the variant that only continues from branches retained for question writing.

ArmTool-required (%)Teacher solve (%)CallsSourcesTW ≥4 (%)BC-130 (%)
Full ACTS (Table 2)33.2870.1731.609.6957.0145.38
Without RCD35.8467.8431.879.3045.7243.85

This ablation uses the same starting document–entity pairs and maximum construction search budget as the existing ablations in Table 2. We keep MPF’s evidence-selection rule unchanged from full ACTS and only restrict next-round search to the retained branches.

With RCD, ACTS achieves a higher teacher solve rate, a larger proportion of graphs with TW ≥4, and better BC-130 performance. By choosing continuation independently of evidence retention, RCD keeps promising search branches available for later evidence acquisition, expanding the evidence available for question writing.

We will add these results to Table 2 alongside the existing construction ablations.


Response to Q2

Can you compare ACTS and QUEST with the same total number of tokens processed during SFT, counting all epochs? Please also report GPU hours on the same hardware. Does ACTS still perform better?

Thank you for requesting this comparison. We report cumulative training tokens and GPU hours for both methods.

ACTS still outperforms QUEST under comparable cumulative training-token budgets.

QuantityACTS, 120 updatesQUEST
Training trajectories4,6704,516
Training tokens0.990B1.003B
Optimizer updates120120
SFT GPU hours (GPU count × training hours)262.46331.80
BC-13084/130 (64.62%)78/130 (60.00%)

Both runs start from the same Qwen3.8-27B base, use the same training templates, and run on H200 GPUs. For ACTS, we retain all complete training trajectories and shorten training to match QUEST’s update count.

The added comparison shows that ACTS answers six more BC-130 questions correctly under a comparable cumulative training-token budget.


We hope these responses and the new experiments address the reviewer's concerns, and we would be grateful if the reviewer would consider further strengthening the support for our paper. We are happy to answer any further questions.

Comments on the paper

1 comment

Dear AC and Reviewers,

We sincerely thank the ACs, SACs, PCs, and reviewers for their time, careful assessment, and constructive feedback. The discussion has strengthened the evidence for ACTS's central contribution: controlling evidence acquisition before question writing produces effective supervision for search agents. The original submission demonstrated substantial downstream gains, while our rebuttal adds controlled comparisons and direct analyses that strengthen the empirical support for the proposed method.

We summarize the four key findings and the reviewer concerns they address below.

Key evidenceMain resultsReviewer(s)
1. Substantial gains under both standard and stronger inferenceOn full BrowseComp, ACTS improves Qwen3.5-35B-A3B from 40.92% to 61.45% (+20.53 pp). Under matched expanded inference settings, the gain remains substantial: 52.69% → 74.88% (+22.19 pp).H1Uv
2. Advantages under controlled construction and training budgetsCompared with ordinary-graph SA-RW, ACTS achieves 45.38% vs. 36.15% (+9.23 pp) with 2,000 judged-correct trajectories per arm. Against QUEST, ACTS achieves 64.62% vs. 60.00% (+4.62 pp) with 120 optimizer updates and approximately 0.99B cumulative input tokens.H1Uv, dJFf
3. Direct evidence of how MPF and RCD improve evidence acquisitionWith identical candidate pools and retention sizes, MPF retains 19.04% more newly acquired evidence spans. For RCD continuations, 94.76% of newly discovered spans reach the final writer input. Independent teacher searches also open recorded construction-source URLs in 66.37% of eligible traces, versus less than 0.4% under mismatched controls.2ozU, dJFf
4. ACTS-generated questions benefit more from additional searchWe evaluated 200 questions per method from 179 matched starting strata with the same Kimi-K3 solver. When search/open calls increase from a limit of five to the standard setting, ACTS accuracy rises from 61.50% to 72.00% (+10.50 pp), versus 78.00% to 81.50% (+3.50 pp) for ordinary-graph SA-RW. ACTS gains 7.00 pp more from additional retrieval (95% interval: 0.49–13.78 pp).2ozU, H1Uv

Three conclusions emerge from the combined evidence:

  1. Effectiveness: ACTS delivers substantial downstream gains that persist under stronger inference and comparable training budgets.
  2. Mechanism: Controlled MPF and RCD analyses show how evidence is selected, discovered, and preserved during task construction.
  3. Search difficulty: The five-call retrieval intervention demonstrates that ACTS-generated questions benefit significantly more from additional retrieval than ordinary-graph questions under the same solver and matched starting strata.

These results substantially strengthen the empirical case for acquisition-controlled task synthesis and directly address the principal concerns raised during review. We will incorporate the new experiments and clarifications into the revised manuscript.


We respectfully ask the AC to consider the complete body of evidence in the final assessment and believe it provides a strong basis for acceptance.

Best regards,
The authors of Submission #18083

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.