- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 3
Read review
Paper summary
OmniPlex-RL addresses context-window saturation in native multimodal agents by dividing trajectories into segments connected through hypergraph memory. The memory stores evidence notes and tool-call addresses that allow source media to be reopened. Training uses one rollout per prompt, with segment-specific contexts and a per-environment moving reward baseline. The method is evaluated on twelve benchmarks covering text, images, documents, and audio-video inputs.
Critical strengths
- The analysis of context-window saturation is valuable. It identifies how native multimodal observations can exhaust context capacity before the tool-call budget is reached.
- The advantage design for single-rollout RL is sensible. Maintaining moving reward statistics separately for each environment provides a practical reference for advantage estimation without requiring multiple rollouts per prompt.
Critical weaknesses
-
The paper uses token-level F1 as both the training reward and evaluation metric. However, lexical overlap does not reliably measure answer correctness and may reward factually incorrect answers. The authors should additionally report answer-level accuracy and specify how correctness is determined for open-ended questions.
-
The benefits of single-rollout RL are insufficiently validated. A controlled comparison with GRPO or another established RL baseline is needed under the same agent framework and comparable training budgets.
-
The presentation requires substantial revision. Despite extensive formalization, the notation and equations often obscure the underlying mechanisms, while the accompanying prose is difficult to follow. Clearer explanations and concrete examples would improve readability.
Questions for authors
-
The “no hypergraph memory” ablation removes the entire read-back mechanism. It demonstrates the utility of memory retrieval, but does not isolate the contribution of the hypergraph structure. A comparison with standard vector retrieval, using identical notes, source addresses, and context budgets, is needed to establish the benefits of entity associations and candidate expansion.
-
The quality of note generation and graph construction remains insufficiently examined. Do the notes introduce unsupported claims or omit important qualifications? The authors should assess these errors and analyze their effects on subsequent retrieval and final answers.
-
Although notes and hand-over summaries are generated by the same policy model, the system introduces additional generation calls, media reprocessing, embedding, and retrieval operations. Reporting only environment-call counts is insufficient to characterize the full cost. End-to-end task latency should also be evaluated.
I will consider adjusting my ratings if my concerns are well responded.
6 comments
Response to Reviewer 28rV (Part 2/3)
W2: "A controlled comparison with GRPO ... under the same agent framework and comparable training budgets."
Thank you very much. We ran the controlled comparison you asked for (§4.9; Figure 9a–b; Table 12, left). In the same loop and with the same rollouts per step, GRPO loses its learning signal and reaches 21.0 Avg. All at step 64, against 27.5 for OmniPlex-RL.
- Same framework and budget: GRPO trains in our loop with our reward (Eq. (7)), LoRA, KL term and per-segment samples. It changes only the baseline, from our per-modality moving batch baseline to a group baseline, and draws 2 prompts 4 rollouts = 8 rollouts per step, the same number as our 8 prompts 1 rollout:
| same loop, 8 rollouts per step | steps with all-zero advantages | share in steps 1–16 / 17–32 / 33–48 / 49–64 (%) | calls per rollout in training | F1: Avg. Text / Avg. MM | F1: Avg. All (GRPO at step 64) |
|---|---|---|---|---|---|
| GRPO () | 82.8% (53/64) | 56.25 / 81.25 / 93.75 / 100.0 | 1.52 → 0.00 | 10.6 / 6.7 | 21.0 |
| OmniPlex-RL | 0% | 0 / 0 / 0 / 0 | keeps looking (Fig. 6c) | 16.7 / 21.0 | 27.5 |
- Why GRPO falls behind: a group baseline compares a rollout only with the other rollouts of its prompt, so when all four score alike every advantage is zero. Such silent steps become more frequent block by block, and from step 46 on every GRPO step had zero advantage (App. F.6). Its policy also stopped looking, and Avg. MM falls to 6.7, against 21.0 for OmniPlex-RL. We stop GRPO at step 64 because from step 46 on it had no policy-gradient signal, so further steps would apply only the KL term.
- Why ours keeps the signal: our baseline compares every rollout with the running mean of its own environment kind (text, knowledge base, document or stream), so a batch that scores alike still moves the policy and no step is silent (Proposition 3; Figure 6a). In addition, each step covers 4$\times$ as many prompts (8 vs. 2).
- The trained baselines show the same pattern: under their own group baselines, four of the five have 15.8–68.4% of optimizer steps without signal (Table 11), and OmniPlex-RL has none.
- Thank you for suggesting this control. As a result, with the same 8 rollouts per step and only the baseline changed, the gap of 21.0 against 27.5 comes from the baseline.
W3: "The notation and equations often obscure the underlying mechanisms ... concrete examples would improve readability."
Thank you very much. Following your suggestion, we reworked the presentation, so the method can now be followed through redrawn figures and concrete examples before the equations.
- Figures: Figures 1–3 are now vector graphics in one 7 pt font. Figure 3 follows §3.1–§3.3 panel by panel: (a) the native agent and why its window fills, (b) segments, evidence notes, hand-over and read-back, and (c) the single-rollout update. Its caption now says in words how the next segment opens. Data Figures 4–9 use one palette, with legends inside the panels of Figures 6 and 9.
- Examples next to the formulas: Table 1 pairs each carrier formula with one plain sentence. Figure 11 (App. A.2) prints the actual read-back a successor received on MMLB-Doc question 502, each line ending in its re-open call. Moreover, Algorithm 1 (App. C) walks through the three phases in words, and Table 6 traces one question across all nine columns.
- Thank you for this suggestion; it made the method easier to follow. As a result, a reader can see what a segment, a note and a read-back look like before reading the corresponding equation.
Q1: "The 'no hypergraph memory' ablation removes the entire read-back ... A comparison with standard vector retrieval, using identical notes, source addresses, and context budgets."
Thank you very much. The existing ablations isolate the addressed evidence unit and the read-back; the paper has no matched comparison of the hypergraph against a flat index over the same notes and addresses.
- What the ablations isolate (Table 3; §4.5; App. F.3): keeping the same notes but dropping their addresses (w/o coordinates) costs 1.17 Avg. All and 4.0 F1 on folded questions, and dropping the whole read-back ("w/o hypergraph memory") costs 2.54 and 9.8.
- Role of the hypergraph: ① one hyperedge binds the note's entities, the media nodes and the address of one observation (Eq. (5)), "so a picture, its caption and a spoken line stay one fact" (§3.2); ② every read-back line ends in the call that produced it, which keeps provenance traceable (§4.5); ③ re-issuing a full address restores the raw media exactly, which allows one-call re-opening (Proposition 2).
- Thank you for this precise question. Therefore, the claim our evidence supports concerns addressed, re-openable evidence units and their read-back, and the hypergraph is how we bind and address each observation.
Response to Reviewer 28rV (Part 3/3)
Q2(a): "Do the notes introduce unsupported claims or omit important qualifications? ..."
Thank you very much. The design limits what a note may claim and keeps its source one call away, so the successor never has to rely on a note alone.
- Unsupported claims: the note prompt forbids verdicts and option names, and
keepdrops any sentence that states a verdict or restates an option (Eq. (4); Figure 11). A note therefore records what is shown. - Omitted qualifications: every read-back line ends in the call that produced it, so one call re-opens the original page, frames or entry (App. F.3).
- Thank you for raising this point. Hence, a note cannot carry a verdict, and whatever it leaves out stays one call away in the source.
Q2(b): "... effects on subsequent retrieval and final answers."
Thank you very much. Both effects are measured (§4.5, Figure 5; App. F.3; repeated folds in the new App. F.6), and the gold answer reaches the successor mostly through addressed evidence.
- On final answers, removing only the addresses costs 4.0 F1 on folded questions, and removing the read-back costs 9.8.
- On retrieval, the gold answer reaches the successor after 29.6% of folds with the memory vs. 9.7% without (documents: 40.6% vs. 13.3%), through the addressed read-back in 26.3% of folds and through the free-form hand-over in 8.9%.
- Under repeated folds, 6.1% of 512 episodes fold twice, and gold is kept after 30.5% of first and 34.8% of second folds, with no loss from the first fold to the second.
- Thank you for this question. As a result, the effects of the notes on subsequent retrieval and on final answers are quantified, and the addressed read-back carries them.
Q3(a): "additional generation calls, media reprocessing, embedding, and retrieval ..."
Thank you very much. We now log every LLM request, its input tokens by purpose, and wall-clock time (§4.9; Figure 9d; Table 12, right; App. F.6), and compaction is a small share of the total. Per episode:
| per episode | OmniPlex-RL |
|---|---|
| environment calls | 2.68 |
| LLM requests: all / compaction (notes + hand-over) | 4.07 / 0.33 |
| input tokens: policy / compaction / evaluation-time rules | 5,597 / 858 / 2,572 |
| compaction share of the 9,028 input tokens | 9.5% |
| LLM time: all / compaction | 4.8 s / 0.66 s |
| end-to-end wall-clock | 6.4 s |
- Media reprocessing: re-reading media for the notes (Eq. (4)) happens inside the note requests, so it is part of the compaction requests above.
- Embedding and retrieval: read-back adds only embedding requests, no LLM request.
- When it is paid: only episodes that fold pay for notes and a hand-over, as the Conclusion (§5) now states.
- Thank you for asking for the full accounting. Therefore, the additional generation calls, media reprocessing, embedding and retrieval are all counted, and they stay a small share of an episode's cost.
Q3(b): "End-to-end task latency should also be evaluated."
Thank you very much. We evaluated it: a whole episode takes 6.4 s end to end, from the question to the answer (last row of the table above). Thank you for this request. Hence, end-to-end task latency is now reported next to the request and token counts.
Summary of revisions (revised PDF)
| Where (color) | Change | Answers |
|---|---|---|
| §4.9, Table 5 (blue); App. F.6 (red) | Answer-level metrics, incl. an LLM judge | W1(a), W1(b) |
| §4.9 (blue); Figure 9a–b, Table 12 (red) | Same-budget GRPO | W2 |
| §4.9, Figure 9d, Table 12 (red) | Full cost and wall-clock time | Q3(a), Q3(b) |
| App. F.6 (red) | Repeated folds | Q2(b) |
| Figures 1–9 | 1–3 redrawn; 4–9 recolored | W3 |
| §5, App. H (red) | Conclusion and future work | Q3(a) |
At last, we sincerely appreciate your valuable feedback. We have carefully considered all your suggestions and substantially improved and updated the paper accordingly. If our responses and revisions address your concerns, we would be deeply grateful if you could kindly reconsider raising the score to 3 or above. Thank you very much!
Response to Reviewer 28rV (Part 1/3)
Thank you very much for your time and thorough review, and for recognizing that the analysis of context-window saturation is valuable and that the per-environment moving baseline is a sensible design for single-rollout RL. Your concerns are answer-level accuracy next to token-level F1 (W1(a)), how correctness is determined for open-ended questions (W1(b)), a controlled comparison with GRPO (W2), notation and concrete examples (W3), a comparison with standard vector retrieval (Q1), unsupported claims or omitted qualifications in the notes (Q2(a)), their effects on retrieval and final answers (Q2(b)), the additional calls of compaction (Q3(a)) and end-to-end task latency (Q3(b)). We answer each below with evidence now in the paper.
We have substantially improved the paper's writing, figure and table layout, and supplementary experiments, among other details, and we kindly invite you to open the revised PDF to view them.
In the revised PDF, our revisions for you are in blue, together with those in red, which marks the changes that address points raised by several reviewers, i.e., the overlapping concerns. Most of them are in the new §4.9 (Analysis under Controlled Checks, p. 9: Table 5, Figure 9) and the new App. F.6 (pp. 27–28: Table 12). Within §4.9, the Table 5 caption and the paragraphs on answer-level metrics and on GRPO are blue, while the cost paragraph and the Figure 9 caption are red, as are App. F.6 and App. H.
All new runs use the same Qwen2.5-Omni-3B checkpoint, W=8K and K=8, and greedy decoding. Under this protocol the full system scores 27.5 Avg. All (27.8 in a rerun), against the three-run mean of 27.0 in Table 2 (§4.9; App. F.6). The new comparisons below therefore use 27.5 for our full system, and the rescoring of the Table 2 runs uses 27.0.
W1(a): "token-level F1 as both the training reward and evaluation metric ... report answer-level accuracy ..."
Thank you very much. Following your suggestion, we now report answer-level metrics next to token F1 (new Table 5, §4.9; App. F.6), and OmniPlex-RL leads under every one of them, including an LLM judge from outside the Qwen family.
W1(b): "... specify how correctness is determined for open-ended questions."
Thank you very much. For open-ended questions, correctness is determined by the LLM judge under one standard, applied to every column, ours and the baselines alike.