- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces OphMem, an agent-memory framework for resolving visually ambiguous events in ophthalmic surgical videos. During a question, the agent repeatedly retrieves textual procedural strategies from a persistent bank and uses them to guide video inspection and tool calls. After answering, good tool-action steps are abstracted into reusable experience cards, medical claims are checked against a knowledge source, and candidates are added, merged, skipped, or dropped. The bank remains fixed within a question and evolves between questions.
Critical strengths
-
This paper targets an underexplored form of visual ambiguity. Repeated instruments and anatomy can make local surgical views genuinely difficult to distinguish. Retrieving prior inspection strategies is a plausible way to decide which earlier evidence should be examined.
-
The benchmark construction and sequential protocol are reasonable. The authors build three complementary tasks from existing case-separated annotations, document the visual-pair mining rules, preserve causal cutoffs, use exact deterministic scoring, and compare structured memory with raw-case and no-memory variants.
Critical weaknesses
-
The system seems not reproducible. Essential details—including the backbone, retrieval method, prompts, tool budget, medical knowledge source, and memory-update implementation—are missing. The definitions of “successful” actions and medically supported claims are also unclear.
-
The evaluation seems not to isolate the benefit of cross-case memory. Results use only one fixed question order, without repeated runs, order permutations, uncertainty estimates, or irrelevant/frozen-memory controls. Baseline compute is not consistently matched, and the automatically mined “ambiguous” pairs lack clinician validation.
Questions for authors
-
Do the authors have a frozen-bank control using only acquisition/development experience, or a control that retrieves an equal amount of shuffled/irrelevant experience?
-
Did ophthalmologists review the mined evaluation pairs for visual ambiguity, label validity, and necessity of historical evidence?