- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces a valuable benchmark and demonstrates substantial improvements in memory-driven proactive assistance, but limited comparisons with strong baselines and unresolved evaluation validity concerns prevent a stronger recommendation.
Critical strengths
1. Well-designed benchmark and evaluation: The paper introduces a meaningful benchmark that jointly evaluates memory retrieval, intervention timing, and response correctness, addressing an important gap in proactive assistance research.
2. Strong empirical results and ablation studies: GPM substantially improves performance across different models and settings, while controlled experiments and ablations provide convincing evidence for the effectiveness of its key components.
Critical weaknesses
1. Limited comparison with strong baselines: The paper lacks direct comparisons with closely related graph-based memory systems, such as EgoMemo, making it difficult to establish GPM's incremental novelty (Section 6.1).
2. Limited benchmark realism: The benchmark relies on synthetic personal memories and predefined trigger windows, which may not fully reflect complex real-world proactive assistance scenarios (Section 3, Appendix C).
3. Inconsistent trigger-window definition: Section 3.1 and Appendix C appear to define trigger windows differently, raising concerns about the validity of timing-based evaluation.
4. Insufficient evaluation reliability: The paper relies on an LLM judge without reporting independent human agreement on model-generated responses, and lacks confidence intervals or multi-seed evaluations (Sections 3.3 and 6).
5. Limited real-world validation: Although the benchmark uses real egocentric videos, all personal histories and reminder rationales are synthetically constructed. Without evaluation on naturally collected user memories or real-world longitudinal interactions, it remains unclear whether the proposed method generalizes to authentic proactive assistance scenarios.
Questions for authors
Q1. How much of GPM's improvement remains against a strong graph-memory proactive baseline?
EgoMemo already performs memory-augmented proactive reasoning using a semantic knowledge graph and retrieved temporal context. Could the authors adapt an EgoMemo-style baseline to MPA with the same generator backbone, or provide a comparably strong structured-memory baseline with matched inference budget?
Q2. How are intermittent target appearances handled in the actual trigger-window annotations?
Section 3.1 defines the valid window using the first appearance, whereas Appendix C Step 1 describes using the first and last visible timestamps over the clip. Which rule is used by the released annotations and evaluation implementation? How many samples have multiple disjoint target appearances?
8 comments
Response to Reviewer wvxt (2/4)
W2 & W5. Benchmark realism and real-world validation.
W2: "Limited benchmark realism: The benchmark relies on synthetic personal memories and predefined trigger windows, which may not fully reflect complex real-world proactive assistance scenarios (Section 3, Appendix C)."
W5: "Limited real-world validation: Without evaluation on naturally collected user memories or real-world longitudinal interactions, it remains unclear whether the proposed method generalizes to authentic proactive assistance scenarios."
We appreciate the reviewer raising these points. Both concern how our benchmark and method relate to real data, so we answer them together. We first show that our method works on real memories, and then explain how the realism of the synthetic memories and the predefined trigger windows is ensured.
Evaluation on real memories. We evaluate on EgoLife data annotated by EgoServe, where both the egocentric video and the personal history (the user's own earlier recordings) are real. Unlike our benchmark, which annotates the full span from when the target object appears to when it disappears, EgoServe identifies a short trigger window and adds a ±60 s scoring tolerance on each side. Since most EgoServe services provide general assistance rather than memory-driven help (e.g., safety alerts, tool-use and next-step guidance), we keep only the memory-related types: Memory Recall, Memory Link, and Task Reminder. To ensure that the memory evidence lies outside both the test clip and the ±60 s tolerance window, we further keep only clips whose evidence occurs at least 3 minutes before the trigger, yielding 65 clips. The reminder can thus only be justified by memory rather than by the current view. We first follow EgoServe's ±60 s protocol. However, ±60 s is too loose for proactive reminders: a reminder up to a minute early or late still counts as correct, so being scored correct says little about whether it came at the right moment. We therefore additionally report a tighter ±14 s window, comparable to the median trigger-window length in our benchmark (24s+2s tolerance). We compare GPM with EgoMemo, a graph-memory method designed for EgoLife:
| Method | PERFECT, ±60 s (%) | HIT, ±60 s (%) | PERFECT, ±14 s (%) | HIT, ±14 s (%) |
|---|---|---|---|---|
| EgoMemo | 12.3 | 12.3 | 7.7 | 7.7 |
| GPM (ours) | 38.5 | 83.1 | 21.5 | 63.1 |
Under both tolerance settings, GPM clearly outperforms EgoMemo, a method designed specifically for EgoLife: under the tighter ±14 s window, GPM reaches 21.5% PERFECT and 63.1% HIT, versus 7.7% and 7.7% for EgoMemo. This shows that GPM's advantage holds on real memories: on naturally collected memories, GPM still retrieves the right memory and reminds at the right time.
Realism of the synthetic memories. We acknowledge that, to our knowledge, no publicly available egocentric dataset currently covers a user's life over a full year, which is necessary to build long-horizon personal memories; we therefore construct the personal histories synthetically. To make them as realistic as possible, realism is ensured during construction and then verified by human annotators.
-
Built from a coherent simulated life. To make the user's activities unfold realistically, memories are not isolated facts but are embedded in a full simulated life. We first synthesize a persona (biography, relationships, routines). Conditioned on it, we build a year-long event timeline with different categories of events: main-arc milestones, persona-derived interests, neutral daily distractors, and the evidence. Every event is then rendered into timestamped first-person captions (
[V]visible action,[C]conversation). For example (App. D.3), the memory "Watering the basil straight into the middle caused the soil to crater; you decided to water around the edges next time" (~14 days before the trigger) is rendered into a realistic first-person caption stream of daily life (abbreviated):[V] 12:32:58I stand in front of the potted basil plant on the windowsill.[V] 12:33:09I see water pouring from the spout into the middle of the basil's soil.[V] 12:33:18I see the soil begin to sink in, forming a small crater.[C] 12:33:28I mutter to myself, "That crater's not great."[V] 12:33:44I reach forward to touch the edge of the crater with my index finger.[C] 12:33:52I mutter to myself, "Next time, around the edges."This episode is one of many everyday events in a ~618K-token year-long history, so, as in a real lifelog, the relevant moment must be found among many unrelated events.
-
Verified by human annotators. Annotators are shown both the abstract memory and the life stream into which it is rendered, and judge memory–event consistency: whether the rendered life stream is realistic and describes the same event as the memory. 98.4% of judgments answer yes.
Response to Reviewer wvxt (3/4)
Predefined trigger windows. The trigger window of every sample is ensured to be a moment worth intervening, first during construction and then by human review (App. C).
-
Filtered during construction. Each memory is generated for a specific target appearance and must pass a worth-triggering gate, which checks that the memory is grounded in the video, specific to the target, and that seeing the target genuinely warrants a reminder. Each sample then passes model-based quality control.
-
Verified by human annotators against explicit rubrics. 32 trained annotators watch the egocentric clip alongside the target, the memory, and the proposed trigger window; 87.7% of samples are reviewed by three independent annotators. They judge:
- Target visibility: is the target object needed for the assistance clearly visible and is it the described object? (95.8% yes)
- Trigger window: is the proposed window correct? Annotators keep it, adjust its boundaries, or mark it unusable. (82.7% kept, 14.2% adjusted, 3.1% unusable)
- Reason to remind: given the user's memory, is the reminder genuinely warranted at this moment? (93.0% yes)
Each annotator then gives a verdict (accept, fix the window, or reject), and a sample enters the benchmark only if the majority accepts it (1,804 samples, 83.8% of those reviewed).
Summary. GPM's advantage holds on real memories: it clearly outperforms EgoMemo there under both tolerance settings, while the synthetic memories and trigger windows are built to be realistic and are verified by human annotators. Together, these support the realism of our benchmark and the effectiveness of GPM beyond synthetic data.
W3 & Q2. Consistency of the trigger-window definition and intermittent target appearances.
W3: "Inconsistent trigger-window definition: Section 3.1 and Appendix C appear to define trigger windows differently, raising concerns about the validity of timing-based evaluation."
Q2: "How are intermittent target appearances handled in the actual trigger-window annotations? Which rule is used by the released annotations and evaluation implementation? How many samples have multiple disjoint target appearances?"
Thank you for pointing this out, and we are sorry for the confusion. Both questions concern the same issue, so we answer them together. The two descriptions refer to the same window; we first clarify the full construction process of the trigger window, and then answer which rule is used and how many samples have disjoint appearances.
- Locate the target appearance. We first locate the target object and scan it over the entire video, recording when it is visible. Brief gaps of at most 8 s (look-aways or occlusions) are merged, so one continuous appearance forms one window, from its first to its last visible timestamp (App. C, Step 1).
- One appearance per clip. When cutting the test clips, we select each clip to contain exactly one such appearance. Hence, each clip has a single trigger window, and during the rest of the clip the system should remain silent.
- Human verification. Annotators then check each window on the video and manually fine-tune the boundaries of windows they find inappropriate (14.2% adjusted).
Because each clip contains only one appearance, the "first appearance" in Sec. 3.1 and the "first and last visible timestamps" in App. C describe the same window. The first-appearance rule in Sec. 3.1 states the general principle, while App. C describes how this window is concretely built. The benefit of this design is that the right moment is unambiguous: the assistant should intervene at the earliest appropriate moment, and once it has passed, re-raising the same matter at a later reappearance would be a redundant interruption rather than a new opportunity to help. With a single window per clip, the ground truth is correct and unambiguous, and any fire outside it can be cleanly counted as early, late, or repeated triggering. Both the annotations and the evaluation use this single human-verified window. We will revise Sec. 3.1 and App. C to make this consistent.
Which rule is used, and how many samples have disjoint appearances? The released annotations and evaluation implementation use the single human-verified window of each clip. Since each clip is selected to contain exactly one target appearance (with gaps of at most 8 s merged), no sample has multiple disjoint target appearances. This single-window design guarantees that the ground truth is correct and unambiguous: each clip has exactly one moment where a reminder is warranted, so any fire outside this window can be cleanly counted as early, late, or repeated triggering. We additionally include 150 negative clips that contain no moment requiring a reminder at all, to evaluate whether the system stays silent.
Response to Reviewer wvxt (1/4)
Dear Reviewer wvxt,
We sincerely thank you for your detailed review and constructive suggestions. We hope our responses below address your concerns and merit a better score.
W1. Comparison with strong graph-memory baselines (e.g., EgoMemo).
Thank you for this great suggestion. We acknowledge that EgoServe and EgoMemo are excellent, foundational concurrent work on proactive assistance, and EgoMemo is a strong graph-memory method in this direction. Below, we first clarify the setting differences and then establish a direct comparison.
Setting differences. EgoServe studies short-horizon assistance within the same day; our MPA benchmark targets reminders that can only be justified by long-term personal memory, and evaluates them strictly.
Direct comparison. Following your suggestion, we compare GPM with EgoMemo in two ways: (1) in our setting, we adapt EgoMemo to our MPA benchmark and compare it with GPM; (2) in their setting, we run GPM on EgoLife data annotated by EgoServe and compare it with EgoMemo.
(1) EgoMemo on our MPA benchmark. We adapt EgoMemo with its native pipeline and our scoring rule, and evaluate on 100 randomly sampled clips. To match the generator backbone, we also run EgoMemo with GPM's decision model, Qwen3.8-27B:
(2) GPM on EgoServe's EgoLife data. Here both the egocentric video and the personal history (the user’s own earlier recordings) are real. Unlike our benchmark, which annotates the full span from when the target object appears to when it disappears, EgoServe identifies a short trigger window and adds a ±60 s scoring tolerance on each side. Since most EgoServe services provide general assistance rather than memory-driven help (e.g., safety alerts, tool-use and next-step guidance), we keep only the memory-related types: Memory Recall, Memory Link, and Task Reminder. To ensure that the memory evidence lies outside both the test clip and the ±60 s tolerance window, we further keep only clips whose evidence occurs at least 3 minutes before the trigger, yielding 65 clips. The reminder can thus only be justified by memory rather than by the current view. We first follow EgoServe's ±60 s protocol. However, ±60 s is too loose for proactive reminders: a reminder up to a minute early or late still counts as correct, so being scored correct says little about whether it came at the right moment. We therefore additionally report a tighter ±14 s window, comparable to the median trigger-window length in our benchmark (24 s).
In both settings, GPM clearly outperforms EgoMemo.
Why EgoMemo performs poorly in our setting. EgoMemo's pipeline first looks at the current scene and then decides whether to retrieve memory. This design is well suited to cases where the object itself is worth a reminder, such as a visible hazard or an unfinished step on the spot. In memory-driven proactive assistance, however, the object in view is often unremarkable on its own; it is the user's memory that makes it worth a reminder. Without memory, it is hard to tell from the scene alone whether retrieval is needed, so EgoMemo often chooses not to retrieve at the very objects that should trigger a reminder (it issues no reminder on 86% of clips on our benchmark), which leads to its low HIT and PERFECT.
Why GPM performs well. GPM keeps the relevant memory in view and then uses its gate to post-filter what to say based on that memory. This balances both sides: it does not miss memory-driven triggers, while the gate filters out unnecessary reminders.