- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents QMA-RAG, a way to help AI models answer questions about long videos, which have way too many frames to look at all at once. A common trick, retrieval-augmented generation (RAG), first writes short descriptions of the video at three zoom levels: atomic clips (about one minute each), events (a single activity), and macro scenes (several events together). It then picks the frames whose descriptions match the question. The problem is that these levels often describe the same moment, so the limited frame slots get filled with repeats. QMA-RAG fixes this by remembering which clips each description came from. A rule-based planner first decides what the question needs: a small detail, the order of events, or the big picture. The method then adds up the scores from all three levels for each clip (cross-level fusion) and picks clips that are useful but not repetitive. Finally, it gives the answering model up to 320 frames in time order, along with timestamps, retrieved text, and a verification step.
The paper tested this on three benchmarks (MLVU, Video-MME, and LongVideoBench) with three different answering models. QMA-RAG beat Uniform sampling every time, by 3.80 points on average. It also beat the graph-based method Vgent by 1.60 points on average while running a bit faster. It did slightly better than the frame selection methods AKS and DIG too. But when you look at the chosen frames alone, it does about as well as AKS and a bit worse than DIG. So much of its edge comes from the extra text and verification it gives the model, not from picking better frames.
Critical strengths
The paper has a few real strengths, mostly in how the problem is set up and how the experiments are run. The idea itself is not very new, but some parts of the work are solid.
A clear problem. The paper points out that when you search at several levels (atomic clips, events, and macro scenes), different results often point to the same moment, which wastes frame slots. This is a real issue, and Figure 1 explains it clearly. Combining scores per clip (cross-level fusion) is a reasonable way to deal with it.
Useful experiments. The authors test their main idea by keeping everything else the same and changing only how frames are picked. In that test, the method does 1.66 points better than simply removing duplicate frames. They also add the extra inputs one at a time (timestamps, retrieved text, verification), which shows how much each one adds. This also shows that the chosen frames alone are not better than the frame selection methods.
Consistent results. QMA-RAG beats Uniform sampling and Vgent in all nine combinations of benchmarks and answering models, though some of the margins over Vgent are small. It needs no training, works with different answering models, and builds each video’s memory once and reuses it. It also runs a bit faster than Vgent.
Critical weaknesses
The paper has some useful experiments, but there are real problems with how fair its comparisons are, how much is left unexplained, and how far its results can be trusted.
-
When only the chosen frames are given to the model, QMA-RAG does about the same as AKS and worse than DIG (Table 6 vs. Table 2). Its win over these methods comes from extra text and a verification step that they never get, so the comparison isn't really fair (Section 4.1, "Rerun protocol and comparison scope").
-
The rule-based planner and the scoring weights were built on MLVU, and all the ablations and parameter tests were also run on MLVU (Section 4.5; Section 5 opening; Appendix B.3). Every parameter sweep peaks exactly at the chosen setting (Figure 3), which suggests the method may be fitted to this one benchmark.
-
Some of the gains are tiny, like 0.30 points over DIG on LongVideoBench, which is only about 4 questions (Table 2; Table 9), or 0.33 points over Vgent with LongVU on Video-MME (Table 1). Without significance tests, which the authors admit are missing (Appendix E), we can't tell if these gains are real or just noise.
-
Several important parts are never explained. These include the verification step (Section 3.4), the model that writes the video descriptions (Section 3.2), the planner's actual rules, and some terms in the scoring equations (Appendix B.3, Eqs. 4–5). This makes the method hard to reproduce or judge, and no code release is promised (Reproducibility Statement).
-
Most ablations drop the counting task from about 59% to about 42% (Table 7), and that drop alone explains most of the overall losses. Since counting has only 206 questions (Table 8), many of the paper's conclusions rest on a few dozen questions.
-
The method mostly combines known ideas: video graphs, weighted scores, picking relevant but non-repetitive clips, and removing duplicates (Sections 2 and 3.3). It also depends on many hand-picked weights and word-based rules (Appendix B.3) that seem fragile. A simpler planner version even scored below plain Uniform sampling (Table 3).
-
The paper cites close RAG methods like AdaVideoRAG and ViG-RAG, plus VideoTree and A.I.R. (Section 2), but doesn't compare against any of them. It only runs Uniform, Vgent, AKS, and DIG (Section 4.1). It also doesn't test on hour-long videos or on benchmarks built to require several separate pieces of evidence, even though that is exactly what the method claims to handle.
Questions for authors
-
Do AKS and DIG get the same retrieved text, timestamps, and verification step as QMA-RAG, or only their selected frames? On MLVU, QMA-RAG's frames alone score 72.21%, about the same as AKS (72.22%) and below DIG (72.72%). Is it correct, then, that the gains over these methods come from the extra inputs rather than from better frame choices? This matters for how I judge the paper's main contribution.
-
What exactly does the verification step do, and what is the "symbolic answering" mentioned in the failure examples? What text evidence is passed to the answering model, and which model writes the clip, event, and scene descriptions? Does the Uniform baseline also receive subtitles? Without these details, it's hard to tell whether the comparisons with the baselines are fair.
-
Were the planner rules, the scoring weights, and the thresholds chosen by looking at results on the same MLVU questions that were used for the main results and ablations? If so, the parameter tests peaking exactly at the chosen settings, and the larger gains on MLVU than on the other benchmarks, may partly reflect tuning rather than a general improvement.