acceptodds
Under review as a conference paper at ICLR 2027

Scaling Video Understanding: Compact Latent Collaboration for Budget-Constrained Distributed Agents

Abstract

Multi-modal large language models (MLLMs) advance vision–language understanding but face inherent limitations in long-video tasks due to bounded perception context budgets. Existing agentic methods mitigate this via rule-based preprocessing, yet often suffer from information loss, high cost, and reliance on textual intermediates. We propose MACF-Video, an end-to-end Multi-Agent Collaboration Framework that decouples per-agent perception budgets from global video complexity, enabling scalable video understanding while preserving visual fidelity. MACF-Video partitions videos into segments for locally budgeted agents and enables holistic reasoning via an agent-native latent communication protocol. Each agent encodes partial observations into compact, task-sufficient tokens in a shared embedding space, allowing efficient and information-preserving collaboration by a central coordinator. We introduce a curriculum training strategy that progressively enforces semantic alignment, evidence summarization, and cross-agent coordination. Extensive experiments on diverse video understanding benchmarks show that MACF-Video consistently outperforms state-of-the-art MLLMs and multi-agent systems under identical budget constraints, demonstrating the effectiveness of our latent collaboration for scalable video understanding.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video, video understanding, multi-agent

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RfnXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies long-video QA when each model call can ingest only a fixed number of pixels (16 frames at 224×224). MACF-Video splits a video into M temporal segments, each handled by a local agent (Qwen3-VL-8B with LoRA, weights shared) that compresses its segment and the question into K=32 continuous tokens through a two-layer MLP adapter. A coordinator with the same backbone reads the M×K tokens plus the question and produces the answer. Training has three stages: caption reconstruction to align the token space, image QA to make the tokens question-aware, and video QA with M=4 agents; inference uses M=6. On Video-MME, LongVideoBench, LVBench and MLVU-Test, the method improves over Qwen3-VL-8B by 4.5–7.7 points, over a re-implemented MapReduce by 9.7–18.6 points and over a re-implemented LatentMAS by 4.7–8.3 points (Table 1). The paper ablates the curriculum stages (Appendix D), the token count K (Table 2) and the agent count (Figure 4, Appendix G), and Appendix E compares the three communication protocols under a fixed 192-token message budget.

Critical strengths
  1. A controlled comparison of communication protocols. Appendix E fixes the backbone, partitioning, perception budget and a 192-token message budget for all three protocols. The learned latent tokens reach 60.4/56.8/40.2/49.2 on the four benchmarks, against 51.1/42.7/29.7/38.9 for LatentMAS and 44.5/36.9/30.3/32.4 for MapReduce. Both baselines are training-free and MACF-Video's tokens are trained, so the gap measures the trained message format together with its training. Within that scope it is the best-controlled experiment in the paper: once the work is split across agents, trained latent messages carry more task-relevant information than captions or truncated KV states of the same size. Figures 3 and 6 show examples where caption-based messages lead to a wrong answer and the latent tokens do not; in Figure 3 a caption binds the queried object to the wrong entity.
  2. The training recipe is ablated stage by stage. Removing stage 1 or stage 3 lowers Video-MME from 60.4 to 45.5 or 48.2 (Appendix D). This shows that the curriculum is required and makes the method reproducible as a recipe.
Critical weaknesses
  1. The single-model comparison matches neither total visual input nor training data, so the main claim is not established. Table 1 lists 16×224×224 for every budget-constrained row; for MACF-Video this is the per-agent budget. The MACF-Video rows use M=6 (Section 4.1; the M=6 entry of Table 7 equals the 56.8 in Table 1), so each question is answered from 96 frames, six times the input of the single-model rows. Appendix I motivates the per-call budget by deployment limits, and also states that 16×224×224 "serves as a stand-in for the effective per-forward visual context capacity rather than a hard model limit". Section 4.3(i) attributes the gain from more agents to wider temporal coverage. Figure 4 and Table 7 are consistent with this: with one agent, LongVideoBench accuracy is about 48.5 (read from Figure 4; Table 7 starts at M=2), below the 50.7 of the Qwen3-VL-8B row in Table 1, and it rises as agents, and therefore frames, are added. In addition, MACF-Video is fine-tuned with LoRA on LLaVA-Video-178K captions, Video-R1 image and video QA, and Molmo2 data, while the paper describes no fine-tuning of the Qwen3-VL-8B baseline. Two controls are needed: (a) Qwen3-VL-8B given 96 frames at 224×224, which fits well within its context; (b) Qwen3-VL-8B fine-tuned with the same LoRA setup on the stage-3 data, at 16 and at 96 frames. If these controls match MACF-Video, the contribution reduces to splitting one forward pass across several calls, which would then need memory and latency evidence. If MACF-Video is better, the claim becomes much stronger. My rating depends on these results.
  2. The re-implemented multi-agent baselines score below the single model they are built on. MapReduce (46.7/38.2/30.5/33.3) is below Qwen3-VL-8B (55.9/50.7/33.2/41.5) on all four benchmarks, and LatentMAS (55.7/48.5/33.2/42.3) ties or trails it on three. Both baselines see the same six segments. In the original MR. Video paper, the MapReduce design scores 60.8 on LVBench against 48.6 for Gemini-2.0-Flash, the VLM it builds on. Both baselines were modified: MR. Video's Gemini-2.0-Flash captioner and GPT-4o reasoner were both replaced by Qwen3-VL-8B, and the LatentMAS KV caches were concatenated and subsampled to 724 entries. The paper does not show that the re-implementations reproduce published results in their original setting, so the margins over these baselines cannot be attributed to the communication design. Section 4.2 also reports margins over LatentMAS of 9.3/14.1/10.5/10.3 points; these are the margins against the 192-token-capped LatentMAS of Appendix E, while against the Table 1 LatentMAS the margins are 4.7/8.3/7.0/6.9.
Questions for authors

Clarifications that could change my rating:

  1. What does Qwen3-VL-8B score on the four benchmarks with 96 frames at 224×224 (the total input of six agents)?
  2. What does Qwen3-VL-8B score after LoRA fine-tuning on the stage-3 data, at 16 and at 96 frames?
  3. Does the MapReduce re-implementation reproduce the MR. Video results in its original configuration? Why does it fall below the single model it wraps?
  4. What are the exact M=1 results on all four benchmarks?

Presentation fixes: report the total number of frames per question for each row of Table 1; make the LatentMAS margins in Section 4.2 consistent with Table 1; Figure 4 and Table 7 show LongVideoBench, while Section 4.3 says the agent-count study is on MLVU-Test.

Suggestions that do not affect my rating: report peak memory and wall-clock time against a single model with the same total frames; probe what the K tokens encode (segment position, question type); evaluate a multi-camera setting, which matches the distributed-device motivation better than temporal splits of one video.

5 comments
AuthorsOpenReview ↗

Response to Reviewer RfnX (1/2)

W1 & Q1+Q2+Q4: Comparison at matched total frames and training data

As suggested, we fine-tune Qwen3-VL-8B using the same Stage-3 training data and LoRA configuration as ours. We evaluate the original and fine-tuned baselines with 16 and 96 input frames at 224×224 resolution, alongside our single-agent setting (M=1) in Table R3-1 (Table 4 in revised paper).

At 16 frames, our single-agent setting (M=1) is comparable to the baselines but generally slightly below them. This gap comes from compressing the whole video into only 32 communication tokens, a design built for efficient inter-agent communication rather than single-agent inference.

At 96 total frames, our multi-agent setting (M=6) achieves the best accuracy on all long-video benchmarks (Video-MME-Long, LongVideoBench-Long, LVBench, and MLVU-Test), This shows that distributed processing with inter-agent communication remains effective despite compression. As shown in Table R3-2, it also cuts inference latency by 45.6% (0.988 → 0.537 s) and peak memory excluding the backbone by 82.0% (2.95 → 0.53 GiB).

Table R3-1. Comparison with fine-tuned baselines.

MethodTotal input framesVideo-MMEVideo-MME-LongLongVideoBenchLongVideoBench-LongLVBenchMLVU-Test
Qwen3-VL-8B1655.949.950.743.333.241.5
Qwen3-VL-8B + SFT1656.250.751.945.735.842.8
Ours (M=1)1652.348.148.443.632.541.6
Qwen3-VL-8B9662.553.856.647.938.747.4
Qwen3-VL-8B + SFT9661.853.657.347.939.548.3
Ours (M=6)9660.454.656.849.540.249.2

Table R3-2. Cost comparison with the single-model baseline using the same total input.

MethodTotal input framesPeak MemoryPeak Memory (excluding backbone residency)Inference Latency
Qwen3-VL-8B9619.30 GiB2.95 GiB0.988s
Ours (M=6)9617.14 GiB0.53GiB0.537s
AuthorsOpenReview ↗

Response to Reviewer RfnX (2/2)

W2 & Q3: Re-implementation of multi-agent baselines

In our paper, we aim to compare communication representations and mechanisms (text / KV-cache vs. latent tokens) rather than the complete multi-agent workflows. We therefore aligned the baselines with MACF-Video's configuration. We acknowledge the differences from the original published settings and, as suggested, add experiments following those settings for further explanation.

MapReduce. We first explain why the published MR. Video result exceeds that of the single Gemini-2.0-Flash model, while our implementation with the Qwen3-VL-8B backbone performs worse than the single model.

Although MR. Video uses Gemini-2.0-Flash to generate captions, it uses GPT-4o as the reasoner in an iterative agentic loop. The loop also consumes far more input frames, exceeding 512 in our reproduction statistics. These differences may contribute to its higher score (60.8 vs. 48.6).

In our re-implementation, the controlled setting isolates the effect of text-based vs. latent-token communication, rather than comparing full MapReduce implementations. We apply MapReduce* with the same six-segment partition (16 frames per segment) and the same per-agent visual budget as MACF-Video (16*224*224). We also use Qwen3-VL-8B as both captioner and reasoner and run the workflow as a single pass, fully matching the MACF-Video setting. Although we used * to distinguish it from the original MapReduce method, this notation may not be appropriate. We therefore rename this baseline Text-Communication instead of MapReduce* in the revised paper (Section 4.1 and Table 1).

For reference, Table R3-3 summarizes the differences between MR. Video paper and our implementation, though they are not the focus of our work. Replacing only the reasoner (GPT-4o → Qwen3-VL-8B) lowers LVBench accuracy by 6.9 points (60.8 → 53.9). With a single-pass workflow, the text-based communication exceeds the reasoner's context length using the frame inputs in MR. Video.

Table R3-3. Comparison of the published MR. Video configuration and our re-implementation.

VariantCaptionerReasonerWorkflowTotal Input FramesLVBench
Published paperGemini-2.0-FlashGPT-4oLoop>51260.8
Ablation 1Gemini-2.0-FlashQwen3-VL-8BLoop>51253.9
Ablation 2Gemini-2.0-FlashQwen3-VL-8Bsingle-pass>512Out of contxt
Matched with MACF-Video (our paper)Qwen3-VL-8BQwen3-VL-8BSingle-pass9630.5

LatentMAS. Similarly, our implementation of LatentMAS uses 724 KV-cache entries to match MACF-Video's communication setting. Following the reviewer's suggestion, we additionally evaluate LatentMAS with its original full KV-cache communication. As shown in Table R3-4, our method achieves higher accuracy on all long-video benchmarks while using only 0.3% of the communication payload (1.50 vs. 481.77 MiB). We will also rename the original LatentMAS* baseline KVCache-Communication for clarity.

Table R3-4. Comparison with LatentMAS using full KV-caches.

MethodVideo-MMEVideo-MME-LongLongVideoBenchLongVideoBench-LongLVBenchMLVU-Testcommunication payload
LatentMAS (full KV-cache)61.052.356.048.938.345.8481.77 MiB (3426 KV-Cache)
Ours (M=6)60.454.656.849.540.249.21.50 MiB

Presentation Fixes

We have fixed the presentations in the revision as follows:

  • Clarified the total frame number in the caption of Table 1.
  • Corrected the LatentMAS margins in Section 4.2 to match Table 1.
  • Corrected the benchmark name for the agent count ablation from MLVU-Test to LongVideoBench to match Figure 4 and Table 10 (Appendix G).

Dear Reviewer RfnX,

We would like to kindly follow up and ask whether our response has adequately addressed your concerns. If any questions or concerns remain, we would greatly appreciate the opportunity to discuss them further.

Thank you again for your time and thoughtful feedback.

Reviewer_kYm5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MACF-Video is an end-to-end multi-agent framework that enables scalable long-video understanding by decoupling per-agent perception budgets from global video complexity through compact latent-token communication and curriculum training. Under identical budget constraints, it consistently outperforms state-of-the-art MLLMs and multi-agent systems across video understanding benchmarks.

Critical strengths

1.The problem addressed by the paper has clear practical significance. By partitioning long videos and processing the resulting segments with multiple sub-agents, the proposed approach improves accuracy while balancing processing time.

2.The use of latent tokens for communication among sub-agents helps mitigate the information loss that may arise from text-based summarization.

Critical weaknesses

1.The implementation details of the latent tokens are insufficiently specified. In the absence of concrete code, I can only infer from the description that each local segment produces KK fixed tokens as communication latents. This is currently a relatively stable and general approach to latent representations, but it also has limitations: while a fixed number of tokens provides stability, it may introduce rigidity and limited interpretability. The paper does not attempt to explain what information these latents actually encode. Moreover, it reports only an ablation on the fixed value of KK, without a rigorous ablation of the architectural components that constitute the latent-token mechanism.

2.The fairness of the computational budget may be questionable. Although the input budget is set to 16×224×22416 \times 224 \times 224, when multiple sub-agents operate in parallel, the total input processed simultaneously is effectively M×16×224×224M \times 16 \times 224 \times 224, where MM denotes the number of sub-agents. This also explains the increase in inference time.

Questions for authors
  • What motivates the choice of KK? Was a more systematic sweep conducted, or was the relationship between KK and video length, question difficulty, and the number of agents analyzed?
  • If some latent tokens were randomly masked or permuted, for example, by replacing all or part of them with noise, or by randomly substituting latents produced by other sub-agents, would performance degrade? Do different latent tokens exhibit meaningful diversity?
  • Beyond final accuracy, are there experiments that directly verify what visual evidence the latent tokens preserve, such as temporal localization, spatial relations, or entity–attribute binding?
5 comments
AuthorsOpenReview ↗

Response to Reviewer kYm5 (1/2)

W1: Implementation details of latent communication tokens and ablation

As described in Appendix A, we use KK special tokens, denoted by <COM_K>, to mark the communication-token positions. Each local agent processes its video segment and the question together with these tokens. We extract the last-layer hidden states at the marked positions and project them through a two-layer MLP adapter into the coordinator’s embedding space, yielding c(m)∈RK×d\mathbf{c}^{(m)}\in\mathbb{R}^{K\times d}. The coordinator then receives the concatenated communication vectors from all local agents together with the question embeddings and generates the final answer.

The adapter is the architectural component that constitutes the latent-token mechanism. We therefore ablate its design in Table R3-1 (Table 5 in the revised paper).

Table R2-1. Ablation study on architectural components for latent communication.

AdapterVideo-MMELongVideoBenchLVBenchMLVU-Test
w/o Adapter58.754.638.446.1
1-layer MLP60.356.239.748.1
2-layer Transformer60.956.541.048.7
2-layer MLP (Ours)60.456.840.249.2

W2: Comparison under matched total visual input

We compare MACF-Video (M=6) with Qwen3-VL-8B under the same total input of 96 frames in Table R2-2. MACF-Video achieves higher accuracy on the all long-video benchmarks (Video-MME-L is the officially defined long-video subset of Video-MME). The single model scores higher only on overall Video-MME. Shorter videos usually fit within a single context, so they do not need to be split across sub-agents and avoid the information loss from compression and communication.

MACF-Video also reduces peak GPU memory by 82.0% (excluding backbone residency) and latency by 45.6%. All memory figures are measured without FlashAttention-2. The efficiency gains have two sources. First, each segment is processed by a separate sub-agent, so segments can be processed in parallel. Second, each sub-agent compresses its segment into a few compact tokens, which shortens the coordinator's input and reduces its reasoning time.

Table R2-2. Cost and accuracy comparison with Qwen3-VL-8B under the same total visual input (96 frames).

MethodPeak MemoryPeak Memory (exclude backbone residency)LatencyVideo-MMEVideo-MME-LLongVideoBenchLVBenchMLVU-Test
Qwen3-VL-8B (96 frames)19.30 GiB2.95 GiB0.988s62.553.856.638.747.4
Ours17.14 GiB0.53GiB0.537s60.454.656.840.249.2

Q1: Choice of KK.

We clarify that KK is chosen for a fixed per-agent perception budget. In our implementation, each local agent compresses an input of 16×224×22416 \times 224 \times 224 into KK tokens. A fixed KK therefore sets a fixed communication bottleneck for each agent under this budget (fixed compression rate). We select KK empirically through a sweep; under our default perception budget, K=32K=32 provides a favorable accuracy–communication trade-off.

Considering very long videos, we first divide them into segments that fit the per-agent perception budget; longer videos can be divided into more segments. Thus, video length affects the number of segments rather than the choice of KK per agent. We agree that harder questions may require more information, i.e., a lower compression rate, which involves a rate–distortion trade-off. We consider an adaptive KK a promising direction for future work.

AuthorsOpenReview ↗

Response to Reviewer kYm5 (2/2)

Q2: Perturbation analysis of communication tokens

As suggested, we add a perturbation analysis of the communication tokens on Video-MME benchmark in Table R2-3 (Table 6 in the revised paper).

Random masking. We replace a random subset of communication vectors with noise. As the mask ratio rises from 25% to 100%, accuracy falls from 59.5% to 38.3%, with the largest drop when all vectors are masked. This shows that the coordinator relies on the information carried by the communication tokens.

Agent-wise permutation. We randomly swap entire sets of latent tokens between sub-agents processing the same video, keeping the token order within each set. Accuracy drops only slightly, from 60.4% to 60.2%.

Top 25% masking. We also check whether different latent tokens carry different amounts of useful information. We score each token slot's contribution to the correct answer using the absolute Gradient × Input attribution with respect to the answer's log-probability. In each sub-agent, we then mask the 25% of slots with the highest scores (Top 25%). This drops accuracy by 9.9 points, versus only 0.9 points when masking 25% at random, indicating that some tokens carry substantially more task-relevant information than others.

Table R2-3. Perturbation analysis of communication tokens. Δ is the change relative to No mask.

MethodNo maskRandom 25%Random 50%Random 75%Random 100%Top 25%Permutation (agent-wise)
Accuracy60.459.557.354.838.350.560.2
Δ–-0.9-3.1-5.6-22.1-9.9-0.2

Q3: Study on latent token and visual evidence.

To examine how latent communication tokens support different kinds of visual evidence, we consider three common types of visual evidence. For each type, we use the matching Video-MME question categories:

  • Attribute: Attribute Perception
  • Spatial: Spatial Perception & Spatial Reasoning
  • Temporal: Temporal Perception & Temporal Reasoning

For each evidence type, we use the attribution procedure from Q2 to find the top 25% of token slots that matter most for that type. We then mask these slots and evaluate the accuracy drop on all three types. In Table R2-4, each row shows which evidence type was used to select the masked slots, and each column shows which evidence type is evaluated. A visualization is provided in Figure 6 in the revised paper.

Table R2-4. Cross-evidence masking of communication tokens. Accuracy drop (%) after masking the Top-25% token slots selected for each evidence type. Rows: evidence type used to select the masked slots. Columns: evidence type evaluated.

Masked slots selected byAttributeSpatialTemporal
Attribute−27.47−12.68−5.17
Spatial−13.85−19.09−1.72
Temporal−3.64−2.46−6.46

Each evidence type suffers its largest accuracy drop when masked with its own top-ranked slots (the diagonal). This suggests that different latent slots specialize in different types of visual evidence.

Dear Reviewer kYm5,

We would like to kindly follow up and ask whether our response has adequately addressed your concerns. If any questions or concerns remain, we would greatly appreciate the opportunity to discuss them further.

Thank you again for your time and thoughtful feedback.

Reviewer_JicuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MACF-Video partitions a video among local VLMs, compresses query-conditioned observations into latent communication tokens, and uses a coordinator to answer questions. A three-stage curriculum trains alignment, evidence summarization, and aggregation. The paper reports improvements across four video-understanding benchmarks and three backbones.

Critical strengths
  1. Consistent empirical applicability. Improvements across three backbones and four benchmarks suggest that the architecture is not tied to one model. The additional heterogeneous-backbone results strengthen this practical contribution (Table 1; Appendix H).
  2. Useful bottleneck analysis. Varying communication tokens, resolution, and agent count provides evidence about when additional perception helps and when aggregation limits performance (Figures 4–5; Table 2).
Critical weaknesses
  1. Training confounds the communication comparison. MACF receives substantial task-specific training, whereas the described MapReduce and LatentMAS baselines do not receive comparable adaptation. Appendix E matches communication-token counts, but does not isolate the benefit of latent communication from additional training.
  2. The efficiency claim exceeds the accounting provided. Six local agents process 96 frames in total, versus 16 for the capped single-model baselines. Moreover, counts of text tokens, latent vectors, and KV entries are not comparable network-byte costs. Table 3 does not establish an end-to-end accuracy–compute advantage.
Questions for authors
  1. Experimental clarification: With training data and update budgets matched, how much advantage remains over a fine-tuned text-based aggregator and backbone?
  2. Budget clarification: What are total visual tokens, transmitted bytes, peak memory, and end-to-end latency? How does MACF compare with a single model given the same total 96 frames, where feasible?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Jicu

W1 & Q1: Clarifying the communication comparison

To make the comparison of communication representations clearer and more focused, we renamed the two baseline multi-agent communication methods. MapReduce* is now Text-Communication, and LatentMAS* is now KVCache-Communication. The new names distinguish our variants from the original MapReduce and LatentMAS methods, which use different input-frame and backbone configurations.

As suggested, we fine-tuned the aggregators of both baselines with the same training data and the same Qwen3-VL-8B backbone as MACF-Video. Table R1-1 reports the new results, and Table 1 in the revised paper has been updated to match.

MACF-Video still outperforms both fine-tuned baselines on all four benchmarks. Our method offers a compact alternative to text-based or KV-cache-based communication, trains stably and performs better.

Table R1-1. Comparison with fine-tuned communication baselines under the same training data and backbone.

MethodVideo-MMELongVideoBenchLVBenchMLVU-TestTotal input frames
Text-Communication (MapReduce*)46.738.230.533.396
Text-Communication + SFT53.248.334.842.696
KVCache-Communication (LatentMAS*)55.748.533.242.396
KVCache-Communication + SFT57.652.535.844.296
MACF-Video60.456.840.249.296

W2 & Q2: Budget clarification.

We extend the compute comparison in original Table 3 with Text-Communication (MapReduce*) and KVCache-Communication (LatentMAS*). Table R1-2 (Table 3 in the revised paper) reports end-to-end latency, transmitted bytes and peak memory overhead (caculated as the peak memory excluding backbone residency). All methods use the same visual input of 96*224*224 (192 visual tokens).

Table R1-2. Comparison of inference and communcation cost. Accuracy is the mean accuracy accross the four benchmarks.

MethodLatencyTransmitted bytesMemory overheadAccuracy
Text-Communication5.156s1.51KiB1.52 GiB37.2
KVCache-Communication0.649s110.25MiB0.72 GiB44.9
Ours0.537s1.50MiB0.53 GiB51.8

MACF-Video achieves the highest accuracy with the lowest latency and memory overhead. Compared with KVCache-Communication, it transmits 73.5× less data, cuts latency by 17.3%. Compared with Text-Communication, it transmits more data but runs 9.6× faster by avoiding redundant text decoding. MACF-Video therefore gives the best accuracy–latency trade-off with far more compact communication than KV-cache.

To match the total visual input, we add a 96-frame Qwen3-VL-8B baseline in Table R1-3 (Table 4 in the revised paper). MACF-Video achieves higher accuracy on all long-video benchmarks (Video-MME-Long, LongVideoBench, LongVideoBench-Long, LVBench and MLVU-Test) and runs 1.8× faster (0.537 s vs. 0.988 s). The single model scores higher only on overall Video-MME. Its shorter videos may not suffer from out-of-context issues, so they need no distribution across sub-agents and lose no information through compression and communication. These results show that MACF-Video scales beyond a single call's perception budget while retaining competitive accuracy through compact latent communication.

Table R1-3. Comparison with Qwen3-VL-8B with the same total visual input (96 frames)

MethodVideo-MMEVideo-MME-LLongVideoBenchLongVideoBench-LLVBenchMLVU-TestPerc. BudgetLatency
Qwen3-VL-8B62.553.856.647.938.747.496*224*2240.988s
Ours (M=6)60.454.656.849.540.249.216*224*2240.537s

Dear Reviewer Jicu,

We would like to kindly follow up and ask whether our response has adequately addressed your concerns. If any questions or concerns remain, we would greatly appreciate the opportunity to discuss them further.

Thank you again for your time and thoughtful feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Reply & Rebuttal Period Paper Revision Summary

We thank all three reviewers for their constructive feedback on MACF-Video. In response, we:

  • expanded our comparisons under matched visual input and training conditions;
  • clarified how the communication baselines were implemented;
  • added ablations to examine how the latent communication tokens contribute to performance.

All revisions are highlighted in blue in the revised manuscript. Below, we summarize the main updates for each reviewer.


For Reviewer Jicu:

  1. Comparison with multi-agent communication baselines.
    • We add a comparison with a supervised fine-tuned Text-Communication and KVCache-Communication trained on the same data as MACF-Video (Table R1-1; Table 1 in the revised paper).
    • We also report detailed inference and communication cost, including end-to-end inference latency, peak memory overhead, estimated communication payloads (Table R1-2; Table 3 in the revised paper).
  2. Comparison with the single-model baseline. We added a comparison with Qwen3-VL-8B as a single model under the same total input of 96 frames (Table R1-3; Table 4 in the revised paper).

For Reviewer kYm5:

  1. Communication mechanism. We clarify how latent communication is implemented and add an ablation of the adapter architectures it uses (Table R2-1; Table 5 in the revised paper).
  2. Matched-input baseline. We added a comparison with Qwen3-VL-8B at a matched total input of 96 frames (Table R2-2; Table 4 in the revised paper).
  3. Choice of KK. We clarify how we select the number of communication tokens per agent.
  4. Latent token analysis. We added a token perturbation study and an analysis by visual evidence type (Table R2-3 and R2-4; Table 6 and Figure 6 in the revised paper).

For reviewer RfnX:

  1. Fine-tuned single-model baselines. We added comparisons with the original and the supervised fine-tuned Qwen3-VL-8B at matched total inputs of 16 and 96 frames. (Tables R3-1 and R3-2; Table 4 in the revised paper).
  2. Text-communication baseline. We clarify how our controlled text-communication baseline differs from the published MR. Video setup and report an additional experiment to support this comparison (Table R3-3). To make the scope of the comparison clear, we rename the controlled baseline from "MapReduce*" to "Text-Communication" in the revised paper.
  3. Full KV-cache LatentMAS. We added a comparison with LatentMAS using its full KV cache (Table R3-4).
  4. Presentation fixes. We clarify the total frame number in the Table 1 caption, correct the LatentMAS margins in Section 4.2 to match Table 1, and fix the benchmark name in the agent-count ablation (MLVU-Test → LongVideoBench) to match Figure 4 and Table 10 (Appendix G).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.