acceptodds
Under review as a conference paper at ICLR 2027

DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning

Abstract

Reasoning capabilities of multimodal large language models (MLLMs) have improved considerably in recent years. Existing approaches typically rely on explicit chain-of-thought or continuous latent-space trajectories to enhance multi-step reasoning. However, these methods generally assume that an input admits a single latent interpretation and unfold reasoning along a fixed path or under a uniform computation budget. In real-world multimodal settings, visual observations are often subject to occlusion, blur, viewpoint variation, or semantic ambiguity, giving rise to multiple plausible interpretations. A uniform reasoning strategy not only limits the model's ability to explore multiple hypotheses but also incurs high memory usage and rollout cost. We present DLWM (Diverse Latent World Models), a multimodal reasoning framework that combines latent-space reasoning with reinforcement learning. First, we construct a set of diverse latent world hypotheses in continuous latent space, each representing a candidate interpretation of the visual input, and unfold latent reasoning independently on each hypothesis. An orthogonality-based diversity regularizer discourages hypothesis collapse. Second, we formulate the latent reasoning process as a resource-constrained sequential decision problem and introduce a resource-aware reinforcement learning policy that adaptively allocates computation across hypotheses, dynamically deciding whether to expand a reasoning path, prune a hypothesis, merge reasoning paths, or produce the final answer, thereby substantially reducing memory footprint and improving rollout efficiency. Experiments on multiple multimodal reasoning benchmarks demonstrate that DLWM outperforms existing methods by 2–5 points in accuracy while reducing memory usage by 24%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multimodal reasoning, latent reasoning, multi-hypothesis reasoning, adaptive computation, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aSUmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DLWM, a module on top of a frozen Qwen2.5-VL-7B that keeps several latent hypotheses about an ambiguous image. An orthogonality loss keeps the hypotheses distinct, and an RL controller decides at each step whether to expand, stop, merge, or answer under a compute budget. Trained on GQA, VQAv2, and ScienceQA, it reports 2–5 point gains over latent-reasoning baselines and about 24% lower memory than LVR.

Critical strengths
  1. Reasonable motivation and broad ablations. Keeping several interpretations of an ambiguous image and spending compute adaptively is a sensible goal. The ablations cover the number of hypotheses, the compute budget, each controller action, the diversity loss, and an analysis split by question ambiguity.
Critical weaknesses

1. Several reported numbers are inconsistent or impossible. In the per-seed results (Table 9), three separately trained methods vary across seeds in exactly the same way on ScienceQA, MMVP, and VQAv2, so their standard deviations are identical. MMVP has 300 questions, and the paper states that evaluation is deterministic (temperature 0, greedy decoding; Appendix. B.1.2). Under that setup, every evaluation of a given trained model returns the same answers, so each baseline's score should be a multiple of 1/3 point (e.g., 66.0, 66.3, 66.7, same as the numbers reported in the original paper) in single-run or even after "averaging over repeated evaluations" (App. B.1.3). Yet several are not, the single-run baselines in Table 1a (e.g., 57.2, 59.8, 60.5) and most per-seed values in Table 9. The stated architecture (six Transformer blocks at width 4096) cannot fit in the reported parameter budget. The reported peak memory is lower than the memory needed just to hold the frozen backbone's weights. This is consequential because, until these are explained with raw logs, I cannot rely on any of the accuracy or efficiency comparisons.

2. The method is under-specified, and its claimed mechanism is not separated from simple ensembling. The paper never says how the input features are extracted from the frozen model or how an answer is produced. Because each "hypothesis" is a single vector, the "Transformer" reasoning module works like an MLP. There is no frozen-backbone baseline, and the CoT baseline looks badly under-configured (ScienceQA 68.1 vs. about 80+ typically reported for this model). There is also no ensemble or self-consistency control at matched compute. Close prior work on multi-path latent reasoning (e.g., SoftCoT++, Parallel-R1, Mirage) is not cited. This is consequential because the central claim, multi-hypothesis reasoning, cannot be evaluated when the model is undefined and the gain could simply come from ensembling small heads on frozen features.

Questions for authors
  1. Please provide raw per-seed logs for Table 9. Please also explain: the identical seed offsets across methods; the impossible MMVP scores; the per-module parameter counts; and whether peak memory includes the frozen backbone.
  2. How exactly are the input features extracted and the answers produced? How does DLWM compare with a frozen-backbone baseline and an ensemble of independent heads at matched compute?
1 comment

Thank you for checking the numbers closely and for raising the ensemble comparison.

Q1. Can you provide the raw Table 9 logs and explain the seed offsets, MMVP scores, parameter counts, and peak memory?

Seeds. We cannot provide the original per-seed logs or per-question predictions with this response. We recomputed all 12 means and sample standard deviations in Table 7 from the displayed Table 9 values; they agree to the reported precision. All three methods show the same successive changes: +0.2/+0.1 points on VQAv2, +0.2/+0.2 on ScienceQA, and +0.3/+0.2 on MMVP. This arithmetic does not explain why the three methods vary in lockstep. Nor do the three seed averages verify the paired-bootstrap intervals, which depend on paired example-level scores.

MMVP. The official evaluator credits a pair only when both answers are correct. On the full set, one pair changes accuracy by 100/150 ≈ 0.667 points; even scoring the 300 questions individually gives increments of about 0.333. The single-run values 57.2, 59.8, and 60.5 in Table 1a cannot arise under either convention, rounded to one decimal. Nor can a single full run yield 65.8 or 66.1 in Table 9. We described the latter as averages of repeated evaluations, but repeating identical predictions with deterministic grading does not change the score. Without the original denominators and grader outputs, we cannot substantiate the MMVP gain under the protocol we reported.

Parameters. In the supplementary 7B configuration, the six width-4096 reasoning blocks share weights across worlds and steps, not across layers. Their attention projections alone contain 402,653,184 weights. Counting shared weights once gives:

Added moduleParameters
Generator and world embeddings201,363,456
Six-block reasoner1,208,287,232
Controller and heads33,595,397
World scorer8,392,705
Input/output projections and output normalization29,367,296
Total, excluding the backbone1,481,006,086

We withdraw the reported trainable total of 126M (1.8% of the backbone) and the “parameter-matched” label for the K=1 row listed as 126M in Table 12. The 1.481B counts added weights in the released model, not trainable weights measured from the benchmark checkpoints.

Peak memory. Appendix B.1.7 describes bf16 inference on one A100 at batch size eight, using PyTorch’s maximum allocated GPU tensor memory. Resetting the peak counter does not subtract weights already on the GPU. The Qwen checkpoint index lists 16.58 GB (15.45 GiB) of weights alone. The reported 14.2–14.8 GB therefore cannot include the entire bf16 backbone resident on that GPU. We cannot verify which components were resident in the original measurements, and we do not claim the reported 24% difference as a verified end-to-end memory saving.

Q2. How are features extracted and answers produced? How does DLWM compare with frozen Qwen and a compute-matched ensemble?

In the supplementary implementation, frozen Qwen processes the image and question together. We project its final-layer state at the last prompt token from 3584 to 4096 dimensions to obtain z. A shared generator combines z with a learned embedding for each world. Expand updates one world token: its self-attention has only one position to attend to, while the value/output projections and feed-forward layers still transform its state. Attention across worlds occurs in the controller. At each answer position we project each surviving world’s difference from z back to Qwen’s width, add it to the shared decoder state, normalize, and apply the same vocabulary head. The final next-token distribution is a learned weighted average of the world distributions.

Table 6 tests fusion on the same DLWM candidates: on GQA, mean pooling gives 67.8% and learned fusion 69.3%. It does not test independent heads or self-consistency at matched compute. Our ScienceQA score of 68.1% is for generated CoT, not direct answering. Li et al. report 80.76% with direct option scoring and 75.24% with generated CoT in their evaluation. Neither is a result under our protocol. Our comparisons do not establish an advantage over direct frozen Qwen or a matched ensemble.

We will revise the Related Work section to cite SoftCoT++, Parallel-R1, and Mirage and explain how DLWM differs from each. They diversify latent paths, learn parallel text reasoning, and interleave visual latent tokens with text, respectively. Our contribution is the policy that selectively updates, prunes, or merges concurrent multimodal latent states before fusion, not the idea of multiple reasoning paths itself.

Reviewer_xeoTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DLWM, a multimodal reasoning framework that constructs multiple latent hypotheses from a frozen vision-language backbone. A shared reasoning module updates these hypotheses, while an orthogonality regularizer encourages representational diversity. A reinforcement learning controller dynamically selects expansion, pruning, merging, or answering actions, and a learned weighted mixture combines predictions from the retained hypotheses. Experiments on four main benchmarks and six additional zero-shot benchmarks report improved accuracy over the compared latent reasoning methods and approximately 24% lower peak memory than LVR. The paper also includes component ablations, ambiguity analyses, and three-seed comparisons with selected baselines.

Critical strengths
  1. Maintaining several hypotheses and selectively updating or discarding them is a useful approach to visual ambiguity. The controller offers more flexible computation allocation than fixed-length reasoning.

  2. The experiments cover multiple benchmarks and include meaningful ablations and multi-seed comparisons. The ambiguity analysis also provides evidence that the method helps on inputs with multiple plausible interpretations.

Critical weaknesses
  1. The architecture description does not match the reported parameter count. Six standard Transformer blocks with hidden size 4096 would already exceed the claimed 126M total. No compression or sharing across layers is specified to explain this gap, making the implementation and parameter-matched comparisons difficult to assess.

  2. The controller statistics need reconciliation. Table 20 implies only 0.7–0.8 non-Expand actions per example, while Table 1 reports an average of 2.2 active hypotheses from an initial four. Since each Stop or Merge removes only one hypothesis, these numbers do not align under per-example averaging. This weakens the explanation linking adaptive pruning to the reported memory savings.

Questions for authors
  1. Can you provide exact tensor shapes and a module-level parameter breakdown? In particular, how do six 4096-dimensional Transformer blocks fit within 126M trainable parameters, including a 36M controller? Please also explain how latent states are converted into answer predictions.

  2. Can you provide action traces and explicit averaging definitions to reconcile controller steps, hypothesis updates, and active-hypothesis counts, including how terminated episodes are counted? With at most eight actions, each costing no more than one, the default budget of ten cannot be exhausted. What role does this budget play beyond conditioning the policy?

  3. What accuracy does the frozen backbone achieve without the added reasoning modules? How are LVR and the other baselines adapted to the frozen-backbone setting, and which parameters are trained? These details would help assess whether the comparisons preserve the intended capabilities of each method.

1 comment

Thank you for pointing out the discrepancies in the parameter and action counts. We checked both against the supplementary code and the tables in the paper.

Q1. How many parameters does each module add, and how does a latent world produce an answer?

In our supplementary 7B implementation, each Expand runs the selected world through the same six reasoning blocks. Each block has its own fused Q/K/V matrix (12,288 × 4,096) and output matrix (4,096 × 4,096). Across the six blocks, these attention matrices contain 402,653,184 weights, excluding biases and feed-forward layers. This count already accounts for sharing the reasoner across worlds and steps.

The table counts the weights added to Qwen in the supplementary 7B configuration. It excludes the backbone and counts each shared weight once.

ModuleKey dimensionsParameters
World generator and four embeddings8192 → 16384 → 4096; embeddings 4 × 4096201,363,456
Six-block reasonerFused QKV weight 12288 × 4096 per block; FFN 4096 → 16384 → 40961,208,287,232
Controller and heads4096 → 1024; two 8-head blocks, FFN width 409633,595,397
World scorer4096 → 2048 → 18,392,705
Input/output projections and output normalization3584 → 4096 → 358429,367,296
Total added1,481,006,086

We withdraw the reported trainable total of 126M (1.8% of the backbone) and the “parameter-matched” label for the K=1 row listed as 126M in Table 12. In the supplementary 7B configuration, the controller has 33.60M parameters rather than the 36M we reported, and Stage 2 updates only the controller. The 1.481B total counts the parameters in the added modules of our released 7B configuration; it is not a measured trainable total for the Table 12 runs.

To produce an answer, the code takes each active world's final state, subtracts the initial image–question vector, and projects that difference to Qwen's hidden dimension. It adds the result to the shared decoder state at the answer position. After normalization, Qwen's vocabulary head gives a next-token distribution for each world. The final prediction is a weighted average of those distributions, with learned weights.

Q2. How do you count steps, updates, and active worlds? Can the default budget run out?

One Expand updates one world. A Stop removes one world; a Merge replaces two with one. Our supplementary evaluator counts the active worlds just before each decision, including Answer, and ignores padding after the episode ends. We first average these counts within each question, then average the question-level means, giving each question equal weight. The separate num_steps counter excludes Answer, whereas Algorithm 1 counts it as a step.

Our per-question count starts at four and never increases, so the average during an episode cannot be lower than its final count. On GQA, Table 20 reports 4.9 steps and 4.1 updates per question. That leaves at most 0.8 actions that could remove a world, giving a lower bound of about 3.2 active worlds on average (4 − 0.8). The same calculation gives about 3.2 on VQAv2 and ScienceQA, and 3.3 on MMVP. These are lower bounds from the reported action counts, not new measurements. We should not have used the 2.2 in Table 1b to explain the memory difference: it is not supported by the Table 20 counts under our per-question definition. We cannot provide the requested original traces or a corrected measured average in this response.

The controller sees how much of its budget is left. But with the default budget of 10, even eight actions at the maximum cost of one leave two units unused. The rollout ends when the controller selects Answer or reaches the step limit, not because it runs out of budget.

Q3. How well does frozen Qwen do on its own, and what was trained for LVR and the other baselines?

We did not report a direct-answer score for frozen Qwen under our evaluation settings. The 68.1% we reported on ScienceQA in Table 1a is for Qwen prompted to use chain-of-thought, not Qwen answering directly.

Appendix B.1 describes the baseline setup: keep Qwen's vision tower and language model frozen, use the same preprocessing and scoring, and train each method's added modules, where applicable, with the same data, optimizer, batch size, and number of steps. Method-specific hyperparameters follow the released settings where available. One detail matters for LVR: the supplied 7B training scripts set freeze_llm=False, which allows the language model to update. Our released LVR scripts thus differ from the protocol we described in Appendix B.1. We cannot confirm exactly which weights were updated in the Table 1a runs for LVR and the other adapted baselines.

Reviewer_XmcyOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

In current multimodal reasoning, existing methods usually rely on a single reasoning trajectory or a uniform computation budget, which might not be suitable for real-world multimodal settings. To address such problems, this paper proposes DLWM (Diverse Latent World Models), a multimodal reasoning framework that combines multi-hypothesis latent modeling with resource-aware reinforcement learning. The framework consists of two parts: 1. the Latent World Generator creates diverse world hypotheses in continuous space based on the input, with orthogonal regularization to reduce representational collapse. 2. a resource-aware RL controller dynamically allocates computation resources across latent reasoning trajectories. The proposed method has been evaluated on multiple multimodal reasoning benchmarks and shows consistent improvements over other methods.

Critical strengths
  1. This paper is well-motivated. It is reasonable to have multiple hypotheses to handle the ambiguous visual input. Moreover, using an RL controller to dynamically allocate computation resources is also an interesting approach.

  2. Extensive experiments (including various ablation studies) have been done to support the idea, and the method also shows consistent improvements across multiple multimodal reasoning benchmarks compared to other methods. The breadth of these evaluations makes the overall effectiveness of the framework convincing.

Critical weaknesses
  1. The benefits of RL-based computation allocation are not sufficiently justified. Although the RL controller outperforms fixed-depth reasoning, the paper does not compare it with simpler adaptive strategies. For example, Table 16 shows that merged worlds have substantially higher cosine similarity than unmerged worlds. It would be helpful to see whether a similarity-based merging strategy can achieve comparable performance under matched compute.
Questions for authors
  1. How are the latent states converted into final answers? Does the six-block Transformer process a single token or multiple tokens at each time step? Is the frozen MLLM decoder reused when generating the final answers?

  2. How does the RL controller compare with simpler adaptive strategies, such as confidence-based expansion or similarity-based merging strategies?

  3. Since all world representations are generated from the same z, how do the authors ensure that they capture different possible interpretations of the image rather than simply different visual features (like complementary information)?

  4. How are the semantic interpretations of individual latent worlds determined in the qualitative analysis?

  5. How often does the Stop action remove hypotheses that would produce incorrect vs. correct answers?

  6. Since the world-index embedding is used in the RL controller, how is the world-index embedding of the resulting merged world determined? Does it inherit one of the original indices or receive a new embedding?

1 comment

Thank you for your questions about what the controller learns and what an individual world represents. We address them in order.

Q1. How are latent states converted into answers? Does the six-block Transformer process one token or multiple tokens, and is the frozen decoder reused?

Yes, the worlds share the frozen Qwen decoder. The reasoner and controller see different inputs:

ModuleInputOperation
Six-block reasonerOne latent token for the world selected by ExpandUpdates that world's state.
ControllerOne token per active world plus a budget tokenAttends across worlds and selects the next action.

At each answer position, the supplementary implementation adds a projected world-specific update to Qwen's decoder state, applies normalization, and reads next-token probabilities through Qwen's vocabulary head. The final distribution is a learned weighted mixture over the active worlds (Eqs. 5–6). A separate full decoder pass per world is not needed.

Q2. How does RL compare with confidence-based expansion or similarity-based merging?

Cosine similarity selects which pair is eligible to merge; the policy decides whether to merge it now, expand a world, prune one, or answer. Likewise, the policy selects a world to expand rather than applying a fixed confidence threshold. This explains why the merged pairs in Table 16 have high similarity without attributing the pair selection itself to RL. Table 12 reports 69.3% on GQA for full DLWM versus 67.9% for four worlds updated to a fixed depth. The 1.4-point difference is against fixed-depth reasoning; it does not establish an advantage over confidence- or similarity-based control at matched compute.

Q3. If all worlds start from the same input representation, what distinguishes alternative interpretations from complementary features?

We call each world a hypothesis because it maintains a separate state and produces its own candidate answer before fusion. The worlds share the image–question representation but start with different learned world embeddings. Appendix D.5 reports:

DiagnosticWith orthogonality penaltyWithout penalty
Pairwise cosine similarity of world states0.180.47
Examples on which all worlds give the same answer54.2%71.6%

The penalty thus affects both the representations and what they predict. This supports alternative predictive hypotheses; complementary visual features could also produce that diversity. The experiment does not assign a human-interpretable meaning to each state.

Q4. How are the semantic interpretations of individual worlds determined in the qualitative analysis?

For “Is the bench facing the lake?”, Appendices F and I report that worlds 1 and 3 answer Yes, while worlds 2 and 4 answer No. The controller expands the former and merges the latter; the fused answer is Yes. The case illustrates how DLWM handles competing answers. Appendix F states the backrest explanation too strongly: confusing the backrest edge with the seat is a possible explanation for the No answers, not a verified attribution to those worlds.

Q5. How often does Stop remove a hypothesis that would give an incorrect versus a correct answer?

Table 14 measures the effect on the final answer: excluding stopped worlds from fusion gives 69.3% on GQA, versus 68.4% when they are included. The 0.9-point difference supports excluding them in this comparison. Correctness at the moment of pruning is a different statistic: it requires scoring the discarded world's own answer before each Stop. If “would give” means after further reasoning, it also requires continuing that branch. Table 14 does not yield either event rate.

Q6. After a merge, does the resulting world inherit an index or receive a new embedding?

It inherits the first parent's index. In the supplementary implementation, Merge(i,j) stores the merged state in slot i and removes slot j. At the next decision the controller looks up slot i's original index embedding. The latent states are combined; their index embeddings are not averaged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.