1. The advantage of learned routing over simpler alternatives is not established at comparable computational cost.
The methodological contribution builds on established ideas: FlexiViT supports multiple patch sizes within one backbone, AdaPatch adapts patch size according to image properties, and APT explores different patch sizes within an image. MoP's distinguishing component is its use of backbone-specific answer losses to supervise an image-level router, together with its adaptation and empirical analysis in MLLMs. This is a sensible design, but its incremental value needs a more controlled assessment.
Table 8 is useful because it compares MoP and the AdaPatch routing strategy on the same adapted backbone. However, it reports accuracy without the corresponding token counts or computational costs. The average accuracy advantage of 82.6 versus 79.9 therefore does not establish a better accuracy–cost tradeoff: the methods may allocate different amounts of computation.
A comparison at matched average FLOPs, or a small set of accuracy–FLOPs operating points, would address this concern. A complementary baseline would preserve the learned router's patch-selection proportions while randomly reassigning those choices across examples. Because image dimensions can vary, this comparison should also report and control actual computational cost rather than assuming that equal routing proportions imply equal FLOPs.
2. The available ablations do not fully explain the gains of the main model.
Figure 5 shows that most of the accuracy improvement over the original backbone is already present after multi-patch adaptation. Relative to the best fixed branch, the router adds 0.5, 0.1, and 0.1 percentage points on RealWorldQA, AI2D, and MMStar, respectively, and loses 0.4 points on ChartQA. These small accuracy differences do not negate the router's value, since maintaining accuracy at lower cost is itself a useful result. They do, however, make it important to separate the benefits of backbone adaptation from those of routing.
The relevant experiments use different training recipes. The main model uses 4.3M pretraining and 2.9M instruction-tuning samples; Figure 5 uses 4.558M pretraining and 3M OCR-oriented samples; and Table 2 uses 4.558M pretraining and 858K SFT samples. These differences are disclosed in the appendix, and the individual comparisons can still be informative. However, their results cannot be combined directly into a complete attribution of the main model's gains. Table 1 also mixes official and self-evaluated baseline scores, which the authors appropriately distinguish from controlled comparisons.
The most useful addition would be one consolidated comparison under the main model's training recipe and evaluation protocol: continued training of the original architecture, fixed branches of the adapted backbone, and heuristic and learned routing, with both accuracy and cost reported. The existing two-backbone evaluation is a strength; resolving this attribution issue is more important than adding a broad range of architectures or model sizes.
3. The strength and consistency of the instance-level routing evidence need clarification.
Table 3 provides positive evidence for instance-level selection: the router exceeds uniform random selection and the best fixed patch size on all three confusing subsets. However, its margins over the best fixed choice correspond to approximately four additional correct answers on AI2D, one on RealWorldQA, and two on MMStar. These are small absolute differences. Paired uncertainty estimates or tests based on per-example correctness would help establish their robustness. Deterministic decoding removes sampling variability at inference, but does not resolve uncertainty due to the finite evaluation set or router training.
There is also a specific correspondence issue between Figure 5 and Table 3. Under the setup described in Appendix A.1.4, routing cannot change correctness on examples where all fixed branches agree. If both results use the same checkpoint, evaluation set, and protocol, the net gain in correct answers over a fixed branch must therefore be identical on the confusing subset and the full dataset. For RealWorldQA, Table 3 implies approximately one additional correct answer over p=8. On the 765 examples indicated in Figure 3, this would yield approximately 0.13 percentage points of improvement. Figure 5 instead reports 73.1 to 73.6, a gain of 0.5 points.
This may reflect different experimental settings rather than an error, but the paper should identify which checkpoints and protocols produced these results and reconcile the numbers if they are intended to describe the same setting.
4. The task-dependent interpretation goes beyond what image-only routing demonstrates.
Equations (3)–(6) define a router conditioned only on the image, whereas its supervision depends on the image, question, and target answer. This is a valid learning setup: an image-only router can learn patch preferences averaged over the questions associated with similar images. However, it cannot change its decision when the question changes for the same image.
Consequently, dataset-dependent routing distributions do not establish adaptation to the requirements of the current question. Appendix A.3.2 further states that the router training set retains only the shortest question per image and filters questions longer than 40 tokens, so this training setup does not directly examine competing granularity requirements across questions about the same image.
The paper should distinguish image-dependent adaptation from question-dependent adaptation. A same-image, multiple-question analysis would test whether this distinction matters in practice. A question-conditioned router would be an informative extension, but is not necessary if the claims are appropriately scoped.