- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes a routing method to use different image editing frameworks. The proposed method achieves better performance than a strong proprietary baseline, Nano Banana Pro, with a small open-source model.
Critical strengths
- The analysis in Section 3 reveals how different methods can cooperate to improve editing quality.
- The experiment demonstrated improvement on multiple benchmarks.
Critical weaknesses
- The paper is incomplete with multiple errors:
- In Appendix A, experiment details are not provided, but there is a placeholder message saying "[Pending: Confirm the Banana transfer model and all API identifiers, all baseline editor/planner roles, the router and captioner versions, and the Qwen3.8 planner checkpoint.]"
- On Figure 2, Step 2 says "Change the blue trash can to green", which is unrelated to the image.
- In the last row of Figure 3, there is no highlighted red outline in the original image, despite the prompt saying so.
- The first stage was called query profiling in Figure 2 and Section 4.2, but is called captioning later.
- The reported performance doesn't match the official numbers:
- On PICABench, the performance of Nano Banana Pro is previously reported as 66.16 by the PICABench paper, but the performance is 57.89 in this paper.
- On VIBE, the performance of Nano Banana Pro is reported as 65.15 by the official VIBE-Benchmark repository, but the performance is 76.51 in this paper. Also, the performance of FLUX.2-klein is 44.94, way higher than the official performance of the stronger model FLUX.2-dev (30.95). This may arise due to the mismatch of scoring models. The paper uses Qwen3.8-27B, but the official benchmarks used various different scoring models.
- VIBE is an image editing benchmark with 1,034 samples, but Table 4 reported only 900 samples. Similarly, ImgEdit-Basic has 734 samples and ImgEdit-UGE has 47 samples according to the original paper, but Table 4 reported 737 and 45, respectively.
- Since the router component is the primary contribution of the paper, the experiment scale is currently too limited, as the only configuration is with 3 certain methods to route to. The effectiveness of the router should be demonstrated with more/fewer number of methods, or an alternative set of 3 methods. For example, the existing literature has a large number of localized editing methods, so it would help if the paper can further demonstrate that routing works using different methods of localized editing.
- Overall, using a harness on FLUX.2-klein to achieve better performance than Nano Banana Pro is a bold claim. It seems that the current routing scheme (rewrite, local, or spatial) is unable to close fundamental gaps between the two models, such as in text editing (see the "Text" column of Table 3 and Table 4 in https://arxiv.org/pdf/2608.14546). It would be great if this can be acknowledged as a limitation.
Questions for authors
- Could you provide an analysis about why the reported performances significantly differ from the official numbers?
- Could you clarify why the size of the benchmarks differ from official numbers?
- Could you clarify what is "a Banana editor" in Section 5.5? Its performance of 3.150 in Table 6 (Banana Direct) does not match the performance of 3.568 in Table 3 (Nano Banana Pro). Is it a different model?