acceptodds
Under review as a conference paper at ICLR 2027

MACRO: Advancing Multi-Reference Image Generation with Structured Long-Context Data

Abstract

Generating images conditioned on multiple visual references is critical for real-world applications such as multi-subject composition, narrative illustration, and novel view synthesis, yet current models suffer from severe performance degradation as the number of input references grows. We identify the root cause as a fundamental data bottleneck: existing datasets are dominated by single- or few-reference pairs and lack the structured, long-context supervision needed to learn dense inter-reference dependencies. To address this, we introduce MacroData, a large-scale dataset of 400K samples, each containing up to 10 reference images, systematically organized across four complementary dimensions—Customization, Illustration, Spatial reasoning, and Temporal dynamics—to provide comprehensive coverage of the multi-reference generation space. Recognizing the concurrent absence of standardized evaluation protocols, we further propose MacroBench, a benchmark of 4,000 samples that assesses generative coherence across graded task dimensions and input scales. Extensive experiments show that fine-tuning on MacroData yields substantial improvements in multi-reference generation, and ablation studies further reveal synergistic benefits of cross-task co-training and effective strategies for handling long-context complexity. Our website and data are available for anonymous review.

Primary area:
datasets and benchmarks
Keywords:
Multi-reference image generation, Dataset & Benchmark, In-context generation.

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_4RVJOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The generation quality of open models degrades substantially as the number of reference images increases. This paper attributes the problem to a lack of training data containing many reference images and makes two contributions. First, it constructs MacroData, a dataset of 400K samples with up to 10 reference images each, covering four tasks: Customization, Illustration, Spatial, and Temporal. Second, it proposes MacroBench, a benchmark that evaluates performance as a function of the number of reference images. The authors fine-tune Bagel, OmniGen2, and Qwen-Image-Edit on MacroData and report improved performance on MacroBench and OmniContext.

Critical strengths
  • The dataset is large-scale and rich in samples with many reference images. Beyond simple image composition, it unifies illustration generation, novel view synthesis, and next-frame prediction within a single framework, making it a valuable resource for the research community.
  • Performance gains are observed consistently across all three base models. The paper also verifies agreement among multiple VLM judges and the correlation between the judge and human ratings. The ablations on data scale, the proportion of reference counts, and task combinations are also informative.
Critical weaknesses

The critical ones are Weaknesses 1 and 4.

  1. The improvements with many reference images are not validated on an external benchmark independent of the training data. MacroBench uses the same pipeline and prompt format as the training data (MacroData). Although the samples do not overlap, the two come from the same distribution, so models trained on MacroData may have an advantage on MacroBench. Meanwhile, OmniContext, used as the external benchmark, contains at most three reference images, as the paper itself acknowledges. This is insufficient to objectively support the claim that performance improves with more reference images. Results on a multi-reference benchmark such as MultiBanana [2] would be desirable.

  2. Differences from prior work and the novelty of the contribution are not sufficiently discussed. The paper claims that existing benchmarks are limited to customization and editing and include only a few reference images. However, several multi-reference benchmarks already exist [1, 2, 3, 4, 5].

  3. For some tasks, it is questionable whether the intended capabilities are actually being measured.

    • Spatial: The fine-tuned Bagel is reported to outperform Nano Banana Pro. However, the prompts use a dataset-specific spatial convention (e.g., From above, the sequence Front -> Left -> Back -> Right follows a clockwise order), so the score could improve simply by memorizing this convention.
    • Temporal: The task is framed as next-frame prediction, but the actual evaluation criteria are fidelity to the instruction (ISCS) and natural continuity with the previous frames (CCS). If the instruction itself contains specific content of the ground-truth frame, the task does not measure the model's predictive ability.
    • Illustration: The paper itself describes this as a text-centric task (Illustration-only wins its task because it is more text-heavy and semantically distinct), and it is unclear to what extent the reference images are actually used during generation.
    • Furthermore, an overall score computed as a simple average over tasks of such different natures is difficult to interpret.
  4. Verification of training data quality is insufficient. Nano Banana Pro is used to generate supervision for the Customization task, but its output quality is questionable. Even closed models have been reported to degrade as the number of reference images increases [2], and in Table 14 of this paper, Nano Banana Pro's performance likewise declines as the number of references grows. It has also been pointed out that even closed models can introduce unintended modifications and attribute inconsistencies [1]. Nevertheless, the paper reports neither the pass rate of data filtering nor the results of any human quality inspection. If the authors claim that data quality is adequately ensured, they should state the verification procedure and results explicitly; if no such verification was performed, they should conduct one.

References

[1] Xia et al. DreamOmni2: Multimodal Instruction-based Generation and Editing, CVPR 2026.
[2] Oshima et al. MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation, CVPR 2026.
[3] Peng et al. DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation, ICLR 2025.
[4] Jang et al. Identity Decoupling for Multi-Subject Personalization of Text-to-Image Models, NeurIPS 2024.
[5] Zhou et al. OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation, CVPR 2025.

Questions for authors
  1. Can you provide human evaluation results for the fine-tuned models (e.g., pairwise comparisons of outputs before and after fine-tuning)?
  2. How was the quality of the training data verified (e.g., filtering pass rates, whether human visual inspection was conducted, and the agreement rate)?
  3. Is the performance improvement maintained on multi-reference image generation benchmarks whose format differs from the one used in the paper (e.g., MultiBanana [2])?

If the authors address these questions and the weaknesses shown above in their manuscript, then I will consider raising the score.

9 comments

We thank the reviewer for the thorough assessment and constructive suggestions. We agree that an external multi-reference evaluation and a direct audit of data quality are essential, and that our characterization of prior benchmarks was too broad. Below we address each point in order.


W1 / Q3 — External evaluation on MultiBanana

Fine-tuning on MacroData improves both backbones on MultiBanana, and the gain is larger with more references: +1.99 overall and +2.37 at 6–8 references on BAGEL, and +0.75 overall and +0.85 at 6–8 references on OmniGen2.

We evaluated base and fine-tuned BAGEL and OmniGen2 on all 1,247 tasks with 3–8 references in the public MultiBanana release. One task with an empty prompt is excluded for all models. Scoring follows the official protocol with the Qwen3-VL-8B-Instruct judge, five 10-point criteria and the 3:3:1:1:1 weighted total. Models in the same family share inputs, seeds and settings. Official OmniGen2 accepts five references, so for 6–8 references we extend its image-index embedding to ten rows as in Appendix C.2. The tables give the weighted total by task type and reference count in the layout of the MultiBanana paper (Local, Global and Back. are X−1 objects with a local edit, a global edit or a background, and Object is X objects).

Model3·Local3·Global3·Back.3·Object4·Local4·Global4·Back.4·Object5·Local5·Global5·Back.5·ObjectAvg
BAGEL3.944.284.083.664.033.773.263.294.113.202.602.583.57
BAGEL + MACRO5.555.765.356.145.105.624.865.254.915.034.044.675.19
Δ+1.61+1.48+1.27+2.48+1.07+1.85+1.60+1.96+0.79+1.83+1.43+2.09+1.62
OmniGen24.844.514.584.584.263.533.173.333.832.692.622.433.70
OmniGen2 + MACRO4.764.965.584.874.324.353.914.264.503.553.603.584.35
Δ−0.08+0.44+1.00+0.29+0.06+0.83+0.74+0.92+0.67+0.87+0.98+1.16+0.66
Model6·Local6·Global6·Back.6·Object7·Local7·Global7·Back.7·Object8·Local8·Global8·Back.8·ObjectAvg
BAGEL1.721.521.391.261.501.371.241.281.531.351.171.111.37
BAGEL + MACRO3.774.073.573.803.753.923.833.613.903.663.493.443.74
Δ+2.05+2.55+2.18+2.54+2.26+2.55+2.59+2.33+2.38+2.32+2.31+2.33+2.37
OmniGen23.152.221.962.082.931.931.942.192.201.671.811.742.15
OmniGen2 + MACRO3.463.432.953.352.882.492.732.913.532.992.522.803.00
Δ+0.30+1.22+1.00+1.27−0.05+0.56+0.79+0.72+1.34+1.31+0.72+1.06+0.85

BAGEL improves in all 24 cells and OmniGen2 in 22 of 24, with small dips on 3·Local (−0.08) and 7·Local (−0.05). All five criteria improve for both models, by +1.05 to +2.82 on BAGEL and +0.44 to +0.93 on OmniGen2, and every 95% bootstrap interval lies above zero. Our base-model scores closely match those reported by the MultiBanana authors with the same judge [2, supplementary Table 6] and by AutoRef (arXiv:2609.35530).


W2 — Relation to prior multi-reference work

We will revise the statement that existing benchmarks are limited to customization. Prior works either evaluate multi-reference generation without training data or target one sub-problem such as few-subject personalization. MACRO studies scaling to ten references across four capabilities (identity binding, interleaved illustration, 3D spatial reasoning and temporal continuation), and makes this regime trainable and measurable with a scalable 400K training set (52.5% with six or more references) and a benchmark stratified by task and reference count. We will add the following comparison to the related work section:

  • DreamOmni2 [1] proposes a method and a benchmark for instruction-based editing and generation with concrete and abstract attributes from a few references.
  • MultiBanana [2] evaluates multi-reference subject and attribute composition with up to eight references and provides no training data. MACRO contributes a scalable data pipeline that produces 400K training samples over four capabilities, and W1 shows that this data transfers to MultiBanana, especially at 6–8 references.
  • DreamBench++ [3] is a human-aligned benchmark for personalizing a single subject from a single reference.
  • MuDI [4] reduces identity mixing in multi-subject personalization at the model and inference level. MACRO addresses identity binding through large-scale multi-reference supervision.
  • OpenING [5] evaluates interleaved text-and-image generation over 56 tasks with 5,400 human-curated instances. Because it relies on manual annotation, it serves as an evaluation suite, while MACRO provides scalable training supervision for dense conditioning on up to ten references.

W3a — Does Spatial only reflect a memorized direction convention?

The fine-tuned model places new views relative to the references rather than recalling named directions. On real CO3D photographs with continuous camera specifications unseen in training, fine-tuning reduces the mean viewpoint error by 16.0° and raises the share of outputs within 30° of the target from 28% to 46%.

Each case gives three reference views of an object and a target camera written as a continuous orbit from the first view (e.g., "29.6° to the right, 9.7° above, at 0.90 × the distance"), without the ten direction names or the clockwise order. We selected 94 objects from 20 categories with fixed geometric criteria before generation and ran both models with identical inputs and seeds. We measure the rotation error to the true target view with VGGT poses, which are accurate to within 2° on the real photographs.

94 objects, 20 categoriesMean error ↓Median error ↓≤ 15°≤ 30°≤ 45°Better than BAGEL
BAGEL55.9°43.4°6%28%54%—
BAGEL + MACRO39.9°31.6°12%46%64%60 / 94
Δ [95% CI]−16.0° [−24.1, −8.8]−11.8°

W3b — What the Temporal task and metrics measure

The review describes the two metrics with their roles swapped. ISCS (Image Sequence Consistency) scores whether identity, clothing, style and background are kept from the preceding frames. CCS (Context & Action Logic) scores whether the instructed event happens and the motion follows from the last frame. A model that ignores the frames fails on both.

The instruction is written from the full clip and can describe the target event, so the task is text-guided next-keyframe generation. A clip can continue in many valid ways, and the text specifies the intended one. The task remains hard, as base BAGEL scores only 4.27/10. To see how much of the gain depends on the description revealing the target, we rewrote every instruction from the input frames only and reran 100 cases:

N = 100ISCS (input-only)CCS (input-only)Score (input-only)ISCS (full)CCS (full)Score (full)
BAGEL3.794.233.954.024.634.27
BAGEL + MACRO6.447.086.717.117.727.36
Δ [95% CI]+2.65 [1.97, 3.30]+2.85 [2.14, 3.58]+2.76 [2.12, 3.42]+3.09 [2.43, 3.75]+3.09 [2.37, 3.79]+3.09 [2.44, 3.76]

Without target information, fine-tuning still gains +2.76, close to the +3.09 with full descriptions, and the target-aware description adds about 0.65 for the fine-tuned model. We will rename the task to text-guided next-keyframe generation and report both settings.


W3c — Are the references used in Illustration?

The model relies on the reference images. Removing them lowers the score by 1.49, and replacing them with images from another article lowers it by 2.47.

The input is an article passage with its preceding images, and the target is the next image. The references carry details the text leaves open, such as the look of products or people and the page style. On 200 MacroBench cases (50 per bin, sampled deterministically) we ran BAGEL + MACRO with the correct references, the same number of references from a different article, and no references. All outputs are scored against the true article and references with the paper's TCS (text) and ICS (reference) rubric.

BAGEL + MACRO, N = 2001–34–56–78–10TCSICSScore, all
Correct references6.465.576.576.386.376.176.24
Other article's references4.602.623.624.253.983.653.77
No references4.844.165.374.634.954.614.75
Δ correct − none+1.62+1.40+1.20+1.75+1.43+1.56+1.49 [1.04, 1.95]
Δ correct − mismatched+1.86+2.94+2.95+2.13+2.38+2.52+2.47 [1.96, 2.97]

Mismatched references score below no references, so wrong references mislead the model. The drop holds in all four bins and in both TCS and ICS, and excluding the 10 cases whose target duplicates a reference gives +1.56 and +2.48.


W3d — Averaging across tasks of different nature

All MacroBench tasks share the same size and sampling rule (250 cases per task and reference count), so the plain average is an equal-weight summary. It agrees with Tables 14–17, where fine-tuning improves every backbone on every task:

Avg, Tables 14–17CustomizationIllustrationSpatialTemporal
BAGEL3.95 → 8.02 (+4.07)4.42 → 5.62 (+1.20)0.60 → 3.40 (+2.80)3.14 → 5.81 (+2.67)
OmniGen23.74 → 5.87 (+2.13)4.26 → 4.46 (+0.20)0.84 → 1.60 (+0.76)2.71 → 4.51 (+1.80)
Qwen-Image-Edit3.67 → 6.50 (+2.83)3.43 → 4.26 (+0.83)1.10 → 2.68 (+1.58)2.58 → 4.00 (+1.42)

Dropping any one task leaves the Table 1 ranking essentially unchanged, with Bagel + MACRO third behind the two closed-source models. For comparing general capability, weighting tasks by difficulty or score variance may suit better, and we will discuss these alternatives in the paper.

W4a / Q2a — Construction pipeline, filtering and pass rates

Each task has a filter matched to its ground truth, and every reference must be needed to predict the target. Customization, the only task whose targets are generated, loses the most candidates as references increase (pass rate 95.7% at 1–3 references and 53.5% at 8–10). Stage counts, using candidates that reached the final filter as the denominator:

TaskSource poolCandidatesPass quality rulePass rateSelectedKept of candidates
Customization189,692 images151,180107,54671.1%100,00066.1%
Illustration48.1M image–text samples136,885125,52991.7%100,00073.1%
Spatial127,454 assets120,000rule-based only—100,00083.3%
Temporal1M sampled videos138,244113,67282.2%100,00072.3%
  • Customization. Human, object, scene, clothing and style images (OpenSubject, MVImgNet, DL3DV, Vibrant Clothes, WikiArt) are cleaned and aesthetically filtered. Sources are mixed at 8:6:3:2:1, an LLM rejects implausible combinations and writes the instruction, and Nano Banana Pro generates the target. Gemini-3-Flash scores the output against each reference and the instruction, and a candidate passes only if every score is ≥ 6.
  • Illustration. A VLM selects anchor images in OmniCorpus-CC as targets, with the preceding text and images as context. Gemini-3-Pro rewrites the context, removes earlier images that do not contribute to the target, and scores training suitability and guidance. A candidate passes if both scores are ≥ 6 and 1–10 contributing references remain.
  • Spatial. Untextured or low-saturation Objaverse assets and non-standard panoramas from DIT360, Pano360 and Polyhaven are removed. Input and target views are drawn from ten canonical directions, keeping only combinations with visual overlap.
  • Temporal. OmniCorpus-YT videos are split into shots with TransNetV2, keyframes of the same scene are grouped by DINOv2 similarity, and Gemini-3-Flash scores temporal coherence (≥ 6 passes). The last frame becomes the target.

Passing candidates are then sampled evenly across reference bins. Pass rates by bin:

ReferencesCustomizationIllustrationTemporal
1–395.7%97.3%60.2%
4–586.6%95.1%89.0%
6–772.3%93.1%92.5%
8–1053.5%85.0%94.7%

Most Customization rejections at high counts fail per-reference consistency, mirroring the drop of Nano Banana Pro in Table 14. Temporal shows the opposite trend because short clips often change little between shots and are filtered out for weak temporal progression.


W4b / Q2b — Human inspection of training-data quality

A blind human audit agrees with the automatic filter. Only 3 of 120 accepted samples (2.5%, 95% Wilson interval 0.9–7.1%) were judged bad, and overall agreement is 92.9% (Cohen κ = 0.86). We sampled 240 candidates deterministically from fixed shards, 10 accepted and 10 rejected for each of Customization, Illustration and Temporal in each reference bin. The annotator saw the references, instruction and target, but not the filter's decision. When scaling up, the error that matters is a bad sample the filter keeps (LLM pass, human fail), since it enters training. A good sample the filter rejects only reduces data volume.

TaskLLM pass · human passLLM pass · human failLLM fail · human passLLM fail · human failAccepted but badAgreementκ
Customization3733377.5%92.5%0.85
Illustration4002380.0%97.5%0.95
Temporal4009310.0%88.8%0.77
All1173141062.5%92.9%0.86

Of the 17 disagreements, 14 are samples the filter rejected and the annotator accepted, 9 of them Temporal sequences with limited motion. Spatial is not audited because each target is rendered from a 3D asset or projected from a panorama under known camera parameters, so it is the exact ground truth and the unique solution for the given pose. Spatial data is still filtered at the asset and view level (W4a). We will add the audit to the appendix.


Q1 — Pairwise human comparison before and after fine-tuning

Annotators prefer the fine-tuned output for every backbone, in 76–78% of non-tied judgments, and the preference grows with the number of references. Three annotators compared base and fine-tuned outputs, shown in random left/right order without model names, on 40 MacroBench cases per backbone (10 per task, 2 / 2 / 3 / 3 across reference bins, sampled deterministically) and chose the better one or a tie (360 judgments).

ModelFine-tuned betterTieBase betterWin rate w/o ties
Bagel67322176.1%
OmniGen260431777.9%
Qwen70302077.8%
All1971055877.3%
ReferencesVotesFine-tuned betterTieBase betterWin rate w/o ties
1–37229311270.7%
4–57241181375.9%
6–710860321678.9%
8–1010867241779.8%
Reviewer_z3mKOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MacroData, a dataset of 400K samples for conditional image generation. It covers tasks including customization, illustration, novel-view synthesis, and temporal frame prediction. The authors also introduce a 4000-sample benchmark and show consistent improvements by fine-tuning three baselines with proposed training data.

Critical strengths
  • This dataset cover very challenging and useful tasks in conditional image generation. Specifically, the 4 diverse tasks including Customization, Illustration, Spatial and Temporal are addressing the specific different challenges in this field. So it is good to unify them into one dataset.

  • The consistent improvement of baselines on MacroBench (from 3.03 to 5.71 for Bagel, 2.89 to 4.11 for OmniGen2, and 2.69 to 4.36 for Qwen) demonstrates the effectiveness of the constructed training data.

  • The evaluation is fairly comprehensive. It includes comparisons with strong closed-source models, other training datasets, and human annotations. I also appreciate the failure examples in Appendix D, which show remaining problems with missing subjects, viewpoint changes, etc, which are seldom being discussed in recent works.

Critical weaknesses
  • The reference-count experiments does not control single variable (context length) from other factors. Specifically, the input resolution decreases from 1024×1024 for 1–3 references to 512×512 for 6–10 references. The degradation is not necessarily caused by processing longer contexts. Can you evaluate the same examples with increasing numbers of references while controlling input resolution?

  • It is not clear that the temporal task see the final frame before generate a sequence summary. How does the performance change when the summary is based purely on preceding frames? One contradiction in the manuscript -- Appendix B.1 says the Temporal judge receives the ground-truth target, but Figure 26 lists only the preceding frames and generated output.

  • Lack of ablation study on single task training and multi-task training.

Questions for authors
  • How does model with limited reference capacity handle 6-10 inputs? Can you report the more details about training / fine-tuning configurations and how to use the data for training?
4 comments

We thank the reviewer for the careful reading and constructive questions. As the reviewer points out, Appendix B.1 misstates the inputs to the Temporal judge, and we will fix this error (W2b). Below we highlight the resolution control (Table 11, W1) and single-task ablations (Table 3, W3) already in the manuscript, provide a new Temporal experiment using summaries generated solely from the preceding frames (W2a), and detail how 6–10 references and multi-task training are configured (Q1a, Q1b).


W1 — Resolution versus number of references

Table 11 (Appendix C.3) reports this control, and it shows that the drop in performance comes from the larger number of reference images and not from the lower resolution per image. We evaluate the same MacroBench instances at every reference count once under our dynamic input resolution schedule (1024 / 768 / 512 px for 1–3 / 4–5 / 6–10 references) and once with every reference image fixed at 1024 × 1024, holding the output resolution, random seed, and judge constant. Averaged across all four tasks:

ModelInput resolution1–34–56–78–10Avg
BAGELdynamic (1024 / 768 / 512)4.022.682.391.962.76
BAGELfixed 10244.022.752.372.012.79
Δ fixed − dynamic0.00+0.07−0.02+0.05+0.03
BAGEL + MACROdynamic (1024 / 768 / 512)6.125.935.474.845.59
BAGEL + MACROfixed 10246.125.845.484.785.56
Δ fixed − dynamic0.00−0.09+0.01−0.06−0.03

Fixing the input resolution at 1024 px changes the task-averaged score by at most 0.09 in any bin, and the decline from 1–3 to 8–10 references remains essentially the same: 2.06 versus 2.01 for BAGEL, and 1.28 versus 1.34 after fine-tuning. We additionally checked statistical significance: across all 24 model-by-task-by-bin comparisons where the two settings differ (4–5, 6–7 and 8–10 references), two-sided t-tests (N = 250 per bin) find no significant difference (all p > 0.05). We will add the significance test to the paper alongside Table 11.


W2a — Does the Temporal summary see the final frame?

Yes, the summary sees the final frame. In MacroData it is written from the complete clip, including the last frame, which then becomes the generation target. A controlled comparison shows that the fine-tuning gain stays large when the summary sees only the input frames (+2.76, compared with +3.09 for full-clip summaries). The full-clip design is intentional: the task is text-guided next-keyframe generation, in which the text specifies which event happens next and the model must render it consistently with the preceding frames. A video can continue in many valid ways, so without a description of the event the target is underdetermined.

To measure how much the target-aware summary helps, we ran a controlled comparison. We took 100 MacroBench Temporal cases (25 per reference-count bin, sampled deterministically prior to generation) and produced two summaries for each using the same two-stage procedure (an LLM drafter followed by an independent LLM verifier). One summary is written from all frames, as in the paper. The other is written from the input frames only, and its writer never sees the target frame or the original summary. Base and fine-tuned BAGEL then generate from each summary with identical input frames, seeds and settings, giving 400 outputs. These are scored with the paper's CCS/ISCS rubric, and the judge does not see the target frame.

ModelSummary written fromNCCSISCSScoreΔ vs. BAGEL, same summary [95% CI]
BAGELall frames, incl. target1004.634.024.27—
BAGEL + MACROall frames, incl. target1007.727.117.36+3.09 [2.44, 3.76]
BAGELinput frames only1004.233.793.95—
BAGEL + MACROinput frames only1007.086.446.71+2.76 [2.12, 3.42]

Writing the summaries from the input frames alone lowers the score of BAGEL by 0.32 and that of BAGEL + MACRO by 0.65 [0.10, 1.23], so a description of the target event does help. The fine-tuning gain without target information holds in every reference-count bin:

References (25 cases each)BAGELBAGEL + MACROΔ
1–35.607.13+1.53
4–53.517.06+3.55
6–73.036.07+3.04
8–103.656.58+2.93

We will add this comparison to the paper and describe the task as text-guided next-keyframe generation.

W2b — Temporal judge inputs (Appendix B.1 vs. Fig. 26)

Fig. 26 and the evaluation code in our supplementary material are correct: the Temporal judge receives the preceding frames, the instruction and the generated frame, but not the ground-truth frame. The sentence in Appendix B.1 grouping Temporal with Spatial is a typographical error, which we will fix in the revision.

The difference is deliberate. A spatial target view is uniquely determined by the camera pose, so the ground truth is a fair reference. A temporal continuation has many valid answers, and comparing against the single recorded frame would penalize plausible alternatives. The judge therefore scores consistency with the preceding frames (ISCS) and whether the described event happens with plausible motion (CCS).


W3 — Single-task versus multi-task training

This ablation is in Table 3 (Section 5.3). Joint training gives the best average (5.71 versus at most 4.27 for any single task) and the best result on three of the four tasks. We fine-tune BAGEL separately on each 100K task subset and compare with training on all four tasks:

Training data (Table 3)Cust. 1–5Cust. 6–10Illu. 1–5Illu. 6–10Spat. 1–5Spat. 6–10Temp. 1–5Temp. 6–10Avg
Bagel5.372.534.514.340.670.533.362.933.03
Bagel + All8.927.125.695.563.323.486.235.405.71
Bagel + Customization8.616.435.194.920.810.644.433.174.27
Bagel + Illustration5.462.165.935.701.211.225.144.593.92
Bagel + Spatial5.082.244.894.782.722.984.994.133.97
Bagel + Temporal5.251.874.964.841.051.076.175.193.80

Each single-task model mainly improves its own task. The one task where joint training is not best is Illustration, where the Illustration-only model is slightly better (5.81 versus 5.62 averaged over the two bins), likely because this task is the most text-heavy. Each single-task model is trained for one quarter of the full-dataset iterations so that all models complete the same number of epochs over their respective training sets. We will state this training budget explicitly in Section 5.3 and add a pointer to Table 3 in the introduction.


Q1a — How models with limited reference capacity handle 6–10 inputs

BAGEL and Qwen-Image-Edit-2511 take reference images as a sequence without a fixed number of image slots, so they accept ten inputs directly. For these models, the difficulty with many references lies in output quality rather than input capacity. OmniGen2, by contrast, indexes reference images using a learned five-row embedding table. As detailed in Appendix C.1–C.2, we preserve the five pretrained rows and append five new rows initialized from N(0, 0.02²), matching OmniGen2's native initialization scheme, and train all ten rows during fine-tuning. For the un-finetuned OmniGen2 baseline, we apply the same untuned extension so that it can be evaluated beyond five references. All models use the same dynamic input resolution (Table 11).


Q1b — Fine-tuning configuration and data usage

Each MacroData sample is one JSON record: an ordered list of up to ten reference images, an instruction that refers to them as <image 1> … <image n>, and one target image. All four tasks share this format, so the same loader serves single-task and mixed training. During loading, each reference is resized by the dynamic schedule (1024 / 768 / 512 px for 1–3 / 4–5 / 6–10 references) and inserted at the position of its corresponding <image k> placeholder. The model is trained with its native generation loss on the target. We mix in 10% text-to-image data to preserve general generation (Fig. 10), and sample reference-count bins at 2:2:3:3 for Customization and 1:1:1:1 for the other tasks (Fig. 8).

BackboneHardwareParallelismLearning rateBatching and trained parameters
BAGEL32 × H800FSDP2 × 10⁻⁵token-based dynamic batches of up to 32,768 tokens, ViT encoder frozen
OmniGen232 × H800DeepSpeed8 × 10⁻⁷global batch 64, image-index embedding extended to 10
Qwen-Image-Edit-2511128 × H800DeepSpeed1 × 10⁻⁵global batch 128, DiT fine-tuned

The training scripts and configuration files are included in the supplementary material, and we will consolidate these training details into a dedicated appendix section.

I think the authors addressed most of my concerns. I would like to raise the score.

We sincerely thank the reviewer for the constructive feedback. Your suggestions have been valuable in helping us improve the quality and clarity of the paper. We greatly appreciate your time and consideration.

Reviewer_o1SCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MacroData dataset with 400K samples for image generation conditioned on <=10 references. It also proposes MacroBench for evaluting, with 4000 samples. Experiments validate that finetuning on MacroData improves results.

Critical strengths

A useful data and bench for reference-based image generation, with experiments using several models validate the improvements.

Critical weaknesses

The contribution is too narrow. The tasks remain unchanged, the main difference is conditioning on more reference images. The work builds a dataset and a benchmark around this setting, but does not bring new insight or methodological advance.

Questions for authors

Appendix B.3, human validation does not include models tuned on MacroData.

2 comments

We thank the reviewer for recognizing the practical value of MacroData and MacroBench, and we provide the requested human evaluation of our fine-tuned models in Q1.


W1 — Is multi-reference generation only "more references"?

Our work differs fundamentally from prior approaches. We build a data curation pipeline tailored to general multi-reference generation and provide systematic analyses and insights into this regime. With many reference images, effective supervision cannot be obtained simply by feeding more inputs into existing pipelines.

1. Multi-reference generation is a long visual context modeling problem that differs in mechanism from few-reference settings. While 1–3 inputs can often be handled by copying global appearance, conditioning on 6–10 references requires extracting task-relevant cues from each image, composing entities or states across images, and ignoring irrelevant regions to avoid subject dropping and attribute leakage. Because few-reference supervision does not teach this cross-image coordination, open-source models degrade sharply as references increase and even the strongest proprietary models drop noticeably (Table 14). Likewise, fine-tuning on existing few-reference datasets improves MacroBench by at most +0.41, compared with +2.68 from MacroData (Table 1).

2. We expand the task scope to four distinct settings and design a curation pipeline that verifies the necessity of each reference. Prior datasets focus mainly on few-subject customization. In contrast, we study multi-reference generation across four tasks: Customization, multi-view 3D Spatial reasoning, cross-frame Temporal continuation, and interleaved document Illustration, which test attribute binding, geometric correspondence, temporal progression, and narrative grounding, respectively. Crucially, training data for this regime cannot be built simply by scaling up existing pipelines. Synthesis teachers degrade at high reference counts and frequently drop subjects (Nano Banana Pro passes verification only 53.5% of the time at 8–10 inputs), while raw web articles, videos, and multi-view sequences often contain uninformative images or non-overlapping views that allow models to ignore the visual context. We therefore design task-specific verification: plausibility resampling and bidirectional per-reference filtering for synthesized data, removal of non-contributing images with context rewriting for interleaved articles, and temporal-progression and view-overlap constraints for videos and 3D scenes. This ensures that every reference is necessary for predicting the target, and in a blind human audit only 3 of 120 accepted samples (2.5%) were judged bad. Joint training across all four tasks outperforms single-task models (5.71 average vs. at most 4.27, Table 3), consistently improves three backbones (Table 1), and generalizes well to the independent MultiBanana benchmark (+1.99 overall and +2.37 at 6–8 references on BAGEL).

3. We establish practical training principles for long visual contexts and offer initial findings on efficient modeling. On the data and training side (Section 5.3), our ablations show that the best mixture of reference counts depends on the task (Fig. 8), that mixing 10% text-to-image data preserves general generation quality (Fig. 10), and that training-free shortcuts such as input collages (8.02 → 5.60) or test-time thinking (Table 13) cannot substitute for native multi-reference training. On the modeling side (Appendix C.4), our exploratory study highlights two properties (Table 12): as references increase, models rely less on low-level VAE appearance features and more on high-level ViT semantics, and while dropping entire reference images hurts performance (8.02 → 7.42), text-aligned token selection within each image matches or slightly exceeds full attention using only 30% of the visual tokens (8.11, Table 12), suggesting a promising direction for token compression in future multi-reference architectures. We will highlight these distinctions and findings in the revised Introduction.

Q1 — Human validation of the fine-tuned models

Blind human ratings of all three fine-tuned models now complete the Human column of Table 5. Across all eight models the human ranking agrees closely with the Gemini judge (Spearman ρ = 0.976), BAGEL + MACRO ranks third under human evaluation and every VLM judge, and humans rate the fine-tuned models 1.40 to 2.16 points higher than their base models.

We rated 80 outputs per fine-tuned model (240 in total, 5 in each of the 16 task-by-reference-count cells), using the same test instances, task-specific criteria, and scoring formula as the 400 baseline ratings in Appendix B.3. Ratings were blind: the annotator viewed the reference images, the instruction, and the generated output without model names or judge scores.

ModelGeminiGPT-5.4OpusGPT-4.1Human
GPT-Image-1.57.40 (1)6.87 (1)7.08 (1)9.50 (1)6.87 (2)
Nano Banana Pro7.34 (2)6.77 (2)6.81 (2)8.99 (2)7.22 (1)
Bagel + Ours6.42 (3)6.20 (3)6.39 (3)7.94 (3)6.64 (3)
OmniGen2 + Ours4.40 (4)4.65 (5)5.30 (4)6.70 (5)5.24 (4)
Qwen + Ours4.31 (5)4.74 (4)5.17 (5)7.31 (4)4.70 (5)
Bagel3.04 (6)3.50 (6)4.61 (6)6.00 (6)4.51 (6)
Qwen-Image-Edit-25112.56 (7)2.51 (8)3.80 (8)4.53 (8)3.30 (7)
OmniGen22.40 (8)2.97 (7)4.22 (7)5.35 (7)3.08 (8)
Spearman ρ / Kendall τ with Human0.976 / 0.9290.929 / 0.7860.952 / 0.8570.929 / 0.786—

On the same 80 test instances per backbone, base and fine-tuned models compare as follows:

ModelNHuman, baseHuman, fine-tunedΔGemini, fine-tunedPearsonSpearmanKendall
Bagel + MACRO804.516.64+2.126.420.8730.9010.775
OmniGen2 + MACRO803.085.24+2.164.400.9330.9430.826
Qwen + MACRO803.304.70+1.404.310.8240.8080.692
ScopeNPearsonSpearmanKendall
Fine-tuned outputs (new)2400.8860.8950.776
Non-fine-tuned models (Table 6)4000.8330.8440.689
All outputs6400.8490.8580.710

The judge–human correlation on fine-tuned outputs (r = 0.886) is slightly higher than on the non-fine-tuned models (r = 0.833), so the judge does not favor the fine-tuned models.

In a pairwise comparison, three annotators chose between the base and fine-tuned outputs for the same input, shown in random order without model names. Across 40 cases per backbone (360 judgments), the fine-tuned output is preferred for every backbone:

ModelVotesFine-tuned betterTieBase betterFine-tuned win rate, ties excluded
Bagel12067 (55.8%)32 (26.7%)21 (17.5%)76.1% (67/88)
OmniGen212060 (50.0%)43 (35.8%)17 (14.2%)77.9% (60/77)
Qwen12070 (58.3%)30 (25.0%)20 (16.7%)77.8% (70/90)
All360197 (54.7%)105 (29.2%)58 (16.1%)77.3% (197/255)

Comments on the paper

2 comments

We thank all reviewers for their careful reading and constructive suggestions. We have posted detailed responses with additional experiments, including evaluation on the independent MultiBanana benchmark, blind human ratings and pairwise comparisons of all three fine-tuned models, and a human audit of training-data quality. These results provide further evidence of generalization to an external multi-reference benchmark and human preference for the fine-tuned outputs. We also provide task-specific control experiments and clarify the task definitions, evaluation protocol, and distinctions from prior work. We would greatly appreciate it if you could take a look at our responses and share any remaining concerns or questions. We welcome further discussion and would be happy to provide additional clarification wherever helpful.

AuthorsOpenReview ↗

Final Rebuttal Summary

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their thoughtful feedback and constructive discussions. Throughout the rebuttal process, we have conducted additional experiments, expanded our evaluations, and revised the manuscript to address the major concerns raised during the review.

We would like to summarize the key additions and clarifications:

  1. Generalization to independent benchmarks. We evaluated our fine-tuned models on the independent MultiBanana benchmark, covering multi-reference generation with 3–8 references. MACRO fine-tuning consistently improves performance across two backbones, with gains of +1.99 overall and +2.37 at 6–8 references for BAGEL, demonstrating that the benefits extend beyond our own benchmark and become more pronounced in long-context settings.

  2. Expanded human evaluation. We substantially extended our blind human evaluations across all three fine-tuned models. Across 264 paired comparisons, fine-tuned models achieve a 79.8% preference rate (excluding ties), increasing to 81.4% for 8–10 references. Human rankings also show strong agreement with automated evaluation (Spearman ρ = 0.976).

  3. Training data quality and reliability. We expanded the human audit of our data filtering pipeline with three independent annotators. In the challenging 8–10 reference regime, only 4.0% of accepted samples were rejected by human majority vote, supporting the reliability of our structured data construction process.

  4. Task validity and methodological insights. We provided additional controlled experiments on spatial viewpoint consistency, temporal conditioning, reference resolution, and multi-task training. We also clarified our evaluation protocols, corrected inconsistencies, and strengthened the discussion of related multi-reference benchmarks and methods.

Beyond these experiments, we emphasize that MACRO is not simply about increasing the number of reference images. Our contribution is to make structured long-context visual conditioning trainable and measurable across four complementary capabilities through a scalable dataset, a systematic benchmark, and empirical insights into multi-reference learning.

We have incorporated these additions and clarifications into the revised manuscript. We are particularly grateful to the reviewers who engaged in further discussions and acknowledged the improvements made during rebuttal.

We hope that the revised manuscript and additional evidence demonstrate the value of MACRO as a contribution to advancing multi-reference image generation.

Thank you again for your valuable time and consideration.

Best regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.