acceptodds
Withdrawn from ICLR 2027

OmniVAL: Validated Chain-of-Thought Distillation for True Native Omni Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

True native omni models promise a single architecture for understanding text, images, audio, and video, yet a critical gap remains: compact open-source omni models still fall short of proprietary frontier systems, and the gap is widest where it matters most - in *grounding* each answer in the right modality and the right evidence. We trace this gap to how such models are trained. Fine-tuning and distillation transfer final outputs or representations, but never the intermediate reasoning that grounding depends on: deciding which modality holds the evidence, localizing it, and expressing it in the proper coordinate space. We introduce **OmniVAL**, a validated chain-of-thought distillation framework that makes these decisions explicit and transfers them from a frontier teacher to a compact open-source student. A teacher generates modality-routed reasoning traces; **MA-VAL**, a modality-aware rule-based validator, filters unreliable traces and turns the student's remaining errors into corrective feedback in the units of each modality, whether pixels, milliseconds, cell indices, or modality-routing fixes. Perception tools serve only as training-time scaffolding: at inference the student is a pure omni model that answers and grounds in a single autoregressive pass. Across 16 benchmarks spanning documents, GUI agency, video, audio-visual reasoning, voice interaction, ASR, and speaker diarization, OmniVAL improves Nemotron 3 Nano Omni, an open 30B model with 3B active parameters, on every benchmark, raising DocVQA grounding mAP from 71.4 to 85.5 and cutting ASR word error rate by 18%; the resulting student surpasses GPT-4o and all open-source omni baselines and substantially narrows the gap to closed-source frontier systems. We release **OmniVAL-CoT**, 110K validator-verified multimodal reasoning traces, and show that half of this validated corpus outperforms the full unfiltered teacher pool. Implementation: https://anonymous.4open.science/r/OmniVal-216F

Primary area:
foundation or frontier models, including LLMs
Keywords:
Omni, text, images, audio, video, unified architecture, VAL, Annotation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tZ7wOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes OmniVAL, a validated chain-of-thought distillation framework for native omni-modal models. It extends the DocVAL framework to multiple modalities by introducing modality-specific validation rules and cross-modal routing checks. The framework filters teacher-generated reasoning traces and iteratively fine-tunes a Nemotron 3 Nano Omni student using structured corrective feedback. Experiments across 16 benchmarks report improvements in grounding and multimodal understanding.

Critical strengths

Strong empirical performance. The paper reports substantial improvements across diverse benchmarks, particularly on grounding-intensive tasks. The extensive evaluation and ablation studies demonstrate the practical effectiveness of the proposed training pipeline.

Critical weaknesses

1. Limited methodological novelty.

The proposed framework appears to be primarily an engineering extension of existing validated CoT distillation methods, particularly DocVAL. More concerningly, Figure 1 appears almost identical to that of DocVAL, which is unacceptable for an ICLR submission and raises serious concerns about the paper's claimed methodological novelty. Its main components—teacher-generated reasoning traces, rule-based validation, modality-specific feedback, and iterative supervised fine-tuning—largely follow established techniques. The primary extensions involve adapting validation rules to additional modalities and introducing a cross-modal routing consistency check.

The paper does not sufficiently establish what new methodological insight is contributed beyond integrating existing techniques into a broader multimodal training pipeline.

2. Poor presentation quality.

The presentation requires substantial improvement. The writing is unnecessarily verbose and repetitive, making it difficult to identify the core technical contributions. Figures 1 and 2 are overly cluttered, with small text, inconsistent visual organization, and unclear connections between components. Some graphical elements appear potentially AI-assisted or automatically assembled, although their origin cannot be determined from appearance alone. Regardless of how they were produced, the figures lack the clarity and careful technical presentation expected of an ICLR submission.

The current presentation substantially reduces the paper's readability and should be comprehensively revised.

Questions for authors

1. Novelty. What is the fundamental methodological contribution of OmniVAL beyond extending DocVAL to additional modalities? Please clarify which components introduce genuinely new learning mechanisms rather than engineering adaptations of existing methods.

2. Figure similarity and attribution. Figure 1 of OmniVAL bears a striking resemblance to Figure 1 of DocVAL, not only in the overall pipeline but also in the arrangement of components. Although DocVAL is cited in the related work, the figure caption does not acknowledge any adaptation from the prior work. Could the authors clarify whether Figure 1 was adapted from DocVAL and provide appropriate attribution if applicable? This extensive structural and visual similarity also reinforces my concerns about the limited methodological novelty of the proposed framework.

3. True native omni models. The paper repeatedly emphasizes "true native omni models." However, the student backbone already possesses multimodal capabilities before OmniVAL training. Moreover, the term "native" can imply deeper architectural integration, as exemplified by encoder-free multimodal architectures such as Gemma 4 12B, whereas Nemotron 3 Nano Omni still employs separate vision and audio encoders. What precisely distinguishes the "true native omni" models considered in this paper from conventional multimodal architectures? More importantly, what specific contribution does OmniVAL make to native omni-modal modeling beyond fine-tuning an existing multimodal backbone? The authors should clarify the definition, its architectural implications, and its relevance to the proposed method, or reconsider the corresponding claims.

4. Distillation baselines. The experiments include several internal ablations but lack direct comparisons with closely related CoT and validated distillation approaches, particularly DocVAL and other established multimodal reasoning distillation methods, adapted to the same backbone and training budget. Such comparisons are important for establishing the incremental value of the proposed framework.

5. Presentation. I strongly encourage the authors to substantially rewrite the manuscript and redesign the main figures. The current presentation obscures the technical contributions and falls below the standard expected for an ICLR paper.

2 comments

We thank the reviewer and address Figure 1 first.



1. Figure 1: provenance and attribution

Figure 1 was adapted from DocVAL's Figure 1. Its teacher→validator→student→inference layout is inherited. DocVAL is cited in Related Work section, but the caption omitted adaptation credit. We apologize for this oversight. The similarity also obscures OmniVAL's contributions. We will credit DocVAL (Neogi et al., 2025) in the caption, update the reference to ICML 2026, and redraw Figure 1 to separate inherited stages from modality-routed traces, M4 cross-modal validation, and modality-native corrections. We do not claim the inherited stages as novel.



2. Methodological novelty: validating evidence choice

DocVAL established validated teacher CoT, filtering, pixel-level corrections, and iterative refinement for document-image grounding. OmniVAL does not claim a new optimizer or backbone. Its distinct supervision target is which evidence stream supports an answer, and whether the selected route, reasoning trace, grounding, and evidence agree. An omni model may answer from subtitles when speech is required: even a correct answer and precise localization may rely on the wrong source. Answer accuracy and within-stream localization alone cannot detect this failure.

M4 checks (i) predicted versus reference grounding modality, (ii) trace versus emitted-grounding modality, and (iii) consistency with evidence in that modality (§3.2/App. A.3). It enables routing corrections alongside pixel, time, cell, and speaker corrections. This extends the object of validation beyond answer–location agreement.

Table 6 isolates M4, retaining the rest of OmniVAL:

Metric ↑Without M4FullGain
DocVQA mAP84.185.5+1.4
ScreenSpot-Pro63.773.4+9.7
WorldSense60.469.0+8.6
DailyOmni77.086.0+9.0

Gains are modest on DocVQA but larger on cross-stream and GUI grounding. Table 11 reports routing: audio-only 76.3→91.7; joint AV 73.6→88.2. In Table 22's interventions, M4 increases accuracy loss when relevant evidence is removed (32.4→36.8), while reducing the effect of irrelevant evidence removal (9.5→6.2), distractor flips (18.3→12.7), and hallucinations (22.1→16.5). These show observable evidence reliance, not CoT faithfulness. Principle: for multi-stream tasks, validate which source supports an answer, not just where evidence lies.



3. Meaning of “true native omni”

We agree the phrase may imply encoder-free design. Nemotron 3 Nano Omni uses modality-specific vision/audio encoders. Here native omni means one integrated inference model handling text, image, audio, and video without external OCR/ASR services, separate specialist inference pipelines, or validator calls at test time; it does not mean encoder-free. We will clarify the definition and temper “true native” in the title/abstract. OmniVAL changes training supervision, not the pretrained architecture.



4. New matched distillation controls

A complete same-backbone, data/compute-matched DocVAL adaptation remains desirable. Table 6's no-M4 row is not such a reproduction: it isolates routing validation within OmniVAL.

New measured controls, using the same omni backbone and D3∪D4 source pool with matched steps and approximately matched training FLOPs, provide additional points of comparison:

MethodDocVQA mAP ↑ScreenSpot-Pro ↑WorldSense ↑DailyOmni ↑
Validated-CoT SFT80.963.960.080.4
Rejection-sampling FT82.065.260.481.8
Full OmniVAL85.573.469.086.0

Versus matched validated-CoT SFT, OmniVAL gains +4.6 / +9.5 / +9.0 / +5.6 points, respectively. Rejection sampling also trails, especially on ScreenSpot-Pro and WorldSense. These test additional-exposure and sample-selection explanations without constituting a full DocVAL reproduction. Table 15 further fixes B2 examples/steps: full structured feedback improves ScreenSpot-Pro 65.0→73.4 and DailyOmni 80.0→86.0 over corrected targets alone. We will report the protocols and retain the direct-DocVAL-baseline limitation.



5. Presentation

We agree the presentation obscures the contribution. We will redesign Figures 1–2 with larger labels, fewer arrows, an M1–M5 legend, and separate training-only from tool-free inference; shorten teacher-choice discussion; and reorganize the Introduction around inherited workflow → cross-stream failure → M4 supervision → controlled evidence. We will accurately disclose figure-generation assistance, if any.



We hope these explanations and clarifications address the concerns the reviewer had, and kindly request the reviewer to reconsider the score. We're happy to provide further clarifications if needed. Thanks!

Thank you for the additional experiments. The M4 ablations provide useful evidence of its effectiveness, but my concern about methodological novelty remains: the overall pipeline closely follows DocVAL, and M4 alone does not establish a sufficiently distinct methodology.

The manuscript also remains unrevised, and the figures and "true native omni" terminology have not been updated yet. My final score will depend on the revised manuscript.

Reviewer_Zy8nOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces OmniVAL, a validated chain-of-thought distillation framework for native omni-modal models. The authors use Gemini 3.1 Pro with external perception tools to generate multimodal CoT data, and employ a rule-based validator, MA-VAL, to check answer correctness, grounding accuracy, and modality consistency. Approximately 110K validated traces are retained for supervised fine-tuning of Nemotron 3 Nano Omni, followed by iterative refinement using validator-generated corrective feedback.

The authors evaluate the method on 16 benchmarks covering document understanding, GUI tasks, video, audio-visual reasoning, ASR, and speaker diarization. They report substantial performance improvements and provide extensive ablation studies on validation, modality routing, and iterative refinement.

Critical strengths

Broad experimental evaluation with meaningful empirical improvements.

The authors evaluate the approach across multiple modalities and tasks, with ablation studies covering data filtering, iterative refinement, and modality routing. The reported improvements provide useful empirical evidence for the benefits of high-quality grounding supervision.

Critical weaknesses

Limited methodological novelty, with the main contribution being a multimodal extension of existing data validation and fine-tuning approaches.

The core OmniVAL pipeline consists of generating CoT data using a strong teacher, improving data quality through rule-based validation, and performing supervised fine-tuning with iterative corrective feedback. These ideas closely resemble the existing DocVAL framework.

The main extension in MA-VAL is the adaptation of validation rules to different modalities, including spatial coordinates, timestamps, speaker information, and modality routing. While this extension has some engineering value, it does not introduce a substantially new training mechanism or optimization method.

Overall, the paper primarily demonstrates that more accurate grounding annotations, higher-quality training data, and iterative supervision improve model performance. While these findings are reasonable, they offer limited novel research insights. I therefore find the methodological contribution insufficiently novel for ICLR.

Questions for authors

Methodological novelty: Beyond extending rule-based validation to additional modalities and introducing modality-routing checks, what is the fundamental methodological contribution of OmniVAL compared with DocVAL? Do these extensions introduce any new learning principles or generalizable insights?

1 comment

We thank the reviewer for the assessment. We agree that teacher-generated CoT, filtering, dual-mode validation, and iterative SFT build on DocVAL; we do not claim a new optimizer. Our contribution is a distinct supervision problem: learning which evidence stream(s) support an answer and where to ground it, and validating their consistency.



1. What changes fundamentally beyond DocVAL?

DocVAL validates answers and localization in a known document-image setting. An omni model can answer correctly from the wrong source - e.g., subtitles instead of speech. This is a kind-of-evidence error, not simply a degree-of-localization error: a precise box or timestamp cannot verify evidence choice. Answer-level supervision underdetermines evidence provenance.

OmniVAL's cross-modal validator M4 makes that missing variable explicit. It checks grounding type, trace–grounding modality agreement, and compatibility with supporting evidence (Sec. 3.2/App. A.3):

Qmodal=0.51[τp=τgt]+0.31[τc=τp]+0.21[gp consistent with S]Q_{modal}=0.5\mathbf{1}[\tau_p=\tau_{gt}]+0.3\mathbf{1}[\tau_c=\tau_p]+0.2\mathbf{1}[g_p\text{ consistent with }S].

Mismatches lead to route-aware corrective supervision alongside pixel, temporal, cell, and speaker corrections. Inference uses no external tools.

AspectDocVALOmniVAL
Evidence choiceKnown document-image domainEvidence-stream/type selection + grounding
ValidationAnswer/spatial consistencyRoute–trace–grounding–evidence consistency
CorrectionsPixel-levelPixels, time, cells, speakers, routing
StudentDocument VLMNative omni model, tool-free inference


2. Is this more than additional engineering?

M4 is ablated with the remaining pipeline retained (Table 6):

Metric ↑Without M4FullGain
DocVQA mAP84.185.5+1.4
ScreenSpot-Pro63.773.4+9.7
Video-MME76.581.3+4.8
WorldSense60.469.0+8.6
DailyOmni77.086.0+9.0

The effect is modest for fixed-domain DocVQA and larger where evidence selection matters. Table 11 measures that decision: audio-only routing 76.3→91.7 (+15.4) and joint AV routing 73.6→88.2 (+14.6) with M4.

Controlled evidence interventions (Table 22) show a different pattern of reliance: with M4, removing relevant evidence hurts more (32.4→36.8), while removing irrelevant evidence hurts less (9.5→6.2). Distractor flips fall 18.3→12.7, hallucinations 22.1→16.5, and grounding accuracy rises 72.0→78.6. These tests support appropriate evidence reliance, not causal CoT faithfulness.



3. What generalizable learning principle follows?

Principle: Where multiple streams can yield a plausible answer, correctness of the answer and localization within one stream are insufficient to verify the evidence source. Training should therefore validate source–trace–grounding consistency, not just answer–grounding consistency. This makes a testable prediction: source-aware validation should help more on ambiguous cross-stream tasks than on fixed-evidence tasks. Tables 6/11 support that contrast, and Table 22 tests evidence reliance directly. The insight concerns the supervision target, not a particular architecture or optimizer.

Validation matters, but is not the new claim. Table 14 shows unvalidated CoT+grounding at 72.8 DocVQA mAP, below answer-only (74.0) and answer+grounding (77.5); filtered CoT+grounding reaches 78.9. We credit DocVAL with establishing validation's value; OmniVAL additionally checks whether the source of grounding is appropriate.

How corrections are represented also matters. With B2 examples/updates held fixed (Table 15), corrected targets reach 65.0 ScreenSpot-Pro / 80.0 DailyOmni, versus 73.4 / 86.0 with full feedback. A reduced-scale comparison (Table 17) favors structured-feedback SFT over scalar-reward GRPO by +2.7 DocVQA mAP / +4.9 CMM resistance; we do not claim universal superiority over RL.

The approach is not Nemotron-specific. Qwen3-Omni, trained on a 44K subset (Table 24), improves DocVQA mAP 74.2→81.9, ScreenSpot-Pro 56.4→61.7, Video-MME 71.0→76.1, and DailyOmni 72.5→77.8. This supports transfer beyond the original student, where multiple evidence streams compete.

We will distinguish inherited DocVAL mechanisms from route-aware validation and corrective supervision in the Introduction/Related Work. Table 6 isolates M4, not whether an explicit route tag itself is indispensable; a route-tag × M4 factorial ablation would test that separately.



We hope these findings address the novelty and informativeness concerns the reviewer had, and kindly request the reviewer to reconsider the score. We're happy to provide further clarifications if needed. Thanks!

Reviewer_MsevOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes OmniVAL, a validated chain-of-thought distillation framework for compact native omni models.

A frontier teacher (Gemini 3.1 Pro), supported by perception tools used only during training, writes modality-routed reasoning traces with answers and modality-specific grounding. A rule-based validator, MA-VAL, filters these traces to a 110K-example corpus. The student (Nemotron 3 Nano Omni, 3B active parameters) is fine-tuned on this corpus. MA-VAL is then run in verifier mode to convert the student’s errors into corrective feedback in modality-native units (pixel deltas, millisecond deltas, cell indices, routing fixes), and the student is iteratively instruction-tuned on this feedback. At inference the student runs alone. The paper reports improvements on all 16 benchmarks (documents, GUI, video, audio-visual, speech) and claims the student surpasses GPT-4o and all open-source omni baselines.

Critical strengths

The problem is relevant: grounded, verifiable outputs from compact omni models that can run on a single GPU. Making modality routing an explicit, validated step, and identifying cross-modal mis-routing as a distinct failure mode, is a useful framing. The Module 4 ablation (Tables 6 and 11) supports its importance on audio-visual tasks. Inference uses no external tools, which keeps deployment simple. The appendix contains extensive ablations: factorial supervision (Table 14), feedback-content control (Table 15), scaffolding noise (Table 19), and transfer to a second student (Table 24). The planned release of the trace corpus is a potential contribution.

Critical weaknesses

Stage B2 gains may largely come from additional training. Stage B2 adds about 13 iterations × 2 epochs over D4 (Table 4, Table 7), roughly the same budget as Stage B1. It also uses 11K new examples (D4) that Stage B1 never sees. Table 15 suggests much of the gain comes from this extra training: with corrected targets alone, DocVQA mAP already rises from 78.9 to 83.2, out of a total of 85.5. Please provide compute-matched baselines, for example standard SFT on D3 ∪ D4 with the same total number of updates as B1 + B2.

Insufficient method details and failure analysis. Much of the paper (§4 “Model selection”, §5.2 “Teacher choice”, Appendix C.6) is spent justifying the teacher choice. Meanwhile, key details are missing, such as how the feedback F is serialized and how S_struct, S_coord, and S_align are computed (Appendix A.3). There is also no qualitative analysis of rejected traces or of the cases where the student still fails.

Unfair comparison with frontier models. The student is fine-tuned on the training splits of the evaluated benchmarks (§3.3, Appendix A.1), while all frontier models are evaluated zero-shot (§4). The claims of surpassing GPT-4o and narrowing the frontier gap therefore mix the effect of the method with that of in-domain fine-tuning. Please report benchmarks with and without training data in D1 separately.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.