acceptodds
Under review as a conference paper at ICLR 2027

RippleVLM: Fast Structured Visual Extraction via Constrained Block Diffusion

Abstract

Multimodal extraction needs JSON that software can parse and values that match the page. Diffusion vision language models are well suited to that contract: tokens that are already known can be written into the response and held fixed, while unknown positions are filled by denoising, with bidirectional context inside each block. For a fixed-key record the whole object skeleton is known before generation. Template-Anchored Diffusion Decoding (TADD) therefore places field names and delimiters in a response template, denoises only the bounded value spans, and serializes those values into JSON. The returned object contains the requested keys because it was assembled from the template, and a generation cap cannot cut the object off mid-structure. Diffusion-Aligned Visual Projection (DAVP) adapts only the visual projector to this template, yielding models we call RippleVLM. On multiple datasets, RippleVLM-8B returns valid JSON with every required key in 2.50 s and raises content token F1 from 40.3% to 55.0%. A autoregressive model of same size baseline with Outlines reaches 58.4% F1 but passes the same JSON check at 38%, in 4.62 s under a 256-token cap. As the schema grows from 2 to 8 fields, TADD stays at 100% JSON validity, including on DiffusionGemma-26B. These results show that fixed-template diffusion decoding produces parseable JSON for bounded extraction while leaving content accuracy as a learned problem.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Diffusion VLMs

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4FX9OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Template-Anchored Diffusion Decoding fixes field names and structural tokens in advance, allocates a bounded token span to each string value, and denoises only the value positions. The predicted spans are extracted by position and serialized into JSON. Diffusion-Aligned Visual Projection adapts the visual projector while freezing the remaining model.

The primary structured-extraction comparison uses a 50-page PDFA cohort. RippleVLM-8B improves content token-set F1 from 40.3%40.3\% to 55.0%55.0\% relative to its TADD baseline and returns records passing the JSON-plus-keys check on all 50 pages. The reported Qwen3-VL-8B Outlines baseline achieves higher content F1, 58.4%58.4\%, but a lower completion rate under its configured output cap. Additional experiments examine other diffusion backbones, field counts, and general VLM benchmarks.

Critical strengths
  1. The output contract is explicitly limited. The paper distinguishes fixed-key string objects from arbitrary JSON Schema and separates structural validity from semantic correctness (pp. 3–4).
  2. Unfavorable outcomes are disclosed. The manuscript reports unresolved adaptation gains on most variants, inconsistent B32 runs, and regressions on general benchmarks rather than claiming universal improvement (pp. 7–9).
  3. The interface is concrete. Positional value extraction and projector-only adaptation provide a reproducible design in principle, and the paper describes source mappings and arithmetic checks for its saved evaluation records.
Critical weaknesses
  1. Adaptation generalization is not established. Evaluation pages are sampled from the same local PDFA cache used for projector training, without an explicit page-level holdout. The authors acknowledge that evaluation pages may have been seen during training. The reported 8B gain therefore cannot be interpreted as clean held-out improvement (pp. 5 and 8).
  2. The structural comparison is not symmetric. TADD's returned object is constructed under predetermined keys and passed through a serializer, whereas the autoregressive baseline is evaluated on generated text. An autoregressive model that generates field values separately and uses the same serializer would also obtain the relevant structural guarantee. This essential control is missing (p. 4).
  3. The content metric does not evaluate structured extraction. Token-set F1 compares extracted values against full-page OCR text, ignoring word order, repetition, and field binding. Incorrectly placing body text under a title key can still receive credit. Multi-field experiments lack field-level reference values, and heading/body training targets come from heuristic text splitting (pp. 5 and 14–15).
  4. The efficiency comparison has several uncontrolled factors. Autoregressive decoding uses a 256-token cap, TADD allocates 96 tokens per field, and Bard-VL free generation uses a different cap. Image preprocessing and implementations also differ. Raw Outlines outputs and termination reasons are unavailable, so its low completion rate cannot be attributed reliably to a decoding limitation rather than truncation (pp. 5–6 and 14).
  5. The adaptation evidence is small and unstable. Only the 8B gain has a paired interval excluding zero among the reported adaptation comparisons, and that result shares the holdout problem. Two B32 baseline runs differ markedly in structural validity, with no resolved cause. A matched alternative adaptation objective is also absent (pp. 7–8).
  6. A numerical inconsistency remains. The abstract reports 38%38\% Outlines validity, while Table 1 and the results text report 36%=18/5036\%=18/50 (pp. 1 and 6).
Questions for authors
  1. Can you rerun the adaptation evaluation on a strictly document-disjoint holdout and report exact overlap checks between training and evaluation data?
  2. What results are obtained by autoregressive per-field generation with the same field budgets, image preprocessing, and final serializer? Can you compare at matched field-level quality as well as matched latency?
  3. Can you evaluate against annotated field values using per-field accuracy and complete-record correctness, rather than page-level token-set overlap?
  4. Does DAVP outperform a matched alternative projector-training objective using the same data, trainable parameters, and compute? Which benefit is specifically attributable to the diffusion/template training arrangement?
  5. What caused the inconsistent B32 baseline runs, and which Outlines validity value is correct? Can you provide raw outputs, stopping reasons, and repeated-run measurements?
Reviewer_p2dwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies diffusion vision-language models for structured document extraction and proposes RippleVLM, which combines template-based decoding with visual projector adaptation. For tasks with predefined fields and string values, Template-Anchored Diffusion Decoding (TADD) fixes field names and structural tokens in the output template, generates only the field values, and constructs the final JSON through serialization. This separates structural completeness from content prediction. Diffusion-Aligned Visual Projection (DAVP) trains only the visual projector, keeping the visual encoder and language backbone frozen, to improve extraction under this decoding scheme. The evaluation separately measures structural validity, content token F1, and inference latency on document pages, with analyses across models, field counts, and token budgets. The reported results show gains in structural completeness from template-based decoding and improved content extraction from projector adaptation in the 8B model. The paper also examines how adaptation affects general vision-language capabilities.

Critical strengths
  1. The decoding design addresses a practical requirement in structured extraction.For records with predefined keys and string values, TADD fixes the known structure, generates only the field values, and constructs the final JSON through serialization. This separation gives downstream applications explicit control over the output format and reduces incomplete records caused by generation limits. The design makes useful use of diffusion models’ ability to fill unknown positions while retaining supplied content.

  2. The adaptation scheme is simple and reuses existing model backbones.DAVP updates only the visual projector while keeping the visual encoder and language backbone frozen. In the reported 8B evaluation, this adaptation raises content token F1 from 40.3% to 55.0%, providing initial evidence that a restricted training scope can improve extraction under the template interface. Implementations on Bard-VL and DiffusionGemma further demonstrate that the interface can accommodate both block and canvas diffusion architectures, adding practical value beyond a single model implementation.

Critical weaknesses
  1. The technical advance is limited, and the value of the added components is insufficiently established. Dimple [1] already demonstrates how predefined structural tokens can guide diffusion vision-language models in visual information extraction. RippleVLM combines this idea with blockwise decoding, positional extraction, and JSON serialization. Structural completeness largely follows from the template and serialization procedure, leaving the independent benefit of the decoding design unclear. DAVP uses fully masked field prediction to train the visual projector, but lacks comparisons with alternative objectives using the same data and trainable parameters. The contribution therefore appears primarily to adapt existing mechanisms to fixed-field extraction, with limited evidence of benefits beyond structural priors and task-specific fine-tuning.

  2. The evaluation is insufficient to establish field-level accuracy and generalization. Content F1 compares output token sets against full-page OCR text, ignoring field assignments, order, and repetition. Swapping content between fields may leave the score unchanged, so it cannot directly measure whether each field contains the correct content. The multi-field experiments in Appendix C likewise lack annotated field-level ground truth. Sections 4 and 6 also state that training and evaluation pages come from the same PDFA cache without a page-level holdout, allowing possible overlap. These limitations leave both extraction correctness and the generalization of the reported adaptation gains insufficiently validated.

  3. The comparisons do not isolate the sources of the reported structural and efficiency advantages. Outlines [2] generates entire records under a 256-token cap and is evaluated on its generated text, whereas TADD allocates field-value positions in advance and serializes the extracted values into JSON. Differences in structural validity therefore confound truncation, post-processing, and decoding methods. Table 1 also reports higher content F1 for Outlines than for RippleVLM-8B, so latency is not compared at matched content quality. Comparisons with consistent field specifications and serialization policies at similar extraction quality are needed to attribute the observed advantages to the proposed method.

  4. Presentation issues obscure the implemented method. Figure 1 presents “Vanilla Objective” and “TADD Objective” formulations without clearly connecting them to the implemented DAVP objective in Section 3.3. Figure 2 illustrates integer and enumeration types beyond the supported interface of predefined fields with string values. Readers must consult captions and later explanations to distinguish illustrations from actual functionality. Important details, including the heuristic construction of field supervision from OCR text, are primarily placed in Appendix B. The equations in Figure 1 and schema examples in Figure 2 are also too small to read comfortably. These inconsistencies and dispersed details make the method and its supporting evidence harder to assess.

References [1] Yu, R., Ma, X., & Wang, X. (2025). Dimple: Discrete diffusion multimodal large language model with parallel decoding. arXiv preprint arXiv:2505.16990. [2] Willard, B. T., & Louf, R. (2023). Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702.

Questions for authors
  1. What are the substantive technical advances over existing methods? Given Dimple’s use of structural priors and RippleVLM’s reliance on templates, what new problem does TADD solve, and what evidence supports its design? For DAVP, how are its gains distinguished from general document-task fine-tuning?

  2. Do the results demonstrate generalization to unseen documents? Please clarify training–evaluation overlap and provide field-level results on independent documents, as page-level token-set F1 cannot detect incorrect field assignments. Do DAVP’s gains persist after removing overlap and evaluating field correctness?

  3. Are structural validity and efficiency compared fairly? The autoregressive baseline faces a generation-length cap and different assembly and evaluation procedures. How do the validity and latency gaps change under consistent field specifications, serialization, and complete generation, particularly at comparable extraction quality?

Reviewer_YBXTOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RippleVLM for structured visual extraction, combining template-anchored diffusion decoding (TADD) with visual projector adaptation (DAVP). TADD fixes the JSON structure and generates only the value tokens, while DAVP trains the visual projector to improve extraction accuracy. The experiment shows improvements compared to the tested baselines.

Critical strengths
  1. The idea of fixing the JSON structure and only generating the value tokens is straightforward. It effectively avoids incomplete or invalid outputs, which is useful for structured extraction tasks.

  2. The method shows promising improvements in extraction accuracy and inference efficiency, especially with the 8B model. I also appreciate that the authors evaluate different diffusion VLM backbones and discuss the limitations of their approach.

Critical weaknesses
  1. The authors mention that the evaluation data may overlap with the training data, which makes it difficult to judge how much the reported improvement actually generalizes. I think a proper held-out evaluation is necessary.

  2. I am also not fully convinced by the comparison with AR baselines. The 100% JSON validity seems to come mainly from the predefined template and serialization, which could also be applied to AR models by manually injected the template during the autoregressive generation. A comparison under the same output constraints would help clarify whether diffusion decoding actually provides an advantage.

Questions for authors

Same as weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.