acceptodds
Under review as a conference paper at ICLR 2027

VisFlow: Interpretable Flow Matching for Visibility Reconstruction, Characterization, and Recognition

Abstract

Methods for learning continuous representations, such as latent manifolds, are fundamental and ubiquitous in modern machine learning. Yet many geometric objects combine continuous shape information with discrete combinatorial structure. This combination presents a fundamental challenge for representation learning because small geometric perturbations can induce abrupt changes in the underlying discrete representation. As a result, the smoothness assumptions that make continuous representations effective no longer hold. Polygons provide a canonical example. Their vertex locations vary continuously, but infinitesimal changes in geometry can dramatically alter the associated visibility graph. Existing methods address the inverse mapping from visibility graphs to polygon geometry using vertex-, triangulation-, or signed-distance-function (SDF)-based representations. Vertex- and triangulation-based approaches construct polygons through explicit geometric operations. However, they are less effective at modeling the relationship between geometry and visibility. SDF-based approaches capture this relationship more naturally. Their intermediate states, however, do not correspond to explicit stages of polygon construction. We present VisFlow, a novel, interpretable flow formulation that combines continuous SDF representations with explicit polygon evolution. Our method models polygon generation as rectified flow in the latent space of continuous SDFs. Standard rectified flow linearly interpolates between noise and the target SDF, retaining the key limitation of SDF-based generation. Its intermediate states lack an interpretation of polygon formation. In contrast, VisFlow learns approximately linear latent paths to represent explicit stages of polygon growth along SDF normals. We evaluate VisFlow on Visibility Reconstruction and Visibility Characterization. Visibility Reconstruction generates one polygon consistent with a visibility graph \(G\), while Visibility Characterization characterizes all such polygons. VisFlow maintains 20% more visibility relations on Visibility Reconstruction and improves coverage by 30% on Visibility Characterization compared with prior methods. We further show that VisFlow generalizes to Visibility Recognition, where the input graph is not guaranteed to be valid. We also demonstrate that VisFlow extends to a real-world protein reconstruction problem by applying its trajectory-supervision formulation to generate protein structures from contact graphs.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
geometric representations, flow matching, visibility graph

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_78bcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VisFlow, a flow matching method to perform Visibility Characterization, Visibility Reconstruction and Visibility Recognition. The idea of VisFlow is to encode the signed distance function of a polygon into a latent space and learn a latent representation in which stages of polygon formation follow approximately linear trajectories. So also intermediate states along the trajectories that are polygon evolutions . The trajectories are obtained by applying grassfire flow to progressively collapse every polygon to a triangle and then reversing the resulting trajectory. During sampling they draw an initial triangle and evolve its latent representation to generate a polygon condition on the target visibility graph.

VisFlow consists of three main components:

  • The Visibility Prediction Network: this part provides a differentiable approximation of polygon visibility. It is trained using binary cross entropy on geometric visibility predicated and it is later used to provide visibility supervision.
  • SDF Latent Generator: this module is an encoder decoder architecture that represents polygon using SDF latents. It is trained using reconstruction, visibility and alignment and regularization losses.
  • Flow Netwrok: this part models polygon generation from the initial triangle latent to the target polygon latent, conditioned on the target visibility graph. The three components are trained sequentially: first the Visibility Prediction Network, then the SDF Latent Generator, and finally the Flow Network, keeping the previously trained components frozen. The authors also introduce a new synthetic dataset containing 10,000 polygons. The dataset is designed to cover more complex concavities than previous datasets, including VisDiff. The test set contains unseen visibility graphs to evaluate generalization to new visibility structures.
Critical strengths
  1. The experimental results are promising, especially for visibility reconstruction and characterization.
  2. The proposed method including the combination of latent representations, grassfire trajectories, and flow matching is interesting and within the scope of the conference.
  3. The new dataset includes more complex polygon geometries and visibility structures than previous benchmarks.
Critical weaknesses
  • The authors claim that the method is interpretable, but this property is not clearly explained later.
  • The methodology section is not very clear, especially regarding the interaction between the three components and the reader has to read it multiple times in order to understand it, also the reader is required to check the appendix in order to understand many essential part of the paper such as the architectures and the dataset. In particular on this last point, the new dataset is presented as a contribution in the introduction, but its construction and main characteristics are only discussed in the appendix.
  • The contribution of each architectural component is not sufficiently investigated.
  • None of the statement of the Introduction is supported by references (there are no citations at all in the introduction).
Questions for authors
  • How much do visibility supervision, latent alignment, and the flow-based generative model contribute to the final performance compared to simpler alternatives?
5 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their careful evaluation of our work. We appreciate their recognition of the promise of our experimental results. Below, we address the reviewer’s concerns regarding the interpretability claim, ablation study, and presentation.

  • Interpretability Claim: We acknowledge the reviewer’s concern that the interpretability of VisFlow was not explained clearly enough in the manuscript, and we apologize for the lack of clarity in our presentation. We refer to the method as interpretable because the learned latent trajectory is explicitly structured using Grassfire polygon evolution, so intermediate latent states are intended to correspond to meaningful intermediate polygon states rather than arbitrary points in the latent space, as in latent diffusion. However, these intermediate states cannot be directly decoded during normal inference using their corresponding visibility graphs because only the final target visibility graph is available at inference, while the intermediate visibility graphs are unknown. Therefore, to directly evaluate whether the intermediate flow embeddings are interpretable and follow the Grassfire evolution, we perform an additional comparison against the ground-truth Grassfire trajectory on held-out test polygons:

    • We first run Grassfire on the original unseen polygons from the held-out test set. For each polygon, we record the number of Grassfire steps N, the encoder embedding Zₜᴳᵀ of each intermediate state, and its ground-truth visibility graph Gₜᴳᵀ. We then perform N flow steps conditioned only on the target visibility graph G and store the corresponding intermediate flow embeddings Zₜ. We pass both Zₜᴳᵀ and Zₜ through our trained decoder using Gₜᴳᵀ and compare the decoded polygons with the corresponding ground-truth Grassfire polygon at time t using visibility F1 and Chamfer distance. Gₜᴳᵀ is used only for this experimental evaluation and is not available or used during normal inference for our reported results. We report the mean ± standard deviation across all intermediate states and held-out test polygons.

      ApproachChamfer DistanceVisibility F1
      Zₜᴳᵀ0.24 ± 0.090.83 ± 0.05
      Zₜ0.27 ± 0.090.82 ± 0.05

      We observe that replacing Zₜᴳᵀ with Zₜ changes visibility F1 by only 0.01 (0.83 → 0.82) and Chamfer distance by 0.03 (0.24 → 0.27). Since both embeddings are decoded using the same intermediate visibility graph Gₜᴳᵀ and evaluated against the corresponding Grassfire intermediate polygon, this provides evidence that the intermediate flow embeddings correspond closely to the Grassfire evolution at the same stage.

  • Component Ablation: We thank the reviewer for raising the question regarding the contribution of the individual components of VisFlow. We have evaluated these contributions through dedicated ablation studies. Due to the space limitations of the main paper, we placed these results in the appendix rather than in the main text. Specifically:

    • Appendix 9.7 evaluates the contribution of the visibility loss by comparing it against training with only the MSE reconstruction loss.
    • Appendix 9.7 also evaluates the contribution of the inference-time optimization component.
    • Appendix 9.8 compares VisFlow with Cont-VisDiff, the continuous-SDF VisDiff baseline introduced in our work. Cont-VisDiff uses the same continuous SDF latent representation as VisFlow but does not use the proposed Grassfire-based supervision that aligns intermediate latent states with the polygon-propagation trajectory. Instead, it performs standard diffusion directly in the learned continuous SDF latent space. This comparison isolates the contribution of the proposed Grassfire-based supervision from the benefit of the continuous latent representation itself.
  • Presentation: We appreciate the reviewer for highlighting the presentation concerns, and we apologize for the lack of clarity in the main content of the manuscript regarding the dataset contribution. Due to space limitations in the main paper, we placed the detailed dataset construction and characteristics in the appendix. We agree that some of this information would be helpful in the main text, and we will include a concise description of the dataset construction and its key characteristics in the final version. We will also add additional references to support statements in the introduction.

I thank the author for their response. About the change in the presentation it would have been nice to already see some of them in practice here in the revised version.
For the additional experiment regarding the interpretability claim. I appreciate them, but remain unconvinced that it adequately supports the interpretability claim. The experiment primarily demonstrates consistency with the Grassfire evolution under oracle-assisted decoding.

In particular, the decoder relies on ground-truth intermediate visibility graphs, which are unavailable during inference. Therefore, it is unclear how much of the reconstruction quality can be attributed to the learned embeddings rather than to the provided graphs. Moreover, similar reconstruction performance does not necessarily imply that the latent states have an accessible or independently verifiable geometric meaning. I therefore believe that the experiment provides evidence of geometric consistency rather than interpretability.

Regarding the promised improvements to the presentation, it would have been helpful to see at least some of these changes already incorporated into the revised manuscript.

AuthorsOpenReview ↗

Additional Experiment - Interpretability

  • Intermediate-State Evaluation: We thank the reviewer for the clarification. We understand that the main concern with the additional interpretability experiment is that, since the decoder is provided with the ground-truth intermediate visibility graph GtGTG_t^{GT}, it is unclear how much of the reconstruction quality is attributable to the intermediate flow embedding ZtZ_t versus the provided visibility graph. To clarify this, we perform an additional zero-latent experiment, where we replace ZtZ_t with a zero vector while providing the decoder with the same GtGTG_t^{GT}.

    ApproachChamfer DistanceVisibility F1
    ZtGTZ_t^{GT}0.24 ±\pm 0.090.83 ±\pm 0.05
    ZtZ_t0.27 ±\pm 0.090.82 ±\pm 0.05
    Zero Latent0.50 ±\pm 0.090.60 ±\pm 0.05

    Using the same intermediate visibility graph, replacing ZtZ_t with a zero vector increases the Chamfer distance from 0.27 to 0.50 and decreases visibility F1 from 0.82 to 0.60. This isolates the contribution of the latent embedding and shows that the reconstruction quality cannot be attributed to the intermediate visibility graph alone.

  • Presentation Changes: Regarding adding some presentation changes to the current version of the manuscript, we have added the supporting citations to the introduction in the revised manuscript. We will also use the additional page available in the final camera-ready version to include a concise description of the dataset construction and its key characteristics in the main paper.

Reviewer_V7zkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the inverse problem of generating a simple polygon whose visibility graph matches a given graph GG. VisFlow has three parts: (i) an SDF Latent Generator that encodes continuous SDF samples into 6×6 PatchNets-style patch latents and decodes vertices with 50 learnable queries conditioned on GG, trained so that reversed grassfire trajectories (triangle → polygon) lie approximately on straight lines in latent space (alignment loss, Sec. 4.1); (ii) a Visibility Prediction Network built from in-cone and intersection predicates that gives a differentiable surrogate of visibility, used as a training loss and for gradient-based vertex refinement at inference (Sec. 4.2); and (iii) a graph-conditioned rectified-flow U-Net that transports an initial triangle latent to the polygon latent (Sec. 4.3). On a new 50-vertex dataset, VisFlow + refinement raises visibility F1 from 0.76 (VisDiff) to 0.91 (Table 1) and from 0.91 to 0.97 on the original VisDiff data (Table 5); a BFS-based "coverage" metric for characterization strongly favours VisFlow (Table 2); a proxy recognition task reaches 80% accuracy (Fig. 5); and a protein contact-graph variant reaches F1 0.86 on FragFold150 (Table 3). Appendix tables separate the contributions of the continuous representation, polygon propagation and refinement (Tables 6–11).

Critical strengths

A concrete weakness of grid-based SDF reconstruction is identified and fixed. VisDiff drops from 0.91 F1 on its own 25-vertex data to 0.76 on the 50-vertex dataset, which the paper attributes to grid resolution; Table 11 confirms that even given the ground-truth SDF and graph, the VisDiff decoder recovers polygons worse than the continuous-latent decoder (0.74 vs. 0.82–0.83). With the visibility loss (Table 6) and refinement (Table 7), the full system reaches 0.91 on the new data and 0.97 on the old data (Tables 1, 5), improving on all baselines on both.

The ablations are candid about where the gains come from. Table 9 shows that a continuous-latent diffusion baseline (Cont-VisDiff) with the same refinement reaches the same 0.91 F1 as VisFlow, and the text says so plainly ("polygon propagation provides only a small additional improvement"). The benefit of the trajectory formulation is then confined to the characterization experiment (Table 10). I appreciate that the paper does not fold everything into one headline number; it makes the remaining claims easy to locate — and, as below, to scrutinize.

Critical weaknesses

The characterization metric measures local acceptance under source perturbation, not coverage of the realization space. Sec. 3 defines characterization as generating all simple polygons with V(P)=GV(P)=G. What Secs. 5/5.2 measure is the fraction of nodes of a BFS tree in source space (b=2b=2, d=5d=5) whose decoded polygon has visibility F1 above T∈{0.7,0.8}T\in\{0.7,0.8\} and Chamfer distance above TdT_d from previously accepted ones. Three problems: (a) T=0.7T=0.7 and 0.80.8 are below VisFlow's mean reconstruction F1 of 0.91, so almost every sample passes the F1 test and "coverage" degenerates to "does perturbing the source change the output", which a triangle parameterization satisfies nearly by construction (1.00 ± 0.00 in Table 2); (b) the two methods are perturbed in incomparable spaces — Gaussian noise for VisDiff/Cont-VisDiff vs. triangle parameters for VisFlow, each at "10% of the sampling scale" — so equal perturbation budgets do not mean equal exploration in polygon space; (c) nothing measures how many distinct valid realizations exist or are found (the dataset itself contains 20 graph-preserving perturbations per polygon that could serve as a reference set). This matters because, after Table 9, this metric is the only evidence for the benefit of polygon propagation, and it is the basis of the abstract's "30% better coverage" (I also could not tell how the 30% figure is derived from Table 2).

The "interpretable trajectory" claim is not established at inference time. During latent-generator training, each intermediate state is decoded with the visibility graph of the corresponding ground-truth grassfire state (Sec. 4.1, Lcoord\mathcal{L}_{\text{coord}} paragraph). At inference only the final target graph is available (Sec. 4.3). The paper does not say what graph, if any, is used to decode an intermediate latent, and never evaluates whether intermediate states on unseen graphs are simple polygons that follow the grassfire process; Fig. 1 (bottom) is the only evidence and it is a single example. Appendix 9.1 describes only edge-collapse events, leaving unspecified what happens when non-adjacent boundary parts meet in concave polygons (grassfire can split the shape) and how vertex correspondences and the per-vertex graph conditioning are handled as the vertex count changes. Without these, the method is a continuous-latent flow with a triangle source and a trajectory-regularized encoder — which, per Table 9, reconstructs about as well as ordinary latent diffusion.

  • Fig. 1 time labels vs. Sec. 4.1; Fig. 7 arrow labels (see Presentation).
  • Sec. 9: "preleminary"; Sec. 9.2: "details fo SDF Ltent Generator"; Sec. 9.3: "details fo Flow Network"; Sec. 5.3: "threshold TT"; Sec. 9.1: "IInCone∈0,1I_{\text{InCone}}\in 0,1" → "{0,1}\{0,1\}"; Sec. 5 (Baselines): missing period after "appendix 9.10".
  • Table 4 is titled "Vertex Prediction comparison" but compares visibility predictors; it also names the model "Visibility Verifier Network" while the text says "Visibility Prediction Network".
  • Abstract/Conclusion: "22% greater shape diversity" corresponds to the maximum link diameter (11 vs. 9, Table 12); the mean is 48% larger — say which statistic is meant.
  • Fig. 6 panels seem to contain leftover plot titles from the plotting script ("raw decoder", "CA -> PULCHRA"); please check the rendering.
Questions for authors
  • Evaluation (clarification and request; decisive for Weakness 1). Are all reported F1 scores computed with exact geometric visibility rather than the learned predictor? What fraction of outputs match GG exactly, and what fraction are simple (non-self-intersecting)? For characterization: does the advantage persist under independent sampling with a matched sample budget, measured as recall of a reference set of known realizations (e.g., the 20 per-graph instances in your dataset) at a fixed Chamfer threshold?

  • Inference (clarification; decisive for Weakness 2). With only the final target graph available, what graph is supplied when decoding an intermediate flow state? Can you report, on unseen graphs, the fraction of intermediate states that are simple polygons and their agreement with the grassfire state at the same tt? Do the training and inference triangle distributions match?

  • Trajectory construction (clarification). How does the grassfire construction handle non-adjacent boundary encounters in concave polygons, how are ground-truth vertices and graphs represented across changes in vertex count, and how are lower-cardinality states represented with the 50 learnable queries of Appendix 9.2?

  • Protein experiment (clarification; affects how much weight the second contribution carries). The C-VAE and GNN baselines in Table 3 have precision 0.05–0.06, i.e. they do not function on this task, and the VisFlow decoder is additionally conditioned on the amino-acid sequence (Sec. 9.6). Do the baselines receive the sequence? A comparison with a contact-map-based structure method, or at least a distance-geometry / MDS baseline, would make Table 3 meaningful.

5 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their detailed evaluation of our work and for the helpful feedback. We also appreciate their recognition of our clear ablation studies. Below, we address the reviewer’s questions and concerns regarding the reconstruction and characterization metrics, the Grassfire–Flow intermediate-state comparison, trajectory construction, and the protein baseline.

  • Reconstruction Evaluation: The reviewer raised a question about which visibility computation is used to determine the F1 score in the reconstruction experiment. We apologize for the lack of clarity in the manuscript regarding the visibility computation used for the reconstruction metric. We would like to clarify that all reported F1 scores are computed from visibility graphs obtained using exact geometric visibility. After 100 epochs of our proposed inference-time optimization, 100% of the reconstructed outputs are simple polygons, while 5% achieve an exact visibility-graph match with the target graph. The exact-match criterion is substantially stricter than visibility F1, as any incorrect visibility edge causes the reconstruction to be counted as a non-match. Since our inference-time optimization iteratively improves the reconstructed polygon, the exact-match rate can be further increased by using additional optimization steps beyond the 100-epoch budget reported here. However, we were unable to evaluate larger optimization budgets within the rebuttal time constraints.

  • Trajectory Construction: The reviewer requested clarification on how non-adjacent boundary encounters are handled during Grassfire evolution, how intermediate states are represented as the vertex count changes, and how lower-cardinality states are processed by the decoder with 50 learnable queries. We apologize for the lack of clarity in the manuscript regarding these implementation details. Our implementation tracks the Grassfire evolution at the level of individual boundary vertices. When non-adjacent boundary fronts meet, we do not explicitly split the evolving shape into multiple polygon components; instead, we continue to represent the intermediate state using the remaining active vertex trajectories. As vertices become inactive during the collapse, the intermediate state can contain fewer than 50 vertices. We pad these lower-cardinality vertex representations to the fixed maximum of 50 vertices and use a corresponding validity mask. Since our decoder is Transformer-based and uses 50 learnable vertex queries, this fixed-size padded representation allows it to process intermediate states with varying vertex cardinality. The corresponding visibility graph is similarly padded to a 50 × 50 representation, and the losses are computed only over the valid vertices and visibility entries.

  • Protein Baseline: The reviewer requested clarification on whether the C-VAE and GNN baselines receive the same amino-acid sequence conditioning as VisFlow and suggested adding a contact-map-based structure reconstruction baseline. We would like to clarify that both the C-VAE and GNN baselines are trained using the same amino-acid sequence conditioning as VisFlow. To further strengthen the comparison, we additionally evaluate against an MDS-based [1] contact-map reconstruction baseline that reconstructs 3D structure from the contact map. We report contact-graph F1 and Cα-RMSD below. Cα-RMSD measures the geometric deviation between the predicted and ground-truth protein structures after alignment using the Cα atom positions of corresponding residues; lower values indicate better 3D structural reconstruction.

    ApproachContact Graph F1Cα-RMSD (Å)
    MDS [1]0.837.4
    VisFlow0.866.3

    VisFlow achieves higher contact-graph F1 (0.86 vs. 0.83) while reducing Cα-RMSD by approximately 15% (7.4 Å to 6.3 Å) relative to MDS, demonstrating consistent improvement in both contact-graph reconstruction and 3D structural recovery.

  • Presentation Comments: We appreciate the reviewer’s attention to detail in identifying the presentation and typographical issues in the manuscript. We apologize for these errors and will ensure that all corrections are made in the final camera-ready version of the paper. Update: We have addressed the corrections in the revised version of the manuscript

[1] Pietal, M. J., Bujnicki, J. M., & Kozlowski, L. P. (2015). “GDFuzz3D: a method for protein 3D structure reconstruction from contact maps, based on a non-Euclidean distance function.” Bioinformatics, 31(21), 3499–3505.

AuthorsOpenReview ↗

Rebuttal by Authors - Characterization Evaluation

  • Characterization Evaluation: We thank the reviewer for raising concerns regarding our original characterization metric. Specifically, the reviewer notes that (1) the BFS-based metric measures distinct valid outputs under local source perturbations rather than global realization-space coverage, (2) VisFlow and the diffusion baselines are perturbed in different sampling spaces—triangle parameters for VisFlow and Gaussian noise for the diffusion baselines—making their exploration budgets difficult to compare directly, and (3) the BFS metric reports the acceptance rate of sampled perturbations rather than directly measuring the diversity of valid realizations generated by each method. We agree that our original BFS-based characterization metric primarily measures the ability to generate distinct valid realizations under local perturbations of the source rather than direct global coverage of the realization space. We originally adopted the BFS-based metric as a practical proxy because the realization space associated with a visibility graph is continuous and cannot be exhaustively enumerated. To address these concerns, we perform an additional characterization evaluation of VisFlow and Cont-VisDiff using independent sampling with matched sampling budgets, thereby avoiding the local BFS perturbation procedure and comparison across differently scaled perturbation spaces. Cont-VisDiff is the continuous-SDF diffusion baseline introduced in our work. It uses the same continuous SDF latent representation as VisFlow but does not use the proposed Grassfire-based supervision. Instead, it performs standard diffusion directly in the learned continuous SDF latent space. This allows us to evaluate whether the characterization advantage of Grassfire-based training persists under independent sampling as well.

    • We propose using the percentage of distinct realizations obtained through independent sampling under a matched sampling budget as the primary metric. The reviewer suggested measuring recall against the 20 graph-preserving augmentations available in the dataset. However, the 20 augmentations are not available for the held-out test visibility graphs used in our characterization experiment. More importantly, even if a finite reference set were constructed, an independently generated valid polygon could represent a different valid realization while matching none of the reference samples. Reference-set recall would therefore underestimate the diversity captured by the model. We instead measure distinctness among independently generated valid samples for VisFlow and Cont-VisDiff under matched sampling budgets M. To account for variability introduced by random sampling, we repeat each experiment N = 5 times for every sampling budget and report the average distinctness across runs. We use a distance threshold of TdT_d = 0.05 to determine distinct realizations and a visibility F1 threshold of T = 0.8 to determine valid realizations.

      Sampling Budget M (N = 5 runs)VisFlow (Ours) Distinct Realizations (%)Cont-VisDiff Distinct Realizations (%)
      554.2%58.6%
      1054.1%51.9%
      2054.2%43.8%
      5054.3%32.9%

      We observe the same trend as in the BFS-based experiment. As the sampling budget increases, the independently generated Cont-VisDiff samples become increasingly clustered, reducing the fraction of distinct realizations. In contrast, VisFlow maintains a nearly constant fraction of distinct realizations as the sampling budget increases. The approximately 30% coverage improvement stated in the paper refers to the absolute improvement in the original BFS-based experiment from 0.68 for VisDiff to 1.00 for VisFlow, corresponding to approximately 32 percentage points.

AuthorsOpenReview ↗

Rebuttal by Authors - Grassfire–Flow Intermediate-State Comparison

  • Grassfire–Flow Intermediate-State Comparison: We thank the reviewer for raising the important question of whether the intermediate states learned during Grassfire-based training are preserved and used by the flow model at inference. During inference, the flow-matching model is conditioned only on the target visibility graph, as the corresponding intermediate visibility graphs are unavailable. Therefore, to directly evaluate whether the intermediate flow embeddings follow the Grassfire evolution, we perform an additional comparison against the ground-truth Grassfire trajectory on held-out test polygons:

    • We first run Grassfire on the original unseen polygons from the held-out test set. For each polygon, we record the number of Grassfire steps N, the encoder embedding Zₜᴳᵀ of each intermediate state, and its ground-truth visibility graph Gₜᴳᵀ. We then perform N flow steps conditioned only on the target visibility graph G and store the corresponding intermediate flow embeddings Zₜ. We pass both Zₜᴳᵀ and Zₜ through our trained decoder using Gₜᴳᵀ and compare the decoded polygons with the corresponding ground-truth Grassfire polygon at time t using visibility F1 and Chamfer distance. Gₜᴳᵀ is used only for this evaluation and is not available or used during normal inference for our reported results. We report the mean ± standard deviation across all intermediate states and held-out test polygons.

      ApproachChamfer DistanceVisibility F1
      Zₜᴳᵀ0.24 ± 0.090.83 ± 0.05
      Zₜ0.27 ± 0.090.82 ± 0.05

      We observe that replacing Zₜᴳᵀ with Zₜ changes visibility F1 by only 0.01 (0.83 → 0.82) and Chamfer distance by 0.03 (0.24 → 0.27). Since both embeddings are decoded using the same intermediate visibility graph Gₜᴳᵀ and evaluated against the corresponding Grassfire intermediate polygon, this provides evidence that the intermediate flow embeddings correspond closely to the Grassfire evolution at the same stage.

  • Triangle Source Distribution: The training and inference triangle distributions are different. During training, the initial triangles are the root states obtained from the Grassfire propagation trajectories. During inference, we initialize the flow using randomly sampled triangles.

Reviewer_KCdmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VisFlow, an interpretable flow-based method for generating polygons from visibility graphs. The authors argue that geometric objects combining continuous geometry with discrete combinatorial structures pose significant challenges for representation learning, as small geometric perturbations can lead to abrupt changes in the underlying discrete structure. Polygons provide a representative example, where minor changes in vertex positions can substantially alter the corresponding visibility graph. Existing vertex- and triangulation-based approaches rely on explicit geometric construction but struggle to capture the relationship between geometry and visibility, while signed distance function (SDF)-based methods model this relationship more naturally but lack interpretable intermediate construction steps. To address these limitations, VisFlow models polygon generation through rectified flow in the latent space of continuous SDFs. Rather than learning linear trajectories from an unstructured source distribution, the proposed approach learns approximately linear latent trajectories corresponding to explicit stages of polygon growth along SDF normals.

Critical strengths
  1. S1. Clear presentation and mathematical formulation. The paper is well written and presents a coherent and carefully structured narrative. The figures are informative, the mathematical formulation is interesting, and the motivation behind the proposed approach is clearly explained. Overall, the authors have done a good job of connecting the problem formulation, methodological design, and underlying geometric intuition.
  2. S2. Promising experimental results. The proposed approach demonstrates strong performance compared to the evaluated baselines, suggesting that the flow-based formulation is effective for the considered polygon generation tasks. However, as discussed below, the limited selection of baselines and datasets makes it difficult to assess the generality of these improvements.
Critical weaknesses
  1. W1. Limited comparison with state-of-the-art approaches. One of my main concerns is the lack of comparisons with more recent foundation models and other state-of-the-art generative approaches. While I acknowledge that VisDiff is a relatively recent method, introduced in 2026, it is not clear that it represents the strongest available baseline for this problem. Given the rapid progress in generative modeling, I would expect a broader evaluation against alternative approaches, including modern foundation models where applicable. Such comparisons would help establish whether VisFlow provides meaningful advantages over more general-purpose generative architectures.
  2. W2. Limited experimental baselines and dataset diversity. The experimental evaluation appears somewhat limited in terms of the diversity of datasets and baseline architectures. In particular, I would expect the authors to consider additional architectures commonly used in computer vision, including relevant image-based generative models that could be adapted to polygon generation. Although these methods may not explicitly incorporate visibility constraints, comparing against them would provide useful insights into the benefits of the proposed geometric formulation. Furthermore, evaluating the approach on a broader range of polygon configurations and geometric complexities would strengthen the empirical evidence supporting its generalization capabilities.
Questions for authors
  1. Q1. Comparison with general-purpose coding agents and foundation models. Have the authors considered evaluating whether general-purpose coding agents, such as Claude Code, can solve the same polygon generation problem when provided with the visibility graph and appropriate geometric constraints? Such a comparison could help clarify whether the specialized VisFlow architecture offers advantages over more general-purpose reasoning and code-generation systems, particularly in terms of solution validity, computational efficiency, and generation quality.
2 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their careful evaluation and constructive feedback. We appreciate their recognition of VisFlow’s clear presentation and mathematical formulation. Below, we address the concerns regarding the general-purpose coding-agent comparison and limited comparison with experimental baselines and datasets.

  • Claude Agent Comparison: The reviewer suggested comparing VisFlow with a general-purpose coding agent. Since this request is open-ended, as the visibility reconstruction problem can be posed to a coding agent in several different ways, we interpret the suggestion as asking whether Claude can design an approach that maximizes agreement with the ground-truth visibility graph under the visibility F1 metric used in our evaluation. We therefore use Claude Sonnet 5.5 and provide it with the visibility reconstruction problem and the target visibility graph. We then ask Claude to generate a procedure that reconstructs a polygon by directly optimizing the visibility F1 score with respect to the target visibility graph. Claude produces an iterative simulated-annealing-based optimization procedure starting from a convex polygon. Simulated annealing is a well-established stochastic global optimization method [1,2]. We therefore evaluate how its reconstruction performance changes with the available computational budget. Since VisFlow generates a polygon in approximately 25 ms per instance, we evaluate the Claude-generated procedure using computational budgets of 25 ms, 1 s, 5 s, and 10 s per instance. VisFlow alone achieves a visibility F1 of 0.83 without our proposed inference-time optimization approach.

    Time BudgetClaude-Generated Optimization F1
    25 ms0.61 ± 0.05
    1 s0.95 ± 0.02
    5 s0.97 ± 0.02
    10 s0.98 ± 0.014

    We observe that, under the same computational budget of 25 ms per instance, VisFlow achieves 36% higher reconstruction performance than the Claude-generated optimization procedure (0.83 vs. 0.61 F1). With a larger budget of 1 s per instance, the Claude-generated optimizer reaches 0.95 F1, showing that longer iterative optimization can eventually achieve higher reconstruction accuracy. However, this corresponds to up to a 40× larger computational budget than the 25 ms used by VisFlow in our measured comparison. The results therefore highlight the substantial computational advantage of VisFlow, while also showing that additional optimization time can trade computation for higher reconstruction accuracy.

    Motivated by this trade-off, we next evaluate whether the VisFlow prediction can serve as a stronger initialization for the Claude-generated optimization procedure and help it reach high reconstruction accuracy with a smaller computational budget. We therefore initialize the Claude-generated optimizer with the polygon predicted by VisFlow and evaluate the resulting procedure using computational budgets of 100 ms, 250 ms, and 500 ms per instance.

    Time BudgetVisFlow + Claude-Generated Optimization F1
    100 ms0.82 ± 0.06
    250 ms0.90 ± 0.04
    500 ms0.93 ± 0.03

    Using VisFlow initialization, the Claude-generated optimizer reaches 0.90 F1 in 250 ms and 0.93 F1 in 500 ms, compared with 0.95 F1 after 1 s when optimizing from scratch. Thus, VisFlow initialization reaches 0.90 F1 using only 25% of the 1 s computational budget, and reaches 0.93 F1 using only 50% of that budget, while remaining within 0.02 F1 of the 1 s optimization result. This demonstrates that VisFlow can serve as a strong geometry-aware initialization for the Claude-generated optimizer, enabling high-quality solutions to be reached substantially faster.

  • Limited Experimental Baselines and Datasets: We compare VisFlow on the original VisDiff dataset in Appendix 9.5 and now also provide an additional comparison with a general-purpose coding agent. The Cont-VisDiff baseline used in Appendix 9.8 follows the same latent diffusion architecture as [3], but operates on our continuous SDF latent representation. We do not evaluate on datasets such as MNIST or COCO because they do not capture the key ambiguity of the visibility reconstruction and characterization problem: a single visibility graph can correspond to multiple geometrically different polygons. This one-to-many relationship is central to both visibility reconstruction and characterization. We provide an explicit comparison of diversity between datasets in Appendix 9.10.

[1] Kirkpatrick, S., Gelatt, C. D. Jr., and Vecchi, M. P. “Optimization by Simulated Annealing.” Science, 220(4598):671–680, 1983.

[2] Hajek, B. “Cooling Schedules for Optimal Annealing.” Mathematics of Operations Research, 13(2):311–329, 1988.

[3] Rombach, R. et al. “High-Resolution Image Synthesis with Latent Diffusion Models.” CVPR, 2022.

Thank you for the response. I will keep my current score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Author Response and Manuscript Revisions (1/3)

Dear AC and reviewers,

We sincerely thank all reviewers for their thoughtful comments and constructive feedback. We appreciate the reviewers' recognition of VisFlow's contributions and have carefully addressed their concerns through clarifications, additional experiments, and manuscript revisions. We summarize the key points below.

1. Key Clarifications

  • Exact geometric visibility evaluation (Reviewer V7zk): Reviewer V7zk asked whether the reported visibility F1 scores are computed using exact geometric visibility or the learned visibility predictor. We clarified that **all reported reconstruction F1 scores are computed using exact geometric visibility, while the learned Visibility Prediction Network is used only for training and inference-time optimization.

  • Visibility graph used during inference (Reviewer V7zk): Reviewer V7zk asked which visibility graph is used during inference. We clarified that during normal inference, VisFlow is conditioned only on the final target visibility graph. The intermediate visibility graphs along the Grassfire trajectory are unavailable and are not used during inference; they are used only in our new diagnostic Grassfire–Flow intermediate-state experiment.

  • Ablation of individual components (Reviewer 78bc): Reviewer 78bc asked how much the visibility supervision, latent alignment, and flow-based generative model contribute relative to simpler alternatives. We clarified that these contributions are evaluated through dedicated ablation studies. Appendix 9.7 evaluates the contribution of the visibility loss and inference-time optimization, while Appendix 9.8 compares VisFlow with Cont-VisDiff, which uses the same continuous SDF latent representation but does not use the proposed Grassfire-based trajectory supervision. These experiments separate the contributions of the continuous representation, visibility supervision, Grassfire-based latent alignment, and inference-time optimization.

  • Trajectory construction (Reviewer V7zk): Reviewer V7zk requested clarification on how non-adjacent boundary encounters are handled during Grassfire evolution, how changing vertex cardinality is represented, and how lower-cardinality states are processed by the decoder with 50 learnable queries. We clarified that Grassfire evolution is tracked at the level of individual boundary vertices. As vertices become inactive, intermediate states are padded to the fixed maximum of 50 vertices using a validity mask, with the corresponding visibility graphs padded similarly and losses computed only over valid entries.

  • Protein baseline conditioning (Reviewer V7zk): Reviewer V7zk asked whether the C-VAE and GNN baselines receive the same amino-acid sequence conditioning as VisFlow. We clarified that both baselines use the same amino-acid sequence conditioning as VisFlow, ensuring a fair comparison.

AuthorsOpenReview ↗

Summary of Author Response and Manuscript Revisions (2/3)

2. Additional Experiments

  • Independent-sampling characterization (Reviewer V7zk): Reviewer V7zk raised concerns that the original BFS-based characterization metric measures local perturbation behavior and compares perturbations performed in different source spaces. To address this, we introduced an additional experiment using independent sampling under matched sampling budgets for VisFlow and Cont-VisDiff (Our Approach without the grassfire supervision). As the sampling budget increases from 5 to 50, VisFlow maintains approximately 54% distinct realizations, while Cont-VisDiff decreases from 58.6% to 32.9%. This indicates that additional Cont-VisDiff samples increasingly cluster around previously generated realizations, whereas VisFlow continues to generate distinct valid realizations at a consistent rate.

  • Grassfire–Flow intermediate-state evaluation (Reviewers V7zk and 78bc): To evaluate whether the intermediate states generated by the flow model follow the geometrically meaningful Grassfire evolution, we compare the intermediate flow embeddings with the corresponding ground-truth Grassfire embeddings on held-out test polygons. The intermediate flow embeddings achieve 0.82 visibility F1 and 0.27 Chamfer distance, compared with 0.83 F1 and 0.24 Chamfer distance for the corresponding Grassfire embeddings. These results show that the intermediate flow embeddings remain closely aligned with the corresponding Grassfire states. Reviewer 78bc noted that this experiment more directly demonstrates geometric consistency rather than independent interpretability. We refer to VisFlow as interpretable because its generation process is explicitly structured to follow geometrically meaningful intermediate states rather than arbitrary latent states. Based on the reviewer’s distinction, we are considering replacing “interpretable” with “geometrically consistent” in the title and throughout the paper if this terminology is preferred.

  • Zero-latent control (Reviewer 78bc): Reviewer 78bc further noted that, since the ground-truth intermediate visibility graph is provided to the decoder in the Grassfire–Flow intermediate-state evaluation experiment, it is unclear how much of the reconstruction quality can be attributed to the latent embedding itself. To isolate this contribution, we perform an additional zero-latent experiment, replacing the intermediate flow embedding with a zero vector while providing the decoder with the same intermediate visibility graph. Visibility F1 decreases from 0.82 to 0.60, while Chamfer distance increases from 0.27 to 0.50. This shows that the intermediate visibility graph alone does not account for the reconstruction quality and that the intermediate flow embedding contributes substantial geometric information corresponding to the Grassfire state.

  • General-purpose coding-agent comparison (Reviewer KCdm): Following Reviewer KCdm's suggestion, we use Claude Sonnet 5.5 to design an optimization procedure for visibility reconstruction. Under the same computational budget of 25 ms per instance, VisFlow achieves 0.83 F1, compared with 0.61 F1 for the Claude-generated optimizer, corresponding to a 36% relative improvement. With a larger budget of 1 s, the Claude-generated optimizer reaches 0.95 F1. We further initialize the Claude-generated optimizer with the VisFlow prediction, obtaining 0.90 F1 in 250 ms and 0.93 F1 in 500 ms, compared with 0.95 F1 after 1 s when optimizing from scratch. This demonstrates that VisFlow provides a strong geometry-aware initialization for the optimization procedure.

  • MDS protein reconstruction baseline (Reviewer V7zk): Following Reviewer V7zk's suggestion to include a contact-map-based reconstruction baseline, we additionally evaluate an MDS-based method. VisFlow achieves 0.86 contact-graph F1 compared with 0.83 for MDS, while reducing Cα-RMSD from 7.4 Å to 6.3 Å, corresponding to approximately a 15% reduction in geometric reconstruction error.

AuthorsOpenReview ↗

Summary of Author Response and Manuscript Revisions (3/3)

3. Manuscript Changes

We have incorporated the following changes into the revised manuscript:

  • Exact geometric visibility evaluation: Clarified in the evaluation metrics in Section 5 that all reported visibility metrics are computed using exact geometric visibility.
  • Protein baseline: Added the MDS-based contact-map reconstruction baseline to Table 3.
  • Independent-sampling characterization: Added the matched-budget independent-sampling experiment to Appendix 9.8.
  • Grassfire–Flow intermediate-state evaluation: Added the intermediate-state comparison experiment to Appendix 9.9.
  • Claude agent comparison: Added the general-purpose Claude agent comparison to Appendix 9.11.
  • Introduction citations (Reviewer 78bc): Reviewer 78bc noted that statements in the introduction were not supported by references. We have added supporting citations to the relevant statements in the introduction.
  • Presentation corrections: Corrected the typographical and presentation issues identified by Reviewer V7zk.
  • Dataset diversity clarification (Reviewer V7zk): Reviewer V7zk noted that the previously stated 22% greater shape diversity specifically corresponds to the maximum link diameter of the dataset. We have revised the introduction and conclusion to explicitly state that our dataset has a 22% greater maximum link diameter than the state-of-the-art dataset, clarifying the statistic used to characterize higher diversity.

We will also use the additional page available in the final camera-ready version to include a concise description of the dataset construction and its key characteristics in the main paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.