acceptodds
Withdrawn from ICLR 2027

Texture Interpolation as Texton Matching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Texture interpolation asks what should lie between two textures. Standard methods typically blend pixels or features at corresponding spatial locations, implicitly assuming that co-located elements should be mixed. For unrelated textures, however, this correspondence is arbitrary: a crack at one location has no particular relationship to a fiber at the same location in another image. We study whether this hidden pairing choice is itself a major determinant of interpolation quality. We represent textures as collections of local appearance vectors, or textons, and replace positional pairing with appearance-aware matching while keeping the encoder, representation, and decoder fixed. This isolates the effect of correspondence from changes in model capacity or training. Across texture pairs, appearance-aware matching consistently improves the distributional quality of interpolated midpoints over positional feature blending. Soft matching produces the strongest midpoint statistics, whereas one-to-one assignments yield more balanced transitions and closer agreement with the target endpoint. The same intervention also improves a second pretrained texture interpolator without retraining. These results identify correspondence as a first-class design choice in texture interpolation: meaningful transitions depend not only on how strongly two textures are mixed, but also on which local appearances are mixed with one another.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
texture interpolation, optimal transport, correspondence, texton matching, texture synthesis, compositional neural textures, distribution alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LYZqOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates the problem of texture interpolation (morphing between two textures). The authors argue that standard methods, which blend pixels or features based on spatial correspondence, fail for unrelated textures because spatial alignment is arbitrary, resulting in "ghosting" or superimposed transparent patterns. To address this, they reframe texture interpolation as a texton matching problem. By representing textures as sets of local appearance vectors (textons) using pre-trained encoders (such as Compositional Neural Textures [CNT] and Texture Mixer), they employ Optimal Transport (soft entropic matching) to pair textons based on appearance rather than spatial location. Extensive controlled experiments demonstrate that this appearance-aware soft matching significantly improves the distributional quality and perceptual realism of interpolated midpoints compared to naive positional blending, all without requiring any retraining of the underlying generative decoders.

Critical strengths
  1. Elegant Conceptual Reframing: The core insight that texture interpolation is fundamentally a correspondence (matching) problem rather than a simple blending problem is highly intuitive and well-argued. Shifting from spatial alignment to distributional alignment via Optimal Transport provides a theoretically sound solution to the artifacts inherent in positional blending of unregistered textures.

  2. Rigorous Isolation of Variables: The experimental design is exceptionally controlled. By freezing the encoder, representation, and decoder and only altering the pairing rule (positional vs. random permutation vs. soft OT vs. exact OT), the authors cleanly isolate and prove the specific contribution of appearance-aware matching. The mathematical analysis regarding barycentric projection and variance contraction adds solid theoretical grounding to the empirical observations.

Critical weaknesses
  1. Lack of Meaningful and Modern Baselines: The baselines used to validate the method are weak and outdated. The primary comparisons are against naive RGB cross-fading, positional feature blending (field-linear), and optimization-based statistical matching from 2020 (Vacher et al.). While the authors include "Texture Mixer" (a 2019 GAN-based method) and a prompted generative model (GPT-Image-1.5) as a non-parametric upper bound, they completely ignore the massive recent advancements in image morphing, frame interpolation, and latent space interpolation using modern Diffusion Models or Flow Matching. Hence, I would like to say that, because the baselines are largely strawmen (naive linear interpolations in pixel or feature space), it is impossible to judge whether "texton matching" actually advances the state-of-the-art in texture interpolation or merely outperforms trivial linear baselines. A modern latent diffusion model interpolating in its native VAE latent space, or using attention-injection/morphing techniques for nature images, would be a much more meaningful and competitive baseline.

  2. Unclear Purpose and Practical Utility of Texture Morphing: The paper treats texture interpolation as an isolated mathematical exercise without adequately motivating why this task is important or what its downstream applications are. In modern graphics, procedural generation, or material design, practitioners rarely need to "morph" two unrelated 2D texture images into a midpoint. If a designer wants a new material, they typically use procedural noise generators, physically-based material mixing nodes in rendering engines, or simply prompt a modern Text-to-Image model. The paper fails to identify a concrete use case where taking two arbitrary, unregistered texture patches and computing their "Wasserstein barycenter" via a frozen CNT decoder is practically useful.

Questions for authors
  1. How does your texton matching approach compare to modern diffusion-based image morphing or latent interpolation techniques (e.g., interpolating in the latent space of Stable Diffusion or using attention-sharing methods)?

  2. What is the specific downstream application or practical graphics/vision pipeline that necessitates "texture morphing" between two unrelated, unregistered images?

Reviewer_xsbbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that texture interpolation is not only about how strongly two textures are blended, but also about which local texture elements are paired. Using a frozen Compositional Neural Textures decoder, the authors represent images as dense local appearance vectors ("textons") and replace positional pairing with appearance-aware matching. The main method uses balanced entropic optimal transport to define a soft partner map; the paper also studies exact OT, Gaussian moment alignment, target-mean maps, random permutation, and positional baselines.

Validation is based on DTD texture pairs, KID/FID against reference texture sets, per-image realism metrics, balance scores, matched-rendering controls, transfer to Texture Mixer without retraining, and comparison to GPT-Image-1.5 prompted midpoint generation.

Critical strengths
  1. The paper isolates the effect of pairing under fixed encoders and decoders. This is a clean experimental design and makes the paper more informative than simply proposing another interpolation model.

  2. The authors provide multiple controls that answer different failure hypotheses: random permutation for coverage without appearance, Gaussian alignment for moment matching, exact OT for one-to-one matching, and a second pretrained decoder for transfer. This gives the core claim more credibility.

Critical weaknesses
  1. The main evidence for interpolation "quality" comes from FID/KID, realism, and balance metrics, none of which directly measures whether humans perceive a coherent and semantically balanced transition between the two endpoints. The authors acknowledge this, but without even a small user study the perceptual conclusion remains too limited for acceptance.

  2. Some improvements suggest that coarse distribution alignment, not texton-level correspondence per se, may explain much of the gain. Gaussian alignment and target-mean variants are competitive in several settings. The paper should sharpen the claim: what exactly does soft texton matching add beyond moment matching or diffuse distribution alignment? As written, the central contribution is interesting but not yet decisive.

Questions for authors
  1. Would a human preference study on midpoint quality and path coherence agree with the KID/FID ranking, especially for Soft OT versus Gaussian alignment and Texture Mixer?

  2. How stable are the conclusions across interpolation coefficients beyond t = 0.5? Are there cases where midpoint metrics improve but the full path is visually non-monotone or direction-dependent?

  3. Can the authors compare more directly against CNT's native primitive matching, not only field-linear and primitive-linear references?

  4. What are representative failure cases for periodic textures, self-interpolation, and reversed direction? These would help bound the claim.

Reviewer_gKVrOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper Texture Interpolation as Texton Matching, investigates the role of feature correspondence in texture interpolation. Instead of blending spatially aligned features, the proposed approach matches local appearance descriptors using appearance-aware matching, within a frozen encoder-decoder framework. Experiments on CNT and Texture Mixer demonstrate improvements in the distributional quality of interpolated textures without additional training.

Critical strengths
  • The paper presents an interesting perspective on texture interpolation by treating correspondence as an explicit design choice rather than relying on spatial alignment.
  • The method is training-free and computationally efficient.
Critical weaknesses
  • Overall this paper lacks methodological novelty. Optimal transport and feature distribution alignment have been extensively studied in texture synthesis and style transfer.
  • It seems the method cannot convince me that the performances are and the proposed method is essential.
  • The evaluation is overall weak and there lacks a detailed human preference study.
Questions for authors
  • How does the method represent the endpoint images and fusion results close to the two images?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.