acceptodds
Under review as a conference paper at ICLR 2027

Solver-JEPA: A Joint-Embedding Predictive Architecture for Visual Problem Solving

Abstract

Visual problem solving requires transforming visually presented problems into structured solutions under global constraints and objectives. While methods learn to generate solution images or reasoning trajectories, visual problem solving still lacks a representation-learning objective designed to organize the information required for solution construction. Such learning is challenging because the visually observed structure and the underlying decision structure are not equivalent: small local differences can induce discontinuous changes in global feasibility or optimality, while a final solution image reveals only one successful endpoint and leaves the consequences and interactions of alternative decisions unspecified. We introduce **Solver-JEPA**, a solver-supervised joint-embedding predictive architecture that turns each problem into a family of decision-conditioned prediction tasks. During training, partial commitments specify tentative local decisions, and task solvers provide targets describing their consequences for problem structure, global feasibility, attainable solution quality, and compatible complete solutions. Predicting these targets in representation space encourages the visual encoder to preserve solution-relevant local-to-global dependencies. A separately trained decoder reuses the frozen representation to construct complete solution images without solver access at inference. Across four visual problem solving tasks, Solver-JEPA improves strict solution accuracy by up to **11.6** percentage points and provides **15**–**150 times** faster inference than generation-based baselines. Controlled ablations show that joint-embedding alignment and composite-commitment supervision improve the readability of complete solutions from the learned representations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
joint-embedding predictive architecture, Visual Problem Solving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_x8DGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies representation learning for visual problem solving, where images encode constrained planning, optimization, and satisfaction problems.

The so-called Solver-JEPA trains a visual encoder to predict solver-derived consequences of partial commitments, then trains frozen-representation decoders to render solutions.

The paper evaluates Solver-JEPA on Maze, TSP, Sudoku, and FrozenLake/VSP tasks.

Critical strengths

The paper proposes a concrete way to move solver information into representation learning rather than using it only to supervise an end-to-end solution generator. The improvements also show up in the actual task results. Solver-JEPA gets better results on Maze-32, TSP-18, and Sudoku-30. The TSP ablations are also useful: they provide some evidence that both embedding alignment and composite commitments help the frozen representation retain information useful for constructing complete solutions.

Critical weaknesses

I have a serious concern about reproducibility. The method is described at a high level, but key task-specific details are missing. The appendix mentions task-specific query schedules, loss weights, architectural choices, and auxiliary losses, but does not actually spell them out in enough detail. This is particularly problematic for a representation learning paper, where these training choices are central to the method and can have a significant impact on the results.

On the writing and presentation side, there are quite a few half-empty lines, and the appendix is very short and lacks detail. These issues do not really affect my assessment of the paper, but I think the authors should give them some attention to make the submission more polished and complete.

Questions for authors

For each task, what are the exact query-type proportions, commitment-size distributions, relation/witness target encodings, and loss weights used during representation training? Appendix A and Appendix E defer these details to task configurations that are not included.

4 comments

We are grateful for your thoughtful review and recognition of our use of solver information in representation learning, performance gains in challenging settings, and informative TSP ablations. The items you listed for representation training, from query schedules and commitment-size distributions to target encodings, architectures, and loss weights, are now documented in the revised Appendices A–G, as summarized in W1/Q1 below. The revision also extends the TSP ablations with a leave-one-out study of the five query families (Table 3), and adds comparisons with zero-shot Nano Banana 2 (Table 1) and frozen CLIP and DINOv2 encoders (Table 5), as well as evaluations on five out-of-distribution settings (Table 4).

W1 / Q1: Reproducibility and missing task-specific training details, including query schedules and proportions, commitment-size distributions, relation/witness target encodings, architectural choices, loss weights, and auxiliary losses.

Thank you for raising this point. We agree that these choices are central to a representation-learning paper. The revised manuscript expands Appendices A–F and adds Appendix G, which together document the task-specific settings used in our experiments, and we will release the code, task configurations, and data-generation scripts upon publication. Appendices A and E no longer defer to unreleased task configurations: Appendix A now points to the per-task specifications in Appendix G, and Appendix E lists the optimization settings of every task. We summarize the answers to your question below.

Shared setup (Appendices C–E). Labels are computed offline with exact solvers and deterministic procedures (e.g., enumeration or MILP with subtour cuts for TSP, BFS for Maze). Representations are trained for 100 epochs with AdamW (learning rate 1e-4, weight decay 0.05, 4-epoch warmup and cosine decay, gradient clipping 1.0, bf16; Sudoku exceptions in Appendix E) at a global batch of 32 on four A800 GPUs. The objective is L=Lbase+λqLqueryL = L_{\mathrm{base}} + \lambda_q L_{\mathrm{query}} (λq=0.25\lambda_q = 0.25; 0.5 for Sudoku), where LbaseL_{\mathrm{base}} supervises the original problem and LqueryL_{\mathrm{query}} supervises commitment-conditioned queries, including the empty commitment. In LbaseL_{\mathrm{base}}, the alignment and predicted-side semantic losses have weight 1.0, the target-side semantic loss 0.2, and the variance regularizer 0.02 (1.0 and 0.05, respectively, for Maze and FrozenLake), and Sudoku adds a covariance term (0.01). Query types are supervised jointly rather than sampled exclusively. For infeasible commitments, WITNESS and the cost-based VALUE targets are masked (the Sudoku burden remains defined), and every loss is averaged over its valid entries (Appendix C.1).

TSP (Appendix G.1). A commitment has 1–4 decisions, each including or excluding an undirected edge or requiring a directed successor. Each image caches all 2n(n−1)2n(n-1) single-decision commitments, 24 random commitments of size 2–4 (uniform), and three rule-based infeasible ones, with all their subsets. Each step supervises all cached FEASIBILITY, VALUE, and MEMBERSHIP labels (the latter for up to 32 commitments) and one WITNESS from the empty, single-, or multi-decision pool, in a four-epoch 1:1:2 rotation. RELATION encodes pairwise city distances, VALUE the relative increase in optimal tour length, and WITNESS the constrained-optimal tour as a directed successor matrix, which MEMBERSHIP contrasts with a feasible but longer tour. The weights are 1.0 for VALUE and WITNESS, 0.25 for each of two interaction terms (non-additive effects on VALUE), 0.2 for FEASIBILITY and MEMBERSHIP, and 0.1 for RELATION.

Maze and FrozenLake (Appendix G.2). A commitment blocks 1–3 cells or passages. Each image caches 32 random single blocks and 16 random commitments of size 2 or 3 (equally likely), with all their subsets; each step uniformly draws two per image and supervises all query types on both. RELATION encodes the remaining valid moves, reachability, and distance to the goal; VALUE the relative increase in path length (zero on Maze whenever a path remains, since tree mazes have a unique path); and WITNESS the shortest path under the commitment as a cell mask with move or STOP labels, which MEMBERSHIP contrasts with a truncated or reversed original path. All LqueryL_{\mathrm{query}} weights are 1.0, except 0.2 for the target-side semantic term.

Sudoku (Appendix G.3). A commitment has 1–4 decisions on empty cells, each initially assigning (70%) or banning (30%) a digit. Each image caches one family per size k=1k = 1–44 with all 2k2^k subsets (30 subset records); with probability 0.65, one decision is replaced by one conflicting with the unique solution, making the full commitment infeasible. Each step supervises all query types on all subsets of one family per image, with kk rotating across epochs. With no cost to optimize, VALUE is the normalized excess-candidate burden after 0, 1, and 4 propagation rounds (1 upon contradiction), RELATION encodes these candidate digits plus a contradiction flag, and WITNESS is the completed board, which MEMBERSHIP contrasts with boards that swap two digits globally or change one cell. All LqueryL_{\mathrm{query}} weights are 1.0, except 0.2 for the target-side semantic and interaction terms.

Architecture and auxiliary losses. TSP, Maze, and FrozenLake use the encoder architecture of Appendix B. The Sudoku encoder, now also detailed there, processes the 81 cell crops on the fixed exact-cover graph of the Sudoku rules into a 1024×384 state of 729 assignment, 81 cell, and 214 constraint-summary tokens. Appendices B, F, and G give the remaining details: the auxiliary terms of LbaseL_{\mathrm{base}} (e.g., tour ranking and degree terms for TSP, Bellman consistency for Maze), predictor and target-encoder sizes, decoder architectures and image losses, and sampling settings.

W2: Presentation issues.

Thank you for highlighting these issues. We have expanded the appendix as described above in the revised manuscript and will also tighten the layout of the main text in the final version.

We hope these clarifications and the expanded appendix address your concerns, and we would be glad to discuss any remaining questions.

This rebuttal addresses most of my concerns. I will keep my score.

Thank you for your feedback and for your continued support. We are glad that the rebuttal addressed most of your concerns. If any specific point remains unclear, we would be happy to clarify it or to add the corresponding details to the final version.

Reviewer_Kkm9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies visual problem solving, covering visual-spatial planning, Maze path planning, Sudoku, and the traveling salesman problem (TSP). It studies these tasks using a two-stage model: first, it learns a compact image encoder using I-JEPA to enforce a visual representation that learns structural solvers. The second stage learns to decode answers from the learned image encoder using a Deterministic decoder and diffusion models. They report almost 100% accuracy on visual problem-solving benchmarks.

Critical strengths
  1. Experimental results across multiple visual-spatial planning tasks, such as Maze path planning, Sudoku, TSP, show that the learned visual representation benefits answer decoding.

  2. They design several different types of commitment to change one visual problem into several different types of visual problems, such as if I enforce the condition “you have to travel xxx and xxx path”, then the optimal path changes or doesn’t exist (binary question), with these expanded sub-questions, they use I-JEPA to learn an image encoder that has to learn solving structure.

  3. Expanding one visual problem into several structured visual problems is automatic, without requiring human effort.

Critical weaknesses
  1. The paper compares the I-JEPA-style image encoder only with itself, not with generic image encoders such as DINO and CLIP; why can't existing image encoders fulfill this task, and do we need I-JEPA to do it?

  2. There are around 5 types of sub-visual problems; are all 5 types necessary for the I-JEPA model?

  3. Training an image encoder with more decomposed subproblems requires extra computational effort; does this improve performance?

Questions for authors

My main concerns are the three weaknesses mentioned.

  1. Is it necessary to train an image encoder in I-JEPA style? Why can't an off-the-shelf generic image encoder fulfill this task?

  2. Are all of the 5 types of sub-visual problems necessary?

  3. Does training image encoders with more sub-visual problems increase computation for performance?

2 comments

We are grateful for your constructive feedback and recognition of our learned representations and automatic construction of structured subproblems from partial commitments. To address your three main concerns, we have added new experiments to the revised manuscript:

  • With the same decoder, our encoder outperforms frozen CLIP and DINOv2 by 57.8 and 45.3 points on average (W1).
  • A leave-one-out study shows that WITNESS is essential and that VALUE and MEMBERSHIP add 3–4 points in optimal-tour rate (W2).
  • The additional cost is confined to one-time encoder training and does not affect decoding or inference (W3).

W1 / Q1: Comparison with generic image encoders such as DINO and CLIP, and whether I-JEPA-style training is necessary.

Thank you for suggesting this ablation. We first clarify that Solver-JEPA is not I-JEPA. Both predict targets in representation space. I-JEPA predicts embeddings of masked image regions from visible context using an EMA target encoder. Solver-JEPA takes the full problem image as input and uses separate target encoders to embed solver-derived answers about how partial commitments affect structure, feasibility, and attainable solution quality. In Table 3, an image-only JEPA variant (the variant closest to I-JEPA) that predicts solution-image embeddings with an EMA target encoder achieves a frozen-probe optimal-tour rate of 7.40% on TSP, compared with 62.80% for Solver-JEPA. Table 3 also isolates the joint-embedding component itself. With all solver-derived targets kept, removing the embedding-alignment loss lowers the optimal-tour rate from 62.80% to 45.53%. With the original-problem solution as the only target, predicting it directly without a target encoder reaches only 31.13%, compared with 58.33% when its embedding is predicted.

Comparison with generic encoders. Following your suggestion, we evaluated two off-the-shelf alternatives to our encoder on Maze and TSP under the Table 1 protocol: CLIP ViT-B/16 (openai/clip-vit-base-patch16) and DINOv2 ViT-B/14 (facebook/dinov2-base). We froze each encoder, aligned its patch features to a common 32 × 32 grid, and projected each token to the AE input width of 384. We trained the projection jointly with the AE decoder using the same decoder architecture and training settings as in our original experiments (Appendix F.3). We have added this comparison to Sec. 5.4 (Table 5) of the revised manuscript and will extend it to the remaining tasks and the U-Net decoder in the final version.

Encoder (+ AE)Maze-8Maze-16Maze-32TSP-12TSP-15TSP-18Avg.
CLIP ViT-B/16244064564231.7
DINOv2 ViT-B/14689475644544.2
Solver-JEPA (ours)100988688897689.5

With the same AE decoder architecture and training settings, Solver-JEPA outperforms DINOv2 and CLIP by 45.3 and 57.8 percentage points, respectively, averaged over six settings. Our 15.1M-parameter encoder is trained from scratch on 27K images per task, while both baselines use roughly 86M parameters and large-scale external pretraining. On Maze-16 and Maze-32, Solver-JEPA reaches 98% and 86% accuracy, compared with 9% and 4% for DINOv2, the stronger baseline. On TSP, its lead over DINOv2 grows from 13 points at 12 cities to 31 at 18. These results suggest that generic features, learned to recognize visual content, do not expose the decision structure needed to solve these problems, such as maze connectivity and the edges of an optimal tour. This supports the motivation of Solver-JEPA: visual problem solving benefits from representations trained to capture solution-relevant structure, which the tested generic encoders do not provide under the same decoding protocol.

W2 / Q2: Necessity of all five subproblem types.

Thank you for this question. The five types of sub-visual problems correspond to our five query families, and we tested their necessity with a leave-one-out study on TSP. For each family, we trained a new encoder with all of its supervision removed, both on the original problem and under commitments, and evaluated the frozen representation with newly trained probes, as in Table 3. Every variant is compared with a full model trained under the same protocol, in which the solution loss treats all non-tour edges equally instead of weighting them by their regret, a VALUE quantity. This ensures that removing VALUE also removes the regret information from the solution loss, but makes the absolute numbers differ from those in Table 3(a).

Removed familyOpt. tour (%) ↑Value MAE ↓Member BA (%) ↑
None (full model)70.000.019697.86
RELATION71.330.019397.67
FEASIBILITY69.670.019397.71
VALUE67.000.019897.62
WITNESS12.670.023096.57
MEMBERSHIP65.670.019797.73

Not all five families are necessary on every task. On TSP, WITNESS is essential: removing it lowers the optimal-tour rate from 70.00% to 12.67% and also yields the worst Value MAE and Member BA, even with VALUE and MEMBERSHIP supervision retained. This is because the optimal tour under a commitment, which WITNESS predicts, also determines its cost increase and which candidate tours are optimal. Removing VALUE or MEMBERSHIP therefore barely changes its own probe but lowers the optimal-tour rate by 3.00 and 4.33 points, respectively, suggesting that both sharpen exact solution readout. Removing RELATION or FEASIBILITY changes the rate by at most 1.33 points, as TSP distances are directly visible and almost all TSP commitments are feasible. We keep both in the common query interface because, in Maze and Sudoku, they supervise reachability and constraint propagation, which are not directly visible. We have added this study to Sec. 5.4 (Table 3(b)) and Appendix E, and will add the same study for the other tasks to the appendix in the final version.

W3 / Q3: Additional computational cost and performance benefits of training with more subproblems.

The additional cost is confined to representation learning. We compute the solver targets for all visual subproblems once, offline, at a cost of approximately 1–3 CPU-hours per task. During training, each image is encoded once, and all of its selected queries reuse the same representation. After representation learning, the encoder is frozen: the AE and U-Net decoders are trained on the frozen representation, and inference runs only the encoder and a decoder (plus, for Sudoku, the frozen structural adapter), with no queries or solver calls. Decoder training and inference therefore cost the same regardless of which query families are used.

Measured cost. On TSP, we timed 1,000 training steps for the full model and for each leave-one-out variant of W2/Q2, one variant at a time, under identical settings on the same four A800 GPUs. Removing FEASIBILITY, MEMBERSHIP, or RELATION individually changes the average encoder training time per step by less than 8% relative to the full model, suggesting that each family has a modest impact on training cost. The cost is concentrated in VALUE and WITNESS, which account for 23% and 35% of the step time.

Removed familyTime per step (s)Time saved vs. Full
None (Full)0.6715–
FEASIBILITY0.6781−0.99%
MEMBERSHIP0.65093.07%
RELATION0.62347.16%
VALUE0.517222.98%
WITNESS0.437534.85%

Cost and performance. We measured the average encoder training time per step for each of the three commitment settings in Table 3. Supervising only the original-problem solution (empty c) reduces step time by 65% relative to the full design, saving an estimated 10 hours over a 15-hour training run. Its frozen-probe optimal-tour rate, however, drops from 62.80% to 58.33%. Restricting nonempty commitments to first order keeps the per-step cost close to that of the full design, yet the optimal-tour rate reaches only 56.87%. At a comparable per-step cost, the full design also supervises composite commitments and achieves the highest optimal-tour rate (62.80%), a gain of 5.93 percentage points over the first-order variant. This additional cost is incurred once per task during encoder training. Combined with W2/Q2, WITNESS is the most costly and the only essential family, VALUE adds 3.00 points at 23% of the step time, and MEMBERSHIP adds 4.33 points at only 3%.

Regarding presentation, we have stated the distinction from I-JEPA explicitly in Sec. 4.2, and the revision documents the semantics of the five query families (Table 6), their task-specific configurations (Appendix G), and all training settings in Appendices A–G, with all main-text changes marked in blue. We will further streamline the method section in the final version. We would be glad to discuss any remaining questions and would be grateful if you would reconsider your rating.

Reviewer_imz1OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a method that builds on JEPA to generate solution images for visual problem solving. To this end, the authors propose a two-stage framework: the first stage employs JEPA to learn latent representations predictive of solution images, and the second stage trains a decoder to translate these representations into pixel images. Evaluations are conducted on four representative visual reasoning tasks (Maze, TSP, Sudoku, and VSP), and several interesting findings are reported.

Critical strengths
  1. The paper introduces a new JEPA-based method for visual problem solving.
  2. The two-stage learning paradigm is intuitive, though it relies on task- and query-specific target construction.
Critical weaknesses
  1. The evaluation is limited. First, it lacks a comparison with frontier image editing models such as Nano Banana. Second, including a task-specific fine-tuned version of Qwen-Image-Edit would enable a more comprehensive comparison.
  2. The proposed two-stage training method requires task- and query-specific label construction. Moreover, the improvement over existing methods appears limited; for example, Table 1 shows that EndoCot achieves the best performance in 11 settings, while the proposed method does so in only 10.
Questions for authors
  1. A citation is missing on line 048.
  2. The fundamental differences and connections between the proposed method and diffusion image models need clarification. Both learn latent representations that are subsequently decoded into solution images, but diffusion image models seem more elegant to me, as they do not require task- and query-specific label construction. Could the authors provide a more detailed comparison of the two approaches in terms of model size, computational cost, learning efficiency, out-of-domain generalization (e.g., to unseen maze geometries and scales), and other relevant factors? Without this discussion, the motivation for using JEPA for visual problem solving remains unclear.
3 comments

We thank the reviewer for the careful review and for recognizing our JEPA-based approach and intuitive two-stage design. To address your concerns, the revised manuscript adds the following results:

  • Zero-shot Nano Banana 2 averages 39.4% over the 18 settings of Table 1, versus 93.6% and 93.7% for our two decoders, and task-specific fine-tuned Qwen-Image-Edit models are already among our baselines (W1).
  • Under strict verification, our U-Net decoder outperforms EndoCoT by 9–10 points on three of the four challenging settings, and our solver targets take 1–3 CPU-hours per task to construct and are used only for representation learning (W2).
  • Of five new out-of-distribution settings, both of our decoders outperform all baselines on four (TSP-19/20 and Sudoku-28/25), and our 21–86M-parameter pipelines run 30–150× faster than EndoCoT on TSP-18 (Q2).

W1: Comparisons with frontier image editing models such as Nano Banana and task-specific fine-tuned Qwen-Image-Edit.

Task-specific fine-tuned Qwen-Image-Edit models are already among our baselines in Table 1. DiffThinker and DiffThinker++ are per-task LoRA fine-tunes of Qwen-Image-Edit-2509 and Qwen-Image-Edit-2511, respectively (30K instances per task), and EndoCoT fine-tunes Qwen-Image-Edit-2511 with additional latent-reasoning supervision (30K–75K instances per task). Solver-JEPA uses 27K instances per task (DiffThinker's configuration with 10% held out for validation) and no external pretraining.

Table 1 also includes zero-shot Qwen-Image-Edit-2511, a strong open-source image editor. Following your suggestion, we further evaluated Nano Banana 2 (Gemini 3.1 Flash Image) zero-shot via its API, using the same evaluation settings as in Table 1. Nano Banana 2 nearly solves the small VSP grids (94–100% on VSP-3 to VSP-8) but degrades sharply on larger or more constrained settings: 0% on Maze-16/32 and Sudoku-35/30, 3–13% on TSP, and 18–24% on VSP-Super. Both of our decoders match or exceed it on all 18 settings, with average accuracies of 93.6% and 93.7% versus 39.4%. These results have been added to Table 1 of the revised manuscript.

ModelMaze-8Maze-16Maze-32TSP-12TSP-15TSP-18Sudoku-45Sudoku-40Sudoku-35Sudoku-30VSP-3VSP-4VSP-5VSP-6VSP-7VSP-8VSP-Super-16VSP-Super-32Avg.
Nano Banana 21600131132216009898100989894241839.4
Ours (AE)100988688897610010099741001001009910099977993.6
Ours (U-Net)100988489858010010099731001001009810099988393.7

W2: Task- and query-specific label construction and limited performance gains over existing methods such as EndoCoT.

We agree that Solver-JEPA requires supervision beyond problem–solution pairs. Only the representation stage uses the additional query-conditioned targets. We then train the decoders on frozen representations using paired problem–solution images and task-specific auxiliary losses (Appendix F). Inference requires neither queries nor a solver. The five query types share definitions across tasks, and task-specific solvers compute their targets offline without human annotation (Appendices A and G). This incurs a one-time cost of approximately 1–3 CPU-hours per task (Appendix D.2). DiffThinker and EndoCoT also train on solver-generated ground-truth solutions. EndoCoT additionally decomposes these solutions into intermediate targets and uses textual supervision to align its final latent thought state with an embedding of the ground-truth reasoning steps. Our response to Reviewer Kkm9 (W2/Q2) quantifies the contribution of our additional supervision.

For performance improvement, the 11-versus-10 count includes seven settings where EndoCoT and both our decoders achieve 100% accuracy. Head-to-head, our AE decoder outperforms EndoCoT by 3–12 percentage points on five settings (TSP-12/15/18 and Sudoku-35/30), trails by 2–6 points on four (Maze-16/32 and VSP-Super-16/32), and ties at 99–100% on the remaining nine. Our U-Net decoder leads by 4–12 points on the same five settings, trails by 1–6 points on five others, and ties on the remaining eight. The strict image-level verifier (Table 2 of the original manuscript) rejects rendering artifacts that the official parsers can miss. Under this verifier, the same U-Net scores 79.4%, 73.2%, and 75.1% on Maze-32, TSP-18, and Sudoku-30, exceeding EndoCoT by 9.3, 9.9, and 9.5 points, respectively. It trails by 1.7 points on VSP-Super-32 (76.4% vs. 78.1%). Solver-JEPA yields substantial gains in complete-solution correctness on three of the four challenging settings and most new out-of-distribution settings (Q2), with 30–150× faster inference than EndoCoT on TSP-18.

Q1: Missing citation on line 048.

Thank you. We have added the missing citation in the revised manuscript.

Q2: Motivation for JEPA and its relationship to diffusion models, including comparisons of target construction, model size, computational cost, learning efficiency, and out-of-domain generalization.

Both diffusion-based approaches and our design decode solution images from learned representations, but they differ in where problem-solving structure is learned. Representation learning is a standard foundation in visual recognition and vision-language understanding, whereas VPS has mostly been approached from the output side: DiffThinker and EndoCoT adapt pretrained image editors to generate solutions, so decision-relevant structure may remain entangled in the generation process. We instead study representation pretraining for VPS, in which an encoder is first trained to capture solution-relevant structure and solutions are then decoded from the frozen representation. Rather than predicting solution images, Solver-JEPA predicts how tentative decisions affect a problem's structure, feasibility, and attainable solution quality, using automatically generated solver targets (see W2). The two paradigms are complementary: our U-Net decoder is itself a diffusion model, and the same frozen representation supports both deterministic and diffusion decoding.

Diffusion editors indeed avoid query-specific labels, but all of their supervision is derived from the ground-truth solution, which shows only one successful endpoint. Because visual structure and decision structure are not equivalent, the consequences of alternative decisions remain unspecified, and a representation learned this way may capture how a problem looks without exposing what is needed to solve it. Within our framework, Table 3(a) isolates the two ingredients on TSP, decision-structured targets and their prediction in embedding space. An image-only JEPA variant trained on problem–solution image pairs reaches a frozen-probe optimal-tour rate of only 7.40%, versus 62.80% for the full design. Predicting the original-problem solution directly reaches 31.13%, versus 58.33% when its embedding is predicted, and keeping all solver-derived targets while removing only the embedding-alignment loss lowers the full design from 62.80% to 45.53%. The benefit carries over to end-to-end solving on an unseen problem size: on TSP-18 (Fig. 3), our U-Net decoder produces strictly valid tours at almost the same rate as EndoCoT (77.2% vs. 77.8%), but 94.8% of its valid tours are globally optimal, versus 81.4% for EndoCoT and 61.1–62.3% for DiffThinker/DiffThinker++. Since both produce valid tours at similar rates, the higher optimality suggests that our representation captures not only the rules of a valid tour but also the decision structure that determines the optimal one.

Model size, cost, and learning efficiency. Our models are trained from scratch: the encoder has 15.1M parameters for TSP, Maze, and FrozenLake and 49.3M for Sudoku (Appendix B), and the complete inference pipelines have 21.3–66.6M parameters with the AE decoder and 34.8–85.7M with the U-Net decoder, whereas the baselines adapt a 20B MMDiT with additional VLM and VAE components. Each task uses 27K training instances, versus 30K–75K for the baselines on top of their large-scale pretraining. Training takes about 15 hours of representation learning plus 15 hours (AE) or 30 hours (U-Net) of decoder training on four NVIDIA A800 GPUs (Sec. 5.1), i.e., about 120 or 180 GPU-hours per task. Because the baselines start from a pretrained 20B model and use different supervision, these figures do not isolate relative sample efficiency, whereas inference latency can be compared directly on the same hardware: on TSP-18 (Fig. 3), average latency is 0.2 s (AE) and 1.0 s (U-Net), versus 15.0 s for DiffThinker/DiffThinker++ and 30.0 s for EndoCoT, whose accuracy relies on iterative latent reasoning at inference (Table 4 and Fig. 6 of its paper). All runtimes are measured on the same NVIDIA A800 GPU; Sec. 5.3 of the submission incorrectly listed an A6000, which we have corrected.

Generalization. The original manuscript already covers unseen problem scales. TSP-18 lies outside the training range of 12–17 cities and is designated as out-of-distribution by the benchmark (He et al., 2025), although the released data-generation scripts of EndoCoT also include 18-city training instances. Under strict verification, Solver-JEPA nonetheless achieves 65.8% (AE) and 73.2% (U-Net), versus 63.3% for EndoCoT. VSP-7 and VSP-8 are also unseen grid sizes (training covers sizes 3–6, 16, and 32), on which both decoders reach 100% and 99% (Table 1). Following your suggestion, we further extend the shifts in scale and difficulty to TSP with 19 and 20 cities and Sudoku with 28 and 25 clues (30–45 in training), and add an unseen maze geometry, Maze-32 with a loop off the solution path (training mazes are loop-free). We evaluate the same checkpoints on these five settings without additional training. Each setting uses 100 instances generated with the official scripts, shared by all methods and scored with the strict verifier used in Table 2 of the original manuscript. EndoCoT solves no TSP-19/20 instance with its default setting, so we report its results with a larger latent-reasoning budget (Appendix D.1).

ModelMaze-32 w/ loopTSP-19TSP-20Sudoku-28Sudoku-25
DiffThinker374034151
DiffThinker++52362452
EndoCoT644434121
Ours (AE)5160564012
Ours (U-Net)627368439

Beyond the training range of problem size and difficulty, both of our decoders clearly outperform all baselines on TSP-19/20 and Sudoku-28, and they remain ahead on the hardest setting, Sudoku-25, where all baselines nearly fail. Under the unseen maze geometry, our U-Net decoder performs close to EndoCoT and outperforms both DiffThinker variants. We have added this experiment to Sec. 5.2 (Table 4) of the revised manuscript, and Appendix D.1 details its evaluation protocol for all methods.

We hope these clarifications and additional results address your concerns, and we would be grateful if you would reconsider your rating. We would be glad to discuss any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their time and constructive feedback. We have revised the manuscript based on the reviewers' comments. All changes to the main text are marked in blue. They mainly consist of new experiments, a clarification of how Solver-JEPA differs from I-JEPA (Sec. 4.2), a limitations paragraph, and corrections of errors in the submission, such as the missing citation on line 48. The appendix is not marked in blue because it has been largely rewritten: Appendices A–F are expanded, and the new Appendix G documents the task-specific settings of all experiments.

We summarize the new and updated tables below:

  • Table 1 (Sec. 5.2, to Reviewer imz1) adds zero-shot Nano Banana 2 and the backbone of each baseline, showing that both of our decoders match or exceed Nano Banana 2 on all 18 settings (93.6% and 93.7% vs. 39.4% on average) and that the baselines already include task-specific fine-tunes of Qwen-Image-Edit.
  • Table 3(b) (Sec. 5.4, to Reviewer Kkm9) adds a leave-one-out study of the five query families on TSP, showing that WITNESS is essential, VALUE and MEMBERSHIP add 3–4 points in optimal-tour rate, and removing RELATION or FEASIBILITY individually changes it by at most 1.33 points.
  • Table 4 (Sec. 5.2, to Reviewer imz1) evaluates the three task-specific generative baselines and both of our decoders on five new out-of-distribution settings (TSP-19/20, Sudoku-28/25, and Maze-32 with a loop), showing that both of our decoders outperform all baselines on four of them.
  • Table 5 (Sec. 5.4, to Reviewer Kkm9) compares our encoder with frozen CLIP ViT-B/16 and DINOv2 ViT-B/14 under the same AE decoder, showing that our 15.1M-parameter encoder outperforms them by 57.8 and 45.3 points on average.
  • Appendices A–G (to Reviewer x8DG) document the query schedules, commitment distributions, target encodings, architectures, loss weights, and auxiliary losses of each task, as well as the protocols of the out-of-distribution evaluation (Appendix D.1) and the pretrained-encoder baselines (Appendix F.3). Appendix E also reports the training cost of each query family (to Reviewer Kkm9).

We respond to each reviewer's comments individually below and would be glad to discuss any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.