- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies representation learning for visual problem solving, where images encode constrained planning, optimization, and satisfaction problems.
The so-called Solver-JEPA trains a visual encoder to predict solver-derived consequences of partial commitments, then trains frozen-representation decoders to render solutions.
The paper evaluates Solver-JEPA on Maze, TSP, Sudoku, and FrozenLake/VSP tasks.
Critical strengths
The paper proposes a concrete way to move solver information into representation learning rather than using it only to supervise an end-to-end solution generator. The improvements also show up in the actual task results. Solver-JEPA gets better results on Maze-32, TSP-18, and Sudoku-30. The TSP ablations are also useful: they provide some evidence that both embedding alignment and composite commitments help the frozen representation retain information useful for constructing complete solutions.
Critical weaknesses
I have a serious concern about reproducibility. The method is described at a high level, but key task-specific details are missing. The appendix mentions task-specific query schedules, loss weights, architectural choices, and auxiliary losses, but does not actually spell them out in enough detail. This is particularly problematic for a representation learning paper, where these training choices are central to the method and can have a significant impact on the results.
On the writing and presentation side, there are quite a few half-empty lines, and the appendix is very short and lacks detail. These issues do not really affect my assessment of the paper, but I think the authors should give them some attention to make the submission more polished and complete.
Questions for authors
For each task, what are the exact query-type proportions, commitment-size distributions, relation/witness target encodings, and loss weights used during representation training? Appendix A and Appendix E defer these details to task configurations that are not included.
4 comments
Sudoku (Appendix G.3). A commitment has 1–4 decisions on empty cells, each initially assigning (70%) or banning (30%) a digit. Each image caches one family per size – with all subsets (30 subset records); with probability 0.65, one decision is replaced by one conflicting with the unique solution, making the full commitment infeasible. Each step supervises all query types on all subsets of one family per image, with rotating across epochs. With no cost to optimize, VALUE is the normalized excess-candidate burden after 0, 1, and 4 propagation rounds (1 upon contradiction), RELATION encodes these candidate digits plus a contradiction flag, and WITNESS is the completed board, which MEMBERSHIP contrasts with boards that swap two digits globally or change one cell. All weights are 1.0, except 0.2 for the target-side semantic and interaction terms.
Architecture and auxiliary losses. TSP, Maze, and FrozenLake use the encoder architecture of Appendix B. The Sudoku encoder, now also detailed there, processes the 81 cell crops on the fixed exact-cover graph of the Sudoku rules into a 1024×384 state of 729 assignment, 81 cell, and 214 constraint-summary tokens. Appendices B, F, and G give the remaining details: the auxiliary terms of (e.g., tour ranking and degree terms for TSP, Bellman consistency for Maze), predictor and target-encoder sizes, decoder architectures and image losses, and sampling settings.
W2: Presentation issues.
Thank you for highlighting these issues. We have expanded the appendix as described above in the revised manuscript and will also tighten the layout of the main text in the final version.
We hope these clarifications and the expanded appendix address your concerns, and we would be glad to discuss any remaining questions.
This rebuttal addresses most of my concerns. I will keep my score.
Thank you for your feedback and for your continued support. We are glad that the rebuttal addressed most of your concerns. If any specific point remains unclear, we would be happy to clarify it or to add the corresponding details to the final version.
We are grateful for your thoughtful review and recognition of our use of solver information in representation learning, performance gains in challenging settings, and informative TSP ablations. The items you listed for representation training, from query schedules and commitment-size distributions to target encodings, architectures, and loss weights, are now documented in the revised Appendices A–G, as summarized in W1/Q1 below. The revision also extends the TSP ablations with a leave-one-out study of the five query families (Table 3), and adds comparisons with zero-shot Nano Banana 2 (Table 1) and frozen CLIP and DINOv2 encoders (Table 5), as well as evaluations on five out-of-distribution settings (Table 4).
Thank you for raising this point. We agree that these choices are central to a representation-learning paper. The revised manuscript expands Appendices A–F and adds Appendix G, which together document the task-specific settings used in our experiments, and we will release the code, task configurations, and data-generation scripts upon publication. Appendices A and E no longer defer to unreleased task configurations: Appendix A now points to the per-task specifications in Appendix G, and Appendix E lists the optimization settings of every task. We summarize the answers to your question below.
Shared setup (Appendices C–E). Labels are computed offline with exact solvers and deterministic procedures (e.g., enumeration or MILP with subtour cuts for TSP, BFS for Maze). Representations are trained for 100 epochs with AdamW (learning rate 1e-4, weight decay 0.05, 4-epoch warmup and cosine decay, gradient clipping 1.0, bf16; Sudoku exceptions in Appendix E) at a global batch of 32 on four A800 GPUs. The objective is L=Lbase+λqLquery (λq=0.25; 0.5 for Sudoku), where Lbase supervises the original problem and Lquery supervises commitment-conditioned queries, including the empty commitment. In Lbase, the alignment and predicted-side semantic losses have weight 1.0, the target-side semantic loss 0.2, and the variance regularizer 0.02 (1.0 and 0.05, respectively, for Maze and FrozenLake), and Sudoku adds a covariance term (0.01). Query types are supervised jointly rather than sampled exclusively. For infeasible commitments, WITNESS and the cost-based VALUE targets are masked (the Sudoku burden remains defined), and every loss is averaged over its valid entries (Appendix C.1).
TSP (Appendix G.1). A commitment has 1–4 decisions, each including or excluding an undirected edge or requiring a directed successor. Each image caches all 2n(n−1) single-decision commitments, 24 random commitments of size 2–4 (uniform), and three rule-based infeasible ones, with all their subsets. Each step supervises all cached FEASIBILITY, VALUE, and MEMBERSHIP labels (the latter for up to 32 commitments) and one WITNESS from the empty, single-, or multi-decision pool, in a four-epoch 1:1:2 rotation. RELATION encodes pairwise city distances, VALUE the relative increase in optimal tour length, and WITNESS the constrained-optimal tour as a directed successor matrix, which MEMBERSHIP contrasts with a feasible but longer tour. The weights are 1.0 for VALUE and WITNESS, 0.25 for each of two interaction terms (non-additive effects on VALUE), 0.2 for FEASIBILITY and MEMBERSHIP, and 0.1 for RELATION.
Maze and FrozenLake (Appendix G.2). A commitment blocks 1–3 cells or passages. Each image caches 32 random single blocks and 16 random commitments of size 2 or 3 (equally likely), with all their subsets; each step uniformly draws two per image and supervises all query types on both. RELATION encodes the remaining valid moves, reachability, and distance to the goal; VALUE the relative increase in path length (zero on Maze whenever a path remains, since tree mazes have a unique path); and WITNESS the shortest path under the commitment as a cell mask with move or STOP labels, which MEMBERSHIP contrasts with a truncated or reversed original path. All Lquery weights are 1.0, except 0.2 for the target-side semantic term.