acceptodds
Under review as a conference paper at ICLR 2027

Repair or Relocation? When Fixing a Composition Breaks Its Parts

Abstract

Correcting a model's prediction for a combined change can break its previously correct predictions for the constituent changes. Endpoint accuracy alone cannot distinguish these outcomes. We evaluate repair on controlled geometric quartets by tracking constituent preservation on fixed baseline failures, alongside joint successes gained and lost across the full population. On a fresh bank of 2,348 quartets from 873 parent scenes, 76.6–83.4% of raw-coordinate endpoint corrections introduced constituent errors across three seeds. Joint correctness nevertheless improved by 2.9–6.0 percentage points, showing that incomplete repair can coexist with positive net improvement. Geometric inputs increased constituent-preserving repair by 27.4–33.8 points relative to raw inputs. Focal Distillation reduced regression but also suppressed useful corrections under the tested recipe. Rendered-image experiments showed positive joint gains whose magnitude depended on the training recipe. Repair should therefore be assessed through both constituent preservation and the net change in joint correctness.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
compositional generalization, negative flips, representation learning, relational representations, structured prediction, model repair

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_unRWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates model repair in compositional reasoning tasks. It observes that updating a model to correct a prediction for a combined change can inadvertently break previously correct predictions for its constituent changes, a phenomenon the paper characterizes as error relocation. The main contribution is an instance-level evaluation framework that distinguishes constituent-preserving repair from error relocation, capturing behaviors that endpoint accuracy alone can conceal. The framework is evaluated primarily on a synthetic 2D segment-intersection task using both coordinate and rendered-image inputs. The experiments show that constituent regression can occur frequently even when an update improves joint correctness overall, and further examine how representation, change supervision, and regression-control objectives affect this repair–preservation trade-off.

Critical strengths
  1. An intuitive distinction between endpoint correction and constituent-preserving repair.

The paper makes a neat observation: fixing an endpoint error does not necessarily mean the model preserves the correct intermediate parts. Tracking this at the instance level separates true repair from mere error relocation, something aggregate accuracy simply misses. I found this evaluation angle intuitive and clean.

  1. Well-crafted figures.

The figures are clean, informative, and visually well put together, making the setup and key takeaways easy to grasp at a glance.

Critical weaknesses
  1. Mismatch between the broad claims and the narrow empirical scope.

To be clear, presentation issues alone would not lead me to reject the paper. My main concern is the tension between the broad claims and the narrow evidence. The paper frames repair versus relocation as a general evaluation principle for model updates, but virtually all evidence comes from a synthetic segment-intersection toy problem.

Even the image-based experiments simply re-render this same geometric task. The authors concede that "all tasks concern controlled geometry" and acknowledge that cross-renderer and cross-semantic-task generalization remains unestablished. For an evaluation framework, demonstrating applicability across different model families or realistic domains is essential. Without evidence beyond this toy setting, it is difficult to tell whether this phenomenon matters broadly or is an artifact of the setup.

  1. The presentation is overly defensive and fragmented, burying the main story.

Instead of explaining why the central finding matters, the paper gets bogged down in secondary controls and qualifications. The writing feels less like a confident scientific paper and more like hedging against every critique. Almost every result is immediately undercut by a disclaimer:

  • "This analysis did not establish…"
  • "Residual imbalance … limit its scope."

In the alternative explanations section, for example, the text introduces matched endpoint difficulty only to immediately walk back its interpretation; the next paragraph repeats this by stating that same-instance association cannot prove a mechanism. This reads like a disjointed series of disclaimers rather than a clear narrative. Moving these checks to the appendix would allow the main text to focus on the core evaluation idea.

Questions for authors
  1. Core takeaway: What is the main claim you want the reader to take away? Is it simply that constituent regression can happen, that it happens routinely under common fine-tuning methods, or that these metrics should become standard practice for model updates? Clarifying this would help calibrate expectations for the experimental section.

  2. Narrative flow: The idea is good, but the story gets lost in the writing. I would suggest looking at accepted papers, say, at ICLR, to see how they structure their narrative—especially in the introduction and experiments, where the problem is framed and the central finding is made clear right away. With that in mind, how would you rewrite the introduction to better convey this story and highlight the central finding?

Reviewer_v23LOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper illustrates that marginal predictions do not necessarily predict joint outcomes using the domain of a parent scene, two single edits that leave the label unchanged, and a combined edit that flips the label.

Critical strengths
  1. Use of separate development and confirmation bank.
  2. Careful statistical analysis.
Critical weaknesses
  1. That it can be case that P(A|B)>P(A) and P(A|C)>P(A) but P(A|B&C)<P(A) is because evidence doesn’t aggregate. This is a fundamental observation about probability that goes back many decades (eg. Carnap).
  2. The evidence comes from a toy task whose baseline models are near chance on the evaluated cases.
Questions for authors

I fear I am missing something about the significance of the work. Could you explain in the paper why this isn't a result of basic probability theory?

Reviewer_CU5bOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors evaluate whether a model can correct its prediction for a combined change while keeping its correct predictions for the constituent changes. They use a segment-intersection task and compare predictions on the same geometric quartets before and after a model update. They distinguish endpoint correction from full repair and track changes in joint correctness. The raw-coordinate experiments show that joint correctness can improve even with frequent constituent regression. Geometric features improve full repair. They also test Focal Distillation and use rendered images.

Critical strengths
  1. The paper checks whether fixing the prediction for a combined change also keeps the predictions for the individual changes correct. This is useful for understanding what a model update improves and what it makes worse.
  2. The authors check the main findings again on newly generated scenes. This makes the results more convincing within the tested task.
Critical weaknesses
  1. All experiments use the same simple segment-intersection task, even when the input is an image. The task can be solved exactly using geometry. This is useful for a controlled study, but it is unclear whether the findings also apply to more complex compositional problems.
  2. The evaluation measures incomplete repairs carefully, but gives limited insight into why they happen. It also remains unclear whether combining changes creates an additional difficulty beyond the general problem of model updates making previously correct predictions wrong. This limits what we learn about compositional generalization.
Questions for authors

In the larger rendered-image experiment, when the combined prediction is corrected, how often do previously correct constituent predictions become wrong? The baseline is much stronger there, so this would help show whether the issue remains common with a better model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.